2024-05-08T05:27:48.593 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T05:27:48.593 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/fsx}} --name vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697245 --verbose --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697245/orig.config.yaml 2024-05-08T05:27:48.649 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T05:27:48.740 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697245 branch: wip-57676-3 description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/fsx}} email: vshankar@redhat.com first_in_suite: false job_id: '7697245' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-57676-3 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-57676-3 sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a owner: scheduled_vshankar@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5998 sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a sleep_before_teardown: 0 subset: 915/1024 suite: fs:workload suite_branch: wip-57676-3 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHfa5ZtESuaGYbIXHP9LRF+N5NQIT2zk57fWJzH4FtpHbaJ5Eh4BFt0PixB6DDSSPfSDiDQGQaEuP30RNPEp5kA= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEPzCWNuTxfxrSwl9Tw6rzU6xVwEoAZIMOTLJIFwIgpSwpdopiXdgyKXthRLcb9TRKOXAOnjpGcM4SHIWVvRFW0= smithi162.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC5i28NT/U0fyaDx5hyDwH2KwHqHoc96dPf2D2awbKKhdUYLQ/wC6xWRxCN08ciPQGsPrlXb5TBKCgWeMDE1Y0g= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_04:00:39 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T05:27:48.740 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa; will attempt to use it 2024-05-08T05:27:48.741 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa/tasks 2024-05-08T05:27:48.741 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T05:27:48.742 INFO:teuthology.task.internal:Checking packages... 2024-05-08T05:27:48.765 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a' 2024-05-08T05:27:48.765 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T05:27:48.765 INFO:teuthology.packaging:ref: None 2024-05-08T05:27:48.765 INFO:teuthology.packaging:tag: None 2024-05-08T05:27:48.765 INFO:teuthology.packaging:branch: wip-57676-3 2024-05-08T05:27:48.765 INFO:teuthology.packaging:sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:27:48.765 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-57676-3 2024-05-08T05:27:48.909 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3538.gbee299f4 2024-05-08T05:27:48.911 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T05:27:48.918 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T05:27:48.918 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T05:27:48.989 INFO:teuthology.task.internal:Saving configuration 2024-05-08T05:27:49.010 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T05:27:49.016 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T05:27:49.043 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697245', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:22:44.062819', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHfa5ZtESuaGYbIXHP9LRF+N5NQIT2zk57fWJzH4FtpHbaJ5Eh4BFt0PixB6DDSSPfSDiDQGQaEuP30RNPEp5kA='} 2024-05-08T05:27:49.069 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697245', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:22:44.064511', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEPzCWNuTxfxrSwl9Tw6rzU6xVwEoAZIMOTLJIFwIgpSwpdopiXdgyKXthRLcb9TRKOXAOnjpGcM4SHIWVvRFW0='} 2024-05-08T05:27:49.089 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697245', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:22:44.061361', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC5i28NT/U0fyaDx5hyDwH2KwHqHoc96dPf2D2awbKKhdUYLQ/wC6xWRxCN08ciPQGsPrlXb5TBKCgWeMDE1Y0g='} 2024-05-08T05:27:49.089 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T05:27:49.096 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-08T05:27:49.096 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-08T05:27:49.096 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-08T05:27:49.096 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T05:27:49.242 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f24715f1f70>, signals=[15]) 2024-05-08T05:27:49.242 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T05:27:49.248 INFO:teuthology.task.internal:Opening connections... 2024-05-08T05:27:49.248 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-05-08T05:27:49.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:27:49.337 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-05-08T05:27:49.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:27:49.411 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi162.front.sepia.ceph.com 2024-05-08T05:27:49.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:27:49.486 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T05:27:49.493 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-05-08T05:27:49.513 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-05-08T05:27:49.513 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-05-08T05:27:49.569 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-05-08T05:27:49.569 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-05-08T05:27:49.569 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-05-08T05:27:49.570 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-05-08T05:27:49.570 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-05-08T05:27:49.570 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-05-08T05:27:49.570 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T05:27:49.570 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-05-08T05:27:49.570 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-05-08T05:27:49.570 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T05:27:49.570 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-05-08T05:27:49.570 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T05:27:49.570 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T05:27:49.570 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T05:27:49.571 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-05-08T05:27:49.591 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-05-08T05:27:49.609 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-05-08T05:27:49.609 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-05-08T05:27:49.666 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-05-08T05:27:49.666 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-05-08T05:27:49.666 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-05-08T05:27:49.666 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-05-08T05:27:49.666 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-05-08T05:27:49.667 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-05-08T05:27:49.667 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T05:27:49.667 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-05-08T05:27:49.667 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-05-08T05:27:49.667 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T05:27:49.667 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-05-08T05:27:49.667 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T05:27:49.667 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T05:27:49.667 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T05:27:49.667 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-05-08T05:27:49.683 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2024-05-08T05:27:49.701 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2024-05-08T05:27:49.702 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2024-05-08T05:27:49.759 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream" 2024-05-08T05:27:49.759 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="9" 2024-05-08T05:27:49.760 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos" 2024-05-08T05:27:49.760 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora" 2024-05-08T05:27:49.760 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="9" 2024-05-08T05:27:49.760 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el9" 2024-05-08T05:27:49.760 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T05:27:49.760 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2024-05-08T05:27:49.760 INFO:teuthology.orchestra.run.smithi162.stdout:LOGO="fedora-logo-icon" 2024-05-08T05:27:49.760 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T05:27:49.760 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/" 2024-05-08T05:27:49.760 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T05:27:49.760 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T05:27:49.760 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T05:27:49.760 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2024-05-08T05:27:49.783 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T05:27:49.795 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T05:27:49.818 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T05:27:49.818 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-05-08T05:27:49.821 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-05-08T05:27:49.823 DEBUG:teuthology.orchestra.run.smithi162:> test '!' -e /home/ubuntu/cephtest 2024-05-08T05:27:49.839 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T05:27:49.887 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T05:27:49.887 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-05-08T05:27:49.890 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-05-08T05:27:49.892 DEBUG:teuthology.orchestra.run.smithi162:> test -z $(ls -A /var/lib/ceph) 2024-05-08T05:27:49.983 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T05:27:50.061 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T05:27:50.075 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-05-08T05:27:50.075 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-05-08T05:27:50.075 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-05-08T05:27:50.075 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T05:27:50.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:27:50.093 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-08T05:27:50.250 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T05:27:50.250 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-08T05:27:50.250 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-05-08T05:27:50.267 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-437.el9.x86_64 2024-05-08T05:27:50.268 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T05:27:50.268 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-08T05:27:50.268 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum upgrade -y linux-firmware 2024-05-08T05:27:54.370 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.1 MB 00:02 2024-05-08T05:27:56.571 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-05-08T05:28:02.497 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.2 MB 00:02 2024-05-08T05:28:04.422 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-05-08T05:28:07.874 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 6.4 MB/s | 21 MB 00:03 2024-05-08T05:28:12.990 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-08T05:28:15.056 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-05-08T05:28:15.057 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2024-05-08T05:28:15.057 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-05-08T05:28:15.156 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-08T05:28:15.157 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T05:28:15.376 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-05-08T05:28:15.376 DEBUG:teuthology.orchestra.run.smithi112:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-08T05:28:15.378 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi112.front.sepia.ceph.com to complete... 2024-05-08T05:28:40.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08 05:28:40 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-08T05:28:40.605 INFO:teuthology.orchestra.run.smithi112.stderr:FINISHED --2024-05-08 05:28:40-- 2024-05-08T05:28:40.605 INFO:teuthology.orchestra.run.smithi112.stderr:Total wall clock time: 25s 2024-05-08T05:28:40.605 INFO:teuthology.orchestra.run.smithi112.stderr:Downloaded: 1 files, 341M in 25s (13.7 MB/s) 2024-05-08T05:28:40.608 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-08T05:28:40.608 DEBUG:teuthology.orchestra.run.smithi112:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-08T05:28:42.693 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying... ######################################## 2024-05-08T05:28:42.991 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing... ######################################## 2024-05-08T05:28:43.099 INFO:teuthology.orchestra.run.smithi112.stdout:Updating / installing... 2024-05-08T05:28:51.034 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-05-08T05:29:27.866 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi112.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-08T05:29:27.866 DEBUG:teuthology.orchestra.run.smithi112:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-08T05:29:27.961 INFO:teuthology.orchestra.run.smithi112.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-08T05:29:27.961 INFO:teuthology.orchestra.run.smithi112.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-05-08T05:29:27.962 INFO:teuthology.orchestra.run.smithi112.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-08T05:29:27.962 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-05-08T05:29:27.962 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-05-08T05:29:27.962 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-05-08T05:29:27.962 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-05-08T05:29:27.962 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-05-08T05:29:27.962 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-05-08T05:29:27.962 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-08T05:29:27.962 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-08T05:29:27.962 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-08T05:29:27.962 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-08T05:29:27.962 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-08T05:29:27.963 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-08T05:29:27.963 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-08T05:29:27.963 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-08T05:29:27.963 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-08T05:29:27.963 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-08T05:29:27.963 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-08T05:29:27.963 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-08T05:29:27.963 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-08T05:29:27.963 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-08T05:29:27.963 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-08T05:29:27.963 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-08T05:29:27.963 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-08T05:29:27.963 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-08T05:29:27.963 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-08T05:29:27.964 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-08T05:29:27.964 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-05-08T05:29:27.964 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-05-08T05:29:27.964 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-05-08T05:29:27.964 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-08T05:29:27.964 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-05-08T05:29:27.964 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-08T05:29:27.964 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-08T05:29:27.964 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-08T05:29:27.964 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-05-08T05:29:27.964 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-08T05:29:27.964 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-08T05:29:27.964 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-08T05:29:27.965 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-05-08T05:29:27.965 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-08T05:29:27.965 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-08T05:29:27.965 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-08T05:29:27.965 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-08T05:29:27.965 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-08T05:29:27.965 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-08T05:29:27.965 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-08T05:29:27.965 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-08T05:29:27.965 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-08T05:29:27.965 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-08T05:29:27.965 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-08T05:29:27.965 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-08T05:29:27.966 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-08T05:29:27.966 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-08T05:29:27.966 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-08T05:29:27.966 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-08T05:29:27.966 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-08T05:29:27.966 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-08T05:29:27.966 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-08T05:29:27.966 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-08T05:29:27.966 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-08T05:29:27.966 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-08T05:29:27.966 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-08T05:29:27.966 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-08T05:29:27.966 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-08T05:29:27.967 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-08T05:29:27.967 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-08T05:29:27.967 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-08T05:29:27.967 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-08T05:29:27.967 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-08T05:29:27.967 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-08T05:29:27.967 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-08T05:29:27.967 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-08T05:29:27.967 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-08T05:29:27.967 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-08T05:29:27.967 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-08T05:29:27.967 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-08T05:29:27.967 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-08T05:29:27.968 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-08T05:29:27.968 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-08T05:29:27.968 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-08T05:29:27.968 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-08T05:29:27.968 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-08T05:29:27.968 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-08T05:29:27.968 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-08T05:29:27.968 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-05-08T05:29:27.968 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-05-08T05:29:27.968 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-08T05:29:27.968 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-08T05:29:27.968 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-08T05:29:27.968 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-08T05:29:27.969 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-08T05:29:27.969 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-08T05:29:27.969 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-08T05:29:27.969 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-05-08T05:29:27.969 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-08T05:29:27.969 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-08T05:29:27.969 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-05-08T05:29:27.969 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-08T05:29:27.969 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-08T05:29:27.969 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-08T05:29:27.969 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-08T05:29:27.969 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-08T05:29:27.970 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-08T05:29:27.970 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-08T05:29:27.970 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-08T05:29:27.970 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-08T05:29:27.970 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-08T05:29:27.970 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-08T05:29:27.970 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-08T05:29:27.970 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-08T05:29:27.970 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-08T05:29:27.970 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-08T05:29:27.970 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-08T05:29:27.970 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-08T05:29:27.971 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-08T05:29:27.971 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-08T05:29:27.971 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-08T05:29:27.971 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-08T05:29:27.971 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-08T05:29:27.971 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-08T05:29:27.971 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-08T05:29:27.971 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-08T05:29:27.971 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-08T05:29:27.971 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-08T05:29:27.971 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-08T05:29:27.971 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-08T05:29:27.971 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-08T05:29:27.972 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-08T05:29:27.972 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-08T05:29:27.972 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-08T05:29:27.972 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-08T05:29:27.972 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-08T05:29:27.972 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-05-08T05:29:27.972 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-08T05:29:27.972 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-08T05:29:27.972 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-08T05:29:27.972 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-08T05:29:27.972 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-08T05:29:27.972 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-08T05:29:27.972 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-08T05:29:27.972 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-08T05:29:27.973 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-08T05:29:27.973 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-08T05:29:27.973 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-08T05:29:27.973 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-08T05:29:27.973 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-08T05:29:27.973 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-08T05:29:27.973 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-08T05:29:27.973 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-08T05:29:27.973 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-08T05:29:27.973 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-05-08T05:29:27.973 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-08T05:29:27.973 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-05-08T05:29:27.973 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-05-08T05:29:27.973 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-08T05:29:27.974 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-08T05:29:27.974 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-05-08T05:29:27.974 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-08T05:29:27.974 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-05-08T05:29:27.974 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-05-08T05:29:27.974 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-08T05:29:27.974 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-05-08T05:29:27.974 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-05-08T05:29:27.974 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-08T05:29:27.974 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-08T05:29:27.974 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-05-08T05:29:27.974 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-05-08T05:29:27.975 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-08T05:29:27.975 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-08T05:29:27.975 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-05-08T05:29:27.975 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-08T05:29:27.975 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-08T05:29:27.975 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-08T05:29:27.975 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-05-08T05:29:27.975 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-05-08T05:29:27.975 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-08T05:29:27.975 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-05-08T05:29:27.975 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-05-08T05:29:27.975 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-08T05:29:27.975 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-05-08T05:29:27.976 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-08T05:29:27.976 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-05-08T05:29:27.976 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-08T05:29:27.976 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-08T05:29:27.976 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-08T05:29:27.976 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-08T05:29:27.976 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-08T05:29:27.976 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-05-08T05:29:27.976 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-08T05:29:27.976 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-08T05:29:27.976 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-08T05:29:27.976 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-08T05:29:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-08T05:29:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-08T05:29:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-08T05:29:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-05-08T05:29:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-08T05:29:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-08T05:29:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-08T05:29:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-05-08T05:29:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-05-08T05:29:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-08T05:29:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-08T05:29:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-08T05:29:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-08T05:29:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-08T05:29:27.977 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-05-08T05:29:27.978 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-05-08T05:29:27.978 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-05-08T05:29:27.978 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-08T05:29:27.978 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-08T05:29:27.978 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-08T05:29:27.978 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-08T05:29:27.978 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-08T05:29:27.978 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-08T05:29:27.978 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-05-08T05:29:27.978 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-05-08T05:29:27.978 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-08T05:29:27.978 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-08T05:29:27.978 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-05-08T05:29:27.979 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-05-08T05:29:27.979 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-05-08T05:29:27.979 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-08T05:29:27.979 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-08T05:29:27.979 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-08T05:29:27.979 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-08T05:29:27.979 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-08T05:29:27.979 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-05-08T05:29:27.979 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-08T05:29:27.979 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-05-08T05:29:27.979 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-08T05:29:27.979 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-05-08T05:29:27.979 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-08T05:29:27.979 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-05-08T05:29:27.979 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-05-08T05:29:27.980 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-08T05:29:27.980 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-05-08T05:29:27.980 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-08T05:29:27.980 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-05-08T05:29:27.980 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-08T05:29:27.980 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-08T05:29:27.980 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-08T05:29:27.980 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-08T05:29:27.980 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-08T05:29:27.980 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-08T05:29:27.980 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-08T05:29:27.980 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-08T05:29:27.980 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-08T05:29:27.980 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-08T05:29:27.980 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-08T05:29:27.981 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-08T05:29:27.981 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-08T05:29:27.981 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-05-08T05:29:27.981 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-05-08T05:29:27.981 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-08T05:29:27.981 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-05-08T05:29:27.981 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-08T05:29:27.981 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-08T05:29:27.981 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-05-08T05:29:27.981 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-08T05:29:27.981 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-08T05:29:27.981 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-08T05:29:27.981 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-08T05:29:27.981 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-08T05:29:27.981 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-08T05:29:27.982 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-08T05:29:27.982 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-08T05:29:27.982 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-08T05:29:27.982 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-08T05:29:27.982 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-08T05:29:27.982 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-08T05:29:27.982 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-08T05:29:27.982 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-08T05:29:27.982 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-08T05:29:27.982 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-08T05:29:27.982 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-08T05:29:27.982 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-08T05:29:27.982 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-08T05:29:27.983 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-08T05:29:27.983 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-08T05:29:27.983 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-08T05:29:27.983 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-08T05:29:27.983 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-08T05:29:27.983 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-08T05:29:27.983 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-08T05:29:27.983 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-08T05:29:27.983 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-08T05:29:27.983 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-08T05:29:27.983 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-08T05:29:27.983 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-08T05:29:27.983 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-08T05:29:27.983 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-08T05:29:27.983 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-08T05:29:27.984 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-08T05:29:27.984 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-08T05:29:27.984 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-08T05:29:27.984 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-08T05:29:27.984 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-08T05:29:27.984 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-08T05:29:27.984 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-08T05:29:27.984 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-08T05:29:27.984 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-08T05:29:27.984 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-08T05:29:27.984 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-08T05:29:27.984 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-08T05:29:27.984 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-08T05:29:27.984 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-08T05:29:27.985 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-08T05:29:27.985 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-08T05:29:27.985 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-08T05:29:27.985 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-08T05:29:27.985 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-08T05:29:27.985 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-08T05:29:27.985 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-08T05:29:27.985 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-08T05:29:27.985 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-08T05:29:27.985 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-08T05:29:27.985 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-08T05:29:27.985 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-08T05:29:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-08T05:29:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-08T05:29:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-08T05:29:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-08T05:29:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-05-08T05:29:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-08T05:29:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-08T05:29:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-08T05:29:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-05-08T05:29:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-08T05:29:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-08T05:29:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-08T05:29:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-08T05:29:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-08T05:29:27.987 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-08T05:29:27.987 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-08T05:29:27.987 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-08T05:29:27.987 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-08T05:29:27.987 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-08T05:29:27.987 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-08T05:29:27.987 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-08T05:29:27.987 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-08T05:29:27.987 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-08T05:29:27.987 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-08T05:29:27.987 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-08T05:29:27.987 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-08T05:29:27.988 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-08T05:29:27.988 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-08T05:29:27.988 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-08T05:29:27.988 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-08T05:29:27.988 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-08T05:29:27.988 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-08T05:29:27.988 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-08T05:29:27.988 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-08T05:29:27.988 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-08T05:29:27.988 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-08T05:29:27.988 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-08T05:29:27.988 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-08T05:29:27.988 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-08T05:29:27.988 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-08T05:29:27.989 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-08T05:29:27.989 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-08T05:29:27.989 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-08T05:29:27.989 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-08T05:29:27.989 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-08T05:29:27.989 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-08T05:29:27.989 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-08T05:29:27.989 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-08T05:29:27.989 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-08T05:29:27.989 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-08T05:29:27.989 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-08T05:29:27.989 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-08T05:29:27.989 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-08T05:29:27.989 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-08T05:29:27.989 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-08T05:29:27.990 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-08T05:29:27.990 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-08T05:29:27.990 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-08T05:29:27.990 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-08T05:29:27.990 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-08T05:29:27.990 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-08T05:29:27.990 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-08T05:29:27.990 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-08T05:29:27.990 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-08T05:29:27.990 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-08T05:29:27.990 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-08T05:29:27.990 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-08T05:29:27.990 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-08T05:29:27.990 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-08T05:29:27.991 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-08T05:29:27.991 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-08T05:29:27.991 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-08T05:29:27.991 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-08T05:29:27.991 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-08T05:29:27.991 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-08T05:29:27.991 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-08T05:29:27.991 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-08T05:29:27.991 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-08T05:29:27.991 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-08T05:29:27.991 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-08T05:29:27.991 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-08T05:29:27.991 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-08T05:29:27.991 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-08T05:29:27.992 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-08T05:29:27.992 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-08T05:29:27.992 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-08T05:29:27.992 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-08T05:29:27.992 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-08T05:29:27.992 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-08T05:29:27.992 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-08T05:29:27.992 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-08T05:29:27.992 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-08T05:29:27.992 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-08T05:29:27.992 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-08T05:29:27.992 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-08T05:29:27.992 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-08T05:29:27.992 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-08T05:29:27.993 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-05-08T05:29:27.993 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-08T05:29:27.993 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-08T05:29:27.993 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-08T05:29:27.993 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-08T05:29:27.993 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-08T05:29:27.993 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-08T05:29:27.993 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-08T05:29:27.993 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-08T05:29:27.993 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-08T05:29:27.993 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-08T05:29:27.993 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-08T05:29:27.993 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-08T05:29:27.994 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-08T05:29:27.994 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-08T05:29:27.994 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-08T05:29:27.994 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-08T05:29:27.994 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-08T05:29:27.994 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-08T05:29:27.994 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-08T05:29:27.994 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-08T05:29:27.994 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-08T05:29:27.994 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-08T05:29:27.994 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-08T05:29:27.994 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-08T05:29:27.994 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-08T05:29:27.994 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-08T05:29:27.995 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-08T05:29:27.995 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-08T05:29:27.995 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-08T05:29:27.995 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-08T05:29:27.995 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-08T05:29:27.995 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-08T05:29:27.995 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-08T05:29:27.995 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-08T05:29:27.995 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-08T05:29:27.995 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-08T05:29:27.995 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-08T05:29:27.995 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-08T05:29:27.995 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-08T05:29:27.995 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-08T05:29:27.996 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-08T05:29:27.996 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-05-08T05:29:27.996 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-05-08T05:29:27.996 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-08T05:29:27.996 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-08T05:29:27.996 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-05-08T05:29:27.996 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-08T05:29:27.996 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-08T05:29:27.996 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-08T05:29:27.996 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-08T05:29:27.996 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-08T05:29:27.996 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-08T05:29:27.996 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-08T05:29:27.997 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-08T05:29:27.997 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-08T05:29:27.997 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-08T05:29:27.997 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-08T05:29:27.997 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-08T05:29:27.997 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-08T05:29:27.997 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-08T05:29:27.997 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-08T05:29:27.997 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-08T05:29:27.997 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-08T05:29:27.997 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-08T05:29:27.997 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-08T05:29:27.997 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-08T05:29:27.997 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-08T05:29:27.998 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-08T05:29:27.998 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-08T05:29:27.998 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-05-08T05:29:27.998 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-08T05:29:27.998 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-08T05:29:27.998 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-08T05:29:27.998 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-08T05:29:27.998 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-05-08T05:29:27.998 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-05-08T05:29:27.998 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-08T05:29:27.998 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-08T05:29:27.998 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-08T05:29:27.998 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-08T05:29:27.999 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-08T05:29:27.999 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-08T05:29:27.999 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-08T05:29:27.999 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-05-08T05:29:27.999 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-05-08T05:29:27.999 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-08T05:29:27.999 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-05-08T05:29:27.999 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-08T05:29:27.999 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-05-08T05:29:27.999 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-08T05:29:27.999 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-05-08T05:29:27.999 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-08T05:29:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-05-08T05:29:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-05-08T05:29:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-08T05:29:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-05-08T05:29:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-08T05:29:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-05-08T05:29:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-08T05:29:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-05-08T05:29:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-08T05:29:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-05-08T05:29:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-08T05:29:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-05-08T05:29:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-05-08T05:29:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-08T05:29:28.000 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-08T05:29:28.001 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-08T05:29:28.001 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-08T05:29:28.001 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-05-08T05:29:28.001 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-08T05:29:28.001 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-05-08T05:29:28.001 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-08T05:29:28.001 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-08T05:29:28.001 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-08T05:29:28.001 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-08T05:29:28.001 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-08T05:29:28.001 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-08T05:29:28.001 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-08T05:29:28.002 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-08T05:29:28.002 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-05-08T05:29:28.002 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-08T05:29:28.002 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-05-08T05:29:28.002 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-08T05:29:28.002 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-08T05:29:28.002 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-08T05:29:28.002 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-08T05:29:28.002 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-08T05:29:28.002 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-05-08T05:29:28.002 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-08T05:29:28.002 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-08T05:29:28.002 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-08T05:29:28.002 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-08T05:29:28.003 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-05-08T05:29:28.003 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-08T05:29:28.003 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-08T05:29:28.003 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-08T05:29:28.003 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-08T05:29:28.003 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-08T05:29:28.003 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-08T05:29:28.003 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-08T05:29:28.003 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-08T05:29:28.003 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-08T05:29:28.003 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-05-08T05:29:28.003 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-05-08T05:29:28.003 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-08T05:29:28.004 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-08T05:29:28.004 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-08T05:29:28.004 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-08T05:29:28.004 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-08T05:29:28.004 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-08T05:29:28.004 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-08T05:29:28.004 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-08T05:29:28.004 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-08T05:29:28.004 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-08T05:29:28.004 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-08T05:29:28.004 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-08T05:29:28.004 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-08T05:29:28.004 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-08T05:29:28.004 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-08T05:29:28.004 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-08T05:29:28.005 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-05-08T05:29:28.005 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-08T05:29:28.005 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-08T05:29:28.005 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-08T05:29:28.005 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-08T05:29:28.005 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-08T05:29:28.005 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-05-08T05:29:28.005 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-05-08T05:29:28.005 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-08T05:29:28.005 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-08T05:29:28.005 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-08T05:29:28.005 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-08T05:29:28.005 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-08T05:29:28.005 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-08T05:29:28.006 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-08T05:29:28.006 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-05-08T05:29:28.006 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-08T05:29:28.006 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-08T05:29:28.006 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-05-08T05:29:28.006 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-08T05:29:28.006 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-08T05:29:28.006 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-08T05:29:28.006 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-08T05:29:28.006 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-05-08T05:29:28.007 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-05-08T05:29:28.007 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-08T05:29:28.007 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-08T05:29:28.007 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-08T05:29:28.007 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-08T05:29:28.007 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-05-08T05:29:28.007 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-08T05:29:28.007 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-05-08T05:29:28.007 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-08T05:29:28.007 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-08T05:29:28.007 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-08T05:29:28.007 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-08T05:29:28.007 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-08T05:29:28.007 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-08T05:29:28.008 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-08T05:29:28.008 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-05-08T05:29:28.008 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-08T05:29:28.008 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-08T05:29:28.008 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-05-08T05:29:28.008 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-08T05:29:28.008 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-08T05:29:28.008 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-08T05:29:28.008 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-05-08T05:29:28.008 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-08T05:29:28.008 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-05-08T05:29:28.008 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-08T05:29:28.008 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-08T05:29:28.008 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-08T05:29:28.008 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-08T05:29:28.009 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-08T05:29:28.009 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-08T05:29:28.009 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-08T05:29:28.009 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-05-08T05:29:28.009 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-08T05:29:28.009 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-05-08T05:29:28.009 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-05-08T05:29:28.009 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-08T05:29:28.009 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-05-08T05:29:28.009 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-05-08T05:29:28.009 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-08T05:29:28.009 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-08T05:29:28.009 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-08T05:29:28.009 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-05-08T05:29:28.010 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-08T05:29:28.010 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-08T05:29:28.010 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-08T05:29:28.010 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-05-08T05:29:28.010 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-05-08T05:29:28.010 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-08T05:29:28.010 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-08T05:29:28.010 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-08T05:29:28.010 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-08T05:29:28.010 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-05-08T05:29:28.010 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-08T05:29:28.010 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-08T05:29:28.010 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-08T05:29:28.011 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-08T05:29:28.011 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-05-08T05:29:28.011 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-08T05:29:28.011 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-05-08T05:29:28.011 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-08T05:29:28.011 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-05-08T05:29:28.011 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-08T05:29:28.011 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-05-08T05:29:28.011 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-08T05:29:28.011 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-05-08T05:29:28.011 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-08T05:29:28.011 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-08T05:29:28.011 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-08T05:29:28.011 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-05-08T05:29:28.011 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-05-08T05:29:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-08T05:29:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-08T05:29:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-08T05:29:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-08T05:29:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-08T05:29:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-05-08T05:29:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-08T05:29:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-08T05:29:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-05-08T05:29:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-08T05:29:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-08T05:29:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-08T05:29:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-08T05:29:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-08T05:29:28.012 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-05-08T05:29:28.013 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-08T05:29:28.013 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-08T05:29:28.013 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-05-08T05:29:28.013 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-05-08T05:29:28.013 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-08T05:29:28.013 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-05-08T05:29:28.013 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-08T05:29:28.013 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-08T05:29:28.013 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-08T05:29:28.013 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-08T05:29:28.013 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-08T05:29:28.013 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-08T05:29:28.014 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-08T05:29:28.014 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-08T05:29:28.014 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-08T05:29:28.014 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-08T05:29:28.014 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-05-08T05:29:28.014 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-05-08T05:29:28.014 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-08T05:29:28.014 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-05-08T05:29:28.014 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-05-08T05:29:28.014 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-08T05:29:28.014 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-08T05:29:28.014 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-08T05:29:28.015 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-08T05:29:28.015 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-08T05:29:28.015 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-08T05:29:28.015 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-05-08T05:29:28.015 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-08T05:29:28.015 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-08T05:29:28.015 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-08T05:29:28.015 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-05-08T05:29:28.015 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-05-08T05:29:28.015 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-08T05:29:28.015 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-08T05:29:28.015 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-08T05:29:28.015 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-05-08T05:29:28.015 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-08T05:29:28.016 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-05-08T05:29:28.016 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-08T05:29:28.016 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-05-08T05:29:28.016 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-08T05:29:28.016 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-05-08T05:29:28.016 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-08T05:29:28.016 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-05-08T05:29:28.016 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-08T05:29:28.016 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-05-08T05:29:28.016 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-08T05:29:28.016 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-05-08T05:29:28.016 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-08T05:29:28.016 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-05-08T05:29:28.016 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-08T05:29:28.016 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-08T05:29:28.017 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-08T05:29:28.017 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-08T05:29:28.017 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-08T05:29:28.017 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-08T05:29:28.017 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-05-08T05:29:28.017 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-08T05:29:28.017 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-08T05:29:28.017 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-08T05:29:28.017 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-05-08T05:29:28.017 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-08T05:29:28.017 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-08T05:29:28.017 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-08T05:29:28.017 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-08T05:29:28.017 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-08T05:29:28.018 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-08T05:29:28.018 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-05-08T05:29:28.018 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-08T05:29:28.018 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-05-08T05:29:28.018 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-08T05:29:28.018 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-08T05:29:28.018 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-05-08T05:29:28.018 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-08T05:29:28.018 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-08T05:29:28.018 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-05-08T05:29:28.018 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-08T05:29:28.018 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-05-08T05:29:28.018 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-08T05:29:28.018 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-05-08T05:29:28.019 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-08T05:29:28.019 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-08T05:29:28.019 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-05-08T05:29:28.019 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-08T05:29:28.019 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-08T05:29:28.019 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-08T05:29:28.019 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-08T05:29:28.019 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-08T05:29:28.019 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-05-08T05:29:28.019 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-08T05:29:28.019 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-05-08T05:29:28.019 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-08T05:29:28.019 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-08T05:29:28.019 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-08T05:29:28.020 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-08T05:29:28.020 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-05-08T05:29:28.020 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-08T05:29:28.020 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-05-08T05:29:28.020 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-08T05:29:28.020 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-05-08T05:29:28.020 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-08T05:29:28.020 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-05-08T05:29:28.020 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-08T05:29:28.020 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-08T05:29:28.020 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-05-08T05:29:28.020 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-08T05:29:28.020 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-08T05:29:28.020 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-08T05:29:28.021 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-08T05:29:28.021 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-05-08T05:29:28.021 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-08T05:29:28.021 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-08T05:29:28.021 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-08T05:29:28.021 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-05-08T05:29:28.021 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-08T05:29:28.021 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-08T05:29:28.021 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-08T05:29:28.021 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-08T05:29:28.021 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-05-08T05:29:28.021 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-08T05:29:28.021 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-05-08T05:29:28.021 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-08T05:29:28.021 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-05-08T05:29:28.022 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-08T05:29:28.022 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-08T05:29:28.022 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-05-08T05:29:28.022 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-08T05:29:28.022 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-08T05:29:28.022 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-05-08T05:29:28.022 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-08T05:29:28.022 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-08T05:29:28.022 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-05-08T05:29:28.022 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-08T05:29:28.022 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-05-08T05:29:28.022 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-08T05:29:28.022 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-08T05:29:28.022 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-05-08T05:29:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-08T05:29:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-08T05:29:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-08T05:29:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-05-08T05:29:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-08T05:29:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-08T05:29:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-08T05:29:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-08T05:29:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-08T05:29:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-05-08T05:29:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-08T05:29:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-08T05:29:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-05-08T05:29:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-08T05:29:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-08T05:29:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-05-08T05:29:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-08T05:29:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-08T05:29:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-08T05:29:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-08T05:29:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-08T05:29:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-08T05:29:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-08T05:29:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-08T05:29:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-05-08T05:29:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-08T05:29:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-08T05:29:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-08T05:29:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-08T05:29:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-08T05:29:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-08T05:29:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-08T05:29:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-08T05:29:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-08T05:29:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-08T05:29:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-08T05:29:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-08T05:29:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-05-08T05:29:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-08T05:29:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-08T05:29:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-08T05:29:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-08T05:29:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-08T05:29:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-08T05:29:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-08T05:29:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-08T05:29:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-08T05:29:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-05-08T05:29:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-05-08T05:29:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-08T05:29:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-08T05:29:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-08T05:29:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-05-08T05:29:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-08T05:29:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-08T05:29:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-08T05:29:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-08T05:29:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-08T05:29:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-08T05:29:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-05-08T05:29:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-08T05:29:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-05-08T05:29:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-05-08T05:29:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-08T05:29:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-05-08T05:29:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-08T05:29:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-05-08T05:29:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-08T05:29:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-05-08T05:29:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-05-08T05:29:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-08T05:29:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-05-08T05:29:28.028 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-08T05:29:28.029 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-08T05:29:28.029 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-08T05:29:28.029 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-05-08T05:29:28.029 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-05-08T05:29:28.029 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-08T05:29:28.029 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-05-08T05:29:28.029 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-05-08T05:29:28.029 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-08T05:29:28.029 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-08T05:29:28.029 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-08T05:29:28.029 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-08T05:29:28.029 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-08T05:29:28.029 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-08T05:29:28.029 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-08T05:29:28.030 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-08T05:29:28.030 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-08T05:29:28.030 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-08T05:29:28.030 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-08T05:29:28.030 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-08T05:29:28.030 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-05-08T05:29:28.030 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-08T05:29:28.030 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-08T05:29:28.030 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-08T05:29:28.030 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-08T05:29:28.030 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-08T05:29:28.030 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-08T05:29:28.030 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-08T05:29:28.030 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-08T05:29:28.031 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-08T05:29:28.031 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-08T05:29:28.031 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-08T05:29:28.031 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-08T05:29:28.031 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-05-08T05:29:28.031 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-08T05:29:28.031 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-08T05:29:28.031 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-05-08T05:29:28.031 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-05-08T05:29:28.031 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-08T05:29:28.031 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-08T05:29:28.031 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-08T05:29:28.031 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-08T05:29:28.032 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-05-08T05:29:28.032 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-08T05:29:28.032 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-08T05:29:28.032 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-05-08T05:29:28.032 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-05-08T05:29:28.032 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-05-08T05:29:28.032 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-08T05:29:28.032 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-08T05:29:28.032 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-05-08T05:29:28.032 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-08T05:29:28.032 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-08T05:29:28.032 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-08T05:29:28.032 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-08T05:29:28.032 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-08T05:29:28.032 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-08T05:29:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-08T05:29:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-08T05:29:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-08T05:29:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-08T05:29:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-08T05:29:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-08T05:29:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-08T05:29:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-08T05:29:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-08T05:29:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-08T05:29:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-08T05:29:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-08T05:29:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-08T05:29:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-08T05:29:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-08T05:29:28.034 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-08T05:29:28.034 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-08T05:29:28.034 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-08T05:29:28.034 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-08T05:29:28.034 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-08T05:29:28.034 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-08T05:29:28.034 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-08T05:29:28.034 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-08T05:29:28.034 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-08T05:29:28.034 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-08T05:29:28.034 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-05-08T05:29:28.034 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-08T05:29:28.034 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-08T05:29:28.034 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-08T05:29:28.034 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-08T05:29:28.035 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-05-08T05:29:28.036 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-08T05:29:28.036 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-05-08T05:29:28.036 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-08T05:29:28.036 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-08T05:29:28.036 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-05-08T05:29:28.036 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-08T05:29:28.036 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-05-08T05:29:28.036 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-08T05:29:28.036 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-08T05:29:28.036 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-08T05:29:28.036 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-08T05:29:28.036 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-08T05:29:28.036 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-08T05:29:28.036 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-05-08T05:29:28.036 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-08T05:29:28.037 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-08T05:29:28.038 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-08T05:29:28.038 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-08T05:29:28.038 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-05-08T05:29:28.038 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-08T05:29:28.038 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-08T05:29:28.038 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-08T05:29:28.038 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-05-08T05:29:28.038 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-08T05:29:28.039 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-08T05:29:28.039 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-08T05:29:28.039 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-05-08T05:29:28.039 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-08T05:29:28.039 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-05-08T05:29:28.039 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-08T05:29:28.039 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-05-08T05:29:28.039 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-08T05:29:28.039 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-08T05:29:28.039 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-08T05:29:28.039 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-08T05:29:28.040 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-08T05:29:28.041 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-05-08T05:29:28.042 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-05-08T05:29:28.042 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-08T05:29:28.042 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-05-08T05:29:28.042 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-08T05:29:28.042 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-08T05:29:28.042 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-08T05:29:28.042 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-08T05:29:28.042 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-08T05:29:28.042 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-08T05:29:28.042 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-05-08T05:29:28.042 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-08T05:29:28.042 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-05-08T05:29:28.042 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-05-08T05:29:28.042 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-05-08T05:29:28.042 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-08T05:29:28.043 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-08T05:29:28.043 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-08T05:29:28.043 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-08T05:29:28.043 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-08T05:29:28.043 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-08T05:29:28.043 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-08T05:29:28.043 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-08T05:29:28.043 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-08T05:29:28.043 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-08T05:29:28.043 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-08T05:29:28.043 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-08T05:29:28.043 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-08T05:29:28.043 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-08T05:29:28.043 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-08T05:29:28.043 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-08T05:29:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-08T05:29:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-08T05:29:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-05-08T05:29:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-05-08T05:29:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-08T05:29:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-08T05:29:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-08T05:29:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-08T05:29:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-05-08T05:29:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-05-08T05:29:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-08T05:29:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-08T05:29:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-05-08T05:29:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-08T05:29:28.045 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-08T05:29:28.046 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-08T05:29:28.046 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-05-08T05:29:28.046 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-08T05:29:28.046 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-05-08T05:29:28.046 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-08T05:29:28.046 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-08T05:29:28.046 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-05-08T05:29:28.046 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-08T05:29:28.046 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-08T05:29:28.046 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-08T05:29:28.046 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-08T05:29:28.046 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-08T05:29:28.046 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-08T05:29:28.046 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-08T05:29:28.047 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-08T05:29:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-08T05:29:28.049 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-08T05:29:28.049 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-08T05:29:28.049 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-08T05:29:28.049 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-08T05:29:28.049 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-08T05:29:28.049 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-08T05:29:28.049 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-08T05:29:28.049 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-08T05:29:28.049 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-08T05:29:28.049 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-08T05:29:28.049 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-08T05:29:28.049 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-08T05:29:28.049 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-08T05:29:28.049 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-08T05:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-08T05:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-08T05:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-08T05:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-08T05:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-08T05:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-08T05:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-08T05:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-08T05:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-08T05:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-08T05:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-08T05:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-08T05:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-05-08T05:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-08T05:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-08T05:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-08T05:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-08T05:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-08T05:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-08T05:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-08T05:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-08T05:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-08T05:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-08T05:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-08T05:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-08T05:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-08T05:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-08T05:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-08T05:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-05-08T05:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-05-08T05:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-08T05:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-08T05:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-08T05:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-05-08T05:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-05-08T05:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-05-08T05:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-05-08T05:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-05-08T05:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-08T05:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-08T05:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-05-08T05:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-05-08T05:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-05-08T05:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-08T05:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-08T05:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-05-08T05:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-08T05:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-08T05:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-08T05:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-08T05:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-08T05:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-08T05:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-08T05:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-08T05:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-08T05:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-08T05:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-08T05:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-08T05:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-05-08T05:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-08T05:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-08T05:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-08T05:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-08T05:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-08T05:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-08T05:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-08T05:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-08T05:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-08T05:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-08T05:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-08T05:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-08T05:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-08T05:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-08T05:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-08T05:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-08T05:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-08T05:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-08T05:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-08T05:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-08T05:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-08T05:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-08T05:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-08T05:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-08T05:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-08T05:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-08T05:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-05-08T05:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-08T05:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-08T05:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-05-08T05:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-08T05:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-05-08T05:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-08T05:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-05-08T05:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-08T05:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-05-08T05:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-08T05:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-05-08T05:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-08T05:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-05-08T05:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-08T05:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-05-08T05:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-08T05:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-05-08T05:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-08T05:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-05-08T05:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-05-08T05:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-08T05:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-05-08T05:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-05-08T05:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-08T05:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-05-08T05:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-05-08T05:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-08T05:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-05-08T05:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-08T05:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-05-08T05:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-08T05:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-05-08T05:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-08T05:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-05-08T05:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-08T05:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-05-08T05:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-08T05:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-05-08T05:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-08T05:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-05-08T05:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-08T05:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-05-08T05:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-05-08T05:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-08T05:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-05-08T05:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-08T05:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-05-08T05:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-08T05:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-08T05:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-08T05:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-08T05:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-05-08T05:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-08T05:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-08T05:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-05-08T05:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-08T05:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-05-08T05:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-08T05:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-05-08T05:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-08T05:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-08T05:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-08T05:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-08T05:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-08T05:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-08T05:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-08T05:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-08T05:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-08T05:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-08T05:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-08T05:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-08T05:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-08T05:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-08T05:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-08T05:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-08T05:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-08T05:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-08T05:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-08T05:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-08T05:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-08T05:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-08T05:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-08T05:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-08T05:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-08T05:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-08T05:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-08T05:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-08T05:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-08T05:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-08T05:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-08T05:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-08T05:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-08T05:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-08T05:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-08T05:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-08T05:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-08T05:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-08T05:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-08T05:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-08T05:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-08T05:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-08T05:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-08T05:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-08T05:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-08T05:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-08T05:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-08T05:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-08T05:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-08T05:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-08T05:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-05-08T05:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-05-08T05:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-08T05:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-05-08T05:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-08T05:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-05-08T05:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-08T05:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-08T05:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-08T05:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-08T05:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-08T05:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-05-08T05:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-08T05:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-05-08T05:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-08T05:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-05-08T05:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-08T05:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-05-08T05:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-08T05:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-05-08T05:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-05-08T05:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-08T05:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-05-08T05:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-08T05:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-08T05:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-05-08T05:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-08T05:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-05-08T05:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-08T05:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-05-08T05:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-08T05:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-05-08T05:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-08T05:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-05-08T05:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-08T05:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-05-08T05:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-05-08T05:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-05-08T05:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-08T05:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-05-08T05:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-08T05:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-05-08T05:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-05-08T05:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-08T05:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-08T05:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-05-08T05:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-08T05:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-08T05:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-08T05:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-08T05:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-05-08T05:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-08T05:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-08T05:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-05-08T05:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-08T05:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-05-08T05:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-08T05:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-05-08T05:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-08T05:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-08T05:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-08T05:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-08T05:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-05-08T05:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-05-08T05:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-05-08T05:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-08T05:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-05-08T05:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-05-08T05:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-08T05:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-08T05:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-08T05:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-08T05:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-08T05:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-08T05:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-08T05:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-08T05:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-05-08T05:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-08T05:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-08T05:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-08T05:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-08T05:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-08T05:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-05-08T05:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-08T05:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-05-08T05:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-08T05:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-08T05:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-08T05:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-08T05:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-05-08T05:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-08T05:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-05-08T05:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-08T05:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-08T05:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-05-08T05:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-08T05:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-08T05:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-08T05:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-08T05:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-08T05:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-08T05:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-08T05:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-08T05:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-08T05:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-08T05:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-08T05:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-08T05:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-08T05:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-08T05:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-08T05:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-08T05:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-08T05:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-08T05:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-08T05:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-08T05:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-08T05:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-08T05:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-08T05:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-08T05:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-05-08T05:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-08T05:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-05-08T05:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-05-08T05:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-08T05:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-05-08T05:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-08T05:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-08T05:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-08T05:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-08T05:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-05-08T05:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-05-08T05:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-05-08T05:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-08T05:29:28.075 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-08T05:29:28.075 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-08T05:29:28.075 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-08T05:29:28.075 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-08T05:29:28.075 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-05-08T05:29:28.075 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-05-08T05:29:28.075 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-08T05:29:28.075 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-08T05:29:28.075 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-08T05:29:28.075 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-05-08T05:29:28.075 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-08T05:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-08T05:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-08T05:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-08T05:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-08T05:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-08T05:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-08T05:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-08T05:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-08T05:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-08T05:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-05-08T05:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-08T05:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-08T05:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-08T05:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-08T05:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-08T05:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-08T05:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-08T05:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-08T05:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-08T05:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-08T05:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-08T05:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-08T05:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-08T05:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-08T05:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-08T05:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-08T05:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-08T05:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-08T05:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-08T05:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-08T05:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-08T05:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-08T05:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-08T05:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-08T05:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-08T05:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-08T05:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-08T05:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-08T05:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-08T05:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-08T05:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-08T05:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-08T05:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-08T05:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-08T05:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-08T05:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-08T05:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-08T05:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-08T05:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-05-08T05:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-08T05:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-08T05:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-08T05:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-08T05:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-08T05:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-08T05:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-08T05:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-05-08T05:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-08T05:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-08T05:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-08T05:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-08T05:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-08T05:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-08T05:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-08T05:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-08T05:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-08T05:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-08T05:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-08T05:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-08T05:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-08T05:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-08T05:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-08T05:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-08T05:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-08T05:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-08T05:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-08T05:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-08T05:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-08T05:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-08T05:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-08T05:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-08T05:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-08T05:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-08T05:29:28.082 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-08T05:29:28.082 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-05-08T05:29:28.082 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-08T05:29:28.082 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-05-08T05:29:28.082 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-08T05:29:28.082 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-08T05:29:28.082 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-08T05:29:28.082 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-08T05:29:28.082 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-08T05:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-08T05:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-08T05:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-05-08T05:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-08T05:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-05-08T05:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-08T05:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-05-08T05:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-08T05:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-05-08T05:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-05-08T05:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-08T05:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-08T05:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-08T05:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-08T05:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-08T05:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-08T05:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-08T05:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-08T05:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-08T05:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-08T05:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-08T05:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-08T05:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-08T05:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-08T05:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-08T05:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-08T05:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-05-08T05:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-08T05:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-08T05:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-08T05:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-08T05:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-08T05:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-08T05:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-08T05:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-08T05:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-08T05:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-08T05:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-08T05:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-08T05:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-08T05:29:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-08T05:29:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-08T05:29:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-08T05:29:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-08T05:29:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-08T05:29:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-08T05:29:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-08T05:29:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-08T05:29:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-08T05:29:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-08T05:29:28.086 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-08T05:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-08T05:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-08T05:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-08T05:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-08T05:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-08T05:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-08T05:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-08T05:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-08T05:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-08T05:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-08T05:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-08T05:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-08T05:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-08T05:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-08T05:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-08T05:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-08T05:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-08T05:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-08T05:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-08T05:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-08T05:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-08T05:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-08T05:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-08T05:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-08T05:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-08T05:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-08T05:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-08T05:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-08T05:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-08T05:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-08T05:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-08T05:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-08T05:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-08T05:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-08T05:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-05-08T05:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-08T05:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-08T05:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-08T05:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-08T05:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-05-08T05:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-08T05:29:28.090 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-08T05:29:28.090 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-08T05:29:28.090 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-08T05:29:28.090 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-08T05:29:28.090 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-08T05:29:28.090 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-08T05:29:28.090 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-08T05:29:28.090 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-08T05:29:28.090 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-08T05:29:28.090 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-08T05:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-08T05:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-08T05:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-08T05:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-08T05:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-08T05:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-08T05:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-08T05:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-08T05:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-08T05:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-08T05:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-08T05:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-08T05:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-08T05:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-08T05:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-08T05:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-08T05:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-08T05:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-08T05:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-08T05:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-08T05:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-08T05:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-08T05:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-08T05:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-08T05:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-08T05:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-08T05:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-08T05:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-08T05:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-08T05:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-08T05:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-08T05:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-08T05:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-08T05:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-08T05:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-08T05:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-08T05:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-08T05:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-08T05:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-08T05:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-08T05:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-08T05:29:28.094 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-08T05:29:28.094 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-08T05:29:28.094 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-08T05:29:28.094 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-08T05:29:28.094 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-08T05:29:28.094 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-08T05:29:28.094 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-08T05:29:28.094 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-08T05:29:28.094 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-08T05:29:28.094 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-08T05:29:28.094 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-08T05:29:28.094 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-08T05:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-08T05:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-08T05:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-08T05:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-08T05:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-08T05:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-08T05:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-08T05:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-08T05:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-08T05:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-08T05:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-08T05:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-08T05:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-08T05:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-08T05:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-08T05:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-08T05:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-08T05:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-08T05:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-08T05:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-08T05:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-05-08T05:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-08T05:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-08T05:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-05-08T05:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-05-08T05:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-08T05:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-08T05:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-08T05:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-05-08T05:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-08T05:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-05-08T05:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-08T05:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-05-08T05:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-05-08T05:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-08T05:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-08T05:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-08T05:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-08T05:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-08T05:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-08T05:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-08T05:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-08T05:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-08T05:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-08T05:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-08T05:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-08T05:29:28.102 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-08T05:29:28.102 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-08T05:29:28.106 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-08T05:29:28.106 DEBUG:teuthology.orchestra.run.smithi112:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-08T05:29:28.131 INFO:teuthology.orchestra.run.smithi112.stdout:postinstall scriptlet (using /bin/sh): 2024-05-08T05:29:28.131 INFO:teuthology.orchestra.run.smithi112.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-08T05:29:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-08T05:29:28.131 INFO:teuthology.orchestra.run.smithi112.stdout:fi 2024-05-08T05:29:28.131 INFO:teuthology.orchestra.run.smithi112.stdout:for file in vmlinuz System.map config; do 2024-05-08T05:29:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-05-08T05:29:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-05-08T05:29:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: fi 2024-05-08T05:29:28.131 INFO:teuthology.orchestra.run.smithi112.stdout:done 2024-05-08T05:29:28.131 INFO:teuthology.orchestra.run.smithi112.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-05-08T05:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-05-08T05:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-08T05:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-08T05:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout:fi 2024-05-08T05:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-05-08T05:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-05-08T05:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout:fi 2024-05-08T05:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-08T05:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-08T05:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-05-08T05:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout:fi 2024-05-08T05:29:28.133 DEBUG:teuthology.orchestra.run.smithi112:> sudo rpm -qi grub2-tools 2024-05-08T05:29:28.207 INFO:teuthology.orchestra.run.smithi112.stdout:Name : grub2-tools 2024-05-08T05:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout:Epoch : 1 2024-05-08T05:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout:Version : 2.06 2024-05-08T05:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout:Release : 68.el9 2024-05-08T05:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout:Architecture: x86_64 2024-05-08T05:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T05:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout:Group : Unspecified 2024-05-08T05:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout:Size : 8264254 2024-05-08T05:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout:License : GPLv3+ 2024-05-08T05:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T05:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T05:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T05:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T05:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout:Packager : builder@centos.org 2024-05-08T05:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout:Vendor : CentOS 2024-05-08T05:29:28.209 INFO:teuthology.orchestra.run.smithi112.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T05:29:28.209 INFO:teuthology.orchestra.run.smithi112.stdout:Summary : Support tools for GRUB. 2024-05-08T05:29:28.209 INFO:teuthology.orchestra.run.smithi112.stdout:Description : 2024-05-08T05:29:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:29:28.209 INFO:teuthology.orchestra.run.smithi112.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T05:29:28.209 INFO:teuthology.orchestra.run.smithi112.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T05:29:28.209 INFO:teuthology.orchestra.run.smithi112.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T05:29:28.209 INFO:teuthology.orchestra.run.smithi112.stdout:hardware devices. 2024-05-08T05:29:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:29:28.209 INFO:teuthology.orchestra.run.smithi112.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T05:29:28.210 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T05:29:28.210 INFO:teuthology.task.kernel:Updating grub on smithi112 to boot 6.9.0-rc3-g806ec36b2112 2024-05-08T05:29:28.210 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T05:29:28.799 INFO:teuthology.orchestra.run.smithi112.stderr:Generating grub configuration file ... 2024-05-08T05:29:30.227 INFO:teuthology.orchestra.run.smithi112.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T05:29:30.260 INFO:teuthology.orchestra.run.smithi112.stderr:done 2024-05-08T05:29:30.263 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-05-08T05:29:30.289 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.j69gvqKaLH 2024-05-08T05:29:30.289 DEBUG:teuthology.orchestra.run.smithi112:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.j69gvqKaLH 2024-05-08T05:29:30.354 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.j69gvqKaLH 2024-05-08T05:29:30.501 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.j69gvqKaLH is 6KB 2024-05-08T05:29:30.549 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.j69gvqKaLH 2024-05-08T05:29:30.563 DEBUG:teuthology.orchestra.run.smithi112:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T05:29:30.627 INFO:teuthology.orchestra.run.smithi112.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T05:29:30.628 INFO:teuthology.orchestra.run.smithi112.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T05:29:30.628 INFO:teuthology.orchestra.run.smithi112.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T05:29:30.628 INFO:teuthology.orchestra.run.smithi112.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-05-08T05:29:30.629 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-05-08T05:29:30.757 DEBUG:teuthology.orchestra.run.smithi112:> sudo shutdown -r now 2024-05-08T05:30:00.760 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-08T05:30:00.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-05-08T05:30:00.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:30:19.294 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-05-08T05:30:28.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-05-08T05:30:28.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:31:28.299 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T05:31:40.300 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-05-08T05:31:40.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:31:40.711 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-05-08T05:31:41.334 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-05-08T05:31:41.334 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-05-08T05:31:41.334 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-05-08T05:31:41.382 INFO:teuthology.orchestra.run.smithi112.stdout:6.9.0-rc3-g806ec36b2112 2024-05-08T05:31:41.383 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-08T05:31:41.383 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T05:31:41.383 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T05:31:42.384 DEBUG:teuthology.parallel:result is None 2024-05-08T05:31:42.384 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T05:31:42.392 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T05:31:42.392 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T05:31:42.400 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T05:31:42.402 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T05:31:42.450 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T05:31:42.457 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T05:31:42.509 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T05:31:42.509 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T05:31:42.516 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T05:31:42.519 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T05:31:42.636 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T05:31:42.643 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T05:31:42.643 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T05:31:42.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:31:42.660 DEBUG:teuthology.orchestra.run.smithi162:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T05:31:42.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:31:42.677 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T05:31:42.683 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T05:31:42.702 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T05:31:42.753 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:31:42.774 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:31:42.779 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:31:42.798 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:31:42.802 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:31:42.857 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:31:42.862 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T05:31:42.869 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T05:31:42.870 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T05:31:42.876 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T05:31:42.879 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T05:31:42.953 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T05:31:42.963 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T05:31:42.963 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T05:31:42.970 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T05:31:42.973 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T05:31:43.017 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:31:43.098 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:31:43.195 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:31:43.280 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:31:43.360 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:31:43.360 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T05:31:43.435 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:31:43.480 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:31:43.542 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:31:43.584 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:31:43.628 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-08T05:31:43.628 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T05:31:43.694 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:31:43.738 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:31:43.799 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:31:43.843 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:31:43.883 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-08T05:31:43.883 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T05:31:43.949 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-05-08T05:31:43.956 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-05-08T05:31:43.958 DEBUG:teuthology.orchestra.run.smithi162:> sudo service rsyslog restart 2024-05-08T05:31:43.995 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:31:44.022 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:31:44.062 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:31:44.476 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T05:31:44.484 INFO:teuthology.task.internal:Starting timer... 2024-05-08T05:31:44.485 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T05:31:44.562 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T05:31:44.611 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-08T05:31:44.612 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-05-08T05:31:44.732 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-05-08T05:31:44.737 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-05-08T05:31:44.803 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-05-08T05:31:44.805 DEBUG:teuthology.orchestra.run.smithi162:> sudo service auditd rotate 2024-05-08T05:31:44.870 INFO:teuthology.orchestra.run.smithi162.stdout:Rotating logs: 2024-05-08T05:31:44.872 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T05:31:44.872 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-05-08T05:31:44.932 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-05-08T05:31:44.932 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-05-08T05:31:44.959 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-05-08T05:31:44.960 DEBUG:teuthology.orchestra.run.smithi162:> /usr/sbin/getenforce 2024-05-08T05:31:44.980 INFO:teuthology.orchestra.run.smithi162.stdout:Permissive 2024-05-08T05:31:44.980 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi112.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi162.front.sepia.ceph.com': 'permissive'} 2024-05-08T05:31:44.980 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:31:45.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:31:45.071 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:31:45.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:31:45.100 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:31:45.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:31:45.125 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T05:31:45.125 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-05-08T05:31:45.199 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-05-08T05:31:45.224 DEBUG:teuthology.orchestra.run.smithi162:> sudo /usr/sbin/setenforce permissive 2024-05-08T05:31:45.248 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T05:31:45.261 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-08T05:31:45.267 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-08T05:31:45.394 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T05:31:45.410 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T05:31:45.411 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi112.front.sepia.ceph.com,smithi150.front.sepia.ceph.com,smithi162.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T05:37:14.128 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi112.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com'), Remote(name='ubuntu@smithi162.front.sepia.ceph.com')] 2024-05-08T05:37:14.129 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-05-08T05:37:14.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:37:14.240 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-05-08T05:37:14.393 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-05-08T05:37:14.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-08T05:37:14.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:37:14.467 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-05-08T05:37:14.546 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-08T05:37:14.546 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-05-08T05:37:14.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:37:14.621 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-05-08T05:37:14.702 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-05-08T05:37:14.703 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T05:37:14.717 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T05:37:14.717 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T05:37:14.718 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:37:14.725 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T05:37:14.726 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:37:14.728 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T05:37:14.728 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:37:14.756 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T05:37:14.771 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T05:37:14.772 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T05:37:14.785 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T05:37:14.795 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-05-08T05:37:14.806 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-05-08T05:37:14.808 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: ntpd: command not found 2024-05-08T05:37:14.809 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T05:37:14.819 INFO:teuthology.orchestra.run.smithi162.stdout:506 Cannot talk to daemon 2024-05-08T05:37:14.820 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T05:37:14.831 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T05:37:14.833 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T05:37:14.844 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T05:37:14.851 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T05:37:14.887 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:37:14.890 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:37:14.891 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-05-08T05:37:14.891 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:14.891 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:14.891 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:14.891 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:14.902 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:37:14.902 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:37:14.903 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-05-08T05:37:14.903 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:14.903 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:14.903 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:14.903 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:14.925 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-05-08T05:37:14.956 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-05-08T05:37:15.000 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T05:37:15.043 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T05:37:15.168 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:37:15.177 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:37:15.177 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-05-08T05:37:15.177 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:15.177 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:15.177 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:15.177 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:15.178 INFO:teuthology.run_tasks:Running task install... 2024-05-08T05:37:15.187 DEBUG:teuthology.task.install:project ceph 2024-05-08T05:37:15.187 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a'}} 2024-05-08T05:37:15.187 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a'} 2024-05-08T05:37:15.187 INFO:teuthology.task.install:Using flavor: default 2024-05-08T05:37:15.193 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T05:37:15.193 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T05:37:15.194 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'tag': None, 'wait_for_package': False} 2024-05-08T05:37:15.194 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:37:15.196 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'tag': None, 'wait_for_package': False} 2024-05-08T05:37:15.197 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:37:15.198 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'tag': None, 'wait_for_package': False} 2024-05-08T05:37:15.198 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:37:15.349 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/ 2024-05-08T05:37:15.349 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gbee299f4 2024-05-08T05:37:15.412 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/ 2024-05-08T05:37:15.413 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gbee299f4 2024-05-08T05:37:15.422 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/ 2024-05-08T05:37:15.422 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gbee299f4 2024-05-08T05:37:15.464 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T05:37:15.464 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-08T05:37:15.464 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T05:37:15.492 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T05:37:15.492 DEBUG:teuthology.orchestra.run.smithi162:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T05:37:15.559 DEBUG:teuthology.orchestra.run.smithi162:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T05:37:15.585 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T05:37:15.585 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:37:15.585 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T05:37:15.635 DEBUG:teuthology.orchestra.run.smithi162:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T05:37:15.677 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T05:37:15.677 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T05:37:15.699 INFO:teuthology.orchestra.run.smithi162.stdout:check_obsoletes = 1 2024-05-08T05:37:15.700 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum clean all 2024-05-08T05:37:15.802 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T05:37:15.915 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T05:37:15.929 INFO:teuthology.orchestra.run.smithi162.stdout:54 files removed 2024-05-08T05:37:15.957 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T05:37:15.994 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-05-08T05:37:16.000 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-05-08T05:37:16.334 INFO:teuthology.orchestra.run.smithi112.stdout:54 files removed 2024-05-08T05:37:16.372 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T05:37:16.572 INFO:teuthology.orchestra.run.smithi162.stdout:ceph packages for x86_64 236 kB/s | 84 kB 00:00 2024-05-08T05:37:16.615 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T05:37:16.616 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-08T05:37:16.616 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T05:37:16.645 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T05:37:16.645 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T05:37:16.714 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T05:37:16.791 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T05:37:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-05-08T05:37:16.858 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-05-08T05:37:16.956 INFO:teuthology.orchestra.run.smithi162.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-05-08T05:37:17.090 INFO:teuthology.orchestra.run.smithi150.stdout:54 files removed 2024-05-08T05:37:17.118 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T05:37:17.277 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 176 kB/s | 84 kB 00:00 2024-05-08T05:37:17.506 INFO:teuthology.orchestra.run.smithi162.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2024-05-08T05:37:17.848 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 180 kB/s | 84 kB 00:00 2024-05-08T05:37:18.011 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 18 kB/s | 12 kB 00:00 2024-05-08T05:37:18.424 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-05-08T05:37:18.565 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-05-08T05:37:18.869 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-05-08T05:37:19.326 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-08T05:37:21.583 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.1 MB 00:02 2024-05-08T05:37:21.878 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-05-08T05:37:22.314 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.1 MB 00:04 2024-05-08T05:37:24.256 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-05-08T05:37:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-08T05:37:28.130 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-05-08T05:37:30.331 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.2 MB 00:02 2024-05-08T05:37:30.337 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:02 2024-05-08T05:37:31.857 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-05-08T05:37:36.498 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-08T05:37:38.657 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-05-08T05:37:39.066 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:37:39.068 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:37:39.089 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T05:37:39.091 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T05:37:39.092 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T05:37:39.093 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T05:37:39.096 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T05:37:39.099 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T05:37:39.100 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T05:37:39.171 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-05-08T05:37:39.195 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-05-08T05:37:39.195 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-05-08T05:37:39.195 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-05-08T05:37:39.196 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-05-08T05:37:39.196 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T05:37:39.196 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 6.6 k 2024-05-08T05:37:39.196 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:37:39.196 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 890 k 2024-05-08T05:37:39.196 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 145 k 2024-05-08T05:37:39.197 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 1.5 M 2024-05-08T05:37:39.197 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 142 k 2024-05-08T05:37:39.197 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 3.5 M 2024-05-08T05:37:39.197 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 7.4 M 2024-05-08T05:37:39.197 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 49 k 2024-05-08T05:37:39.197 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 11 M 2024-05-08T05:37:39.198 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 49 M 2024-05-08T05:37:39.198 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 273 k 2024-05-08T05:37:39.198 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 763 k 2024-05-08T05:37:39.198 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 225 k 2024-05-08T05:37:39.198 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 27 k 2024-05-08T05:37:39.199 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 18 k 2024-05-08T05:37:39.199 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T05:37:39.199 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T05:37:39.199 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T05:37:39.199 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 32 k 2024-05-08T05:37:39.199 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 747 k 2024-05-08T05:37:39.199 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 127 k 2024-05-08T05:37:39.200 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T05:37:39.200 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T05:37:39.200 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T05:37:39.200 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T05:37:39.200 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T05:37:39.200 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T05:37:39.201 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 161 k 2024-05-08T05:37:39.201 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 321 k 2024-05-08T05:37:39.201 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 299 k 2024-05-08T05:37:39.202 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 100 k 2024-05-08T05:37:39.202 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 85 k 2024-05-08T05:37:39.202 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:37:39.202 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 171 k 2024-05-08T05:37:39.202 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T05:37:39.203 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-05-08T05:37:39.203 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T05:37:39.203 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T05:37:39.203 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T05:37:39.203 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T05:37:39.203 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.4 M 2024-05-08T05:37:39.203 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:37:39.203 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T05:37:39.203 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T05:37:39.203 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-05-08T05:37:39.203 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T05:37:39.203 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T05:37:39.203 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T05:37:39.204 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 21 M 2024-05-08T05:37:39.204 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 34 k 2024-05-08T05:37:39.204 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 2.4 M 2024-05-08T05:37:39.204 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 274 k 2024-05-08T05:37:39.204 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 4.7 M 2024-05-08T05:37:39.204 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 17 M 2024-05-08T05:37:39.204 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 16 k 2024-05-08T05:37:39.204 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 25 k 2024-05-08T05:37:39.204 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T05:37:39.204 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T05:37:39.204 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T05:37:39.204 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T05:37:39.205 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T05:37:39.205 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T05:37:39.205 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T05:37:39.205 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T05:37:39.205 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 163 k 2024-05-08T05:37:39.205 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T05:37:39.205 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T05:37:39.205 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T05:37:39.205 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T05:37:39.205 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T05:37:39.205 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 500 k 2024-05-08T05:37:39.205 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T05:37:39.205 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:37:39.205 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T05:37:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T05:37:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T05:37:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T05:37:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T05:37:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T05:37:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T05:37:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T05:37:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T05:37:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T05:37:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T05:37:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T05:37:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T05:37:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T05:37:39.206 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T05:37:39.207 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T05:37:39.207 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T05:37:39.207 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 45 k 2024-05-08T05:37:39.207 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 135 k 2024-05-08T05:37:39.207 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T05:37:39.207 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T05:37:39.207 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T05:37:39.207 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T05:37:39.207 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T05:37:39.207 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T05:37:39.207 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T05:37:39.207 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T05:37:39.207 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T05:37:39.208 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T05:37:39.208 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T05:37:39.208 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T05:37:39.208 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T05:37:39.208 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T05:37:39.208 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T05:37:39.208 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T05:37:39.208 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T05:37:39.208 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T05:37:39.208 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T05:37:39.208 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T05:37:39.208 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T05:37:39.208 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T05:37:39.208 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T05:37:39.209 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T05:37:39.209 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T05:37:39.209 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T05:37:39.209 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T05:37:39.209 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T05:37:39.209 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T05:37:39.209 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T05:37:39.209 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T05:37:39.209 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T05:37:39.209 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T05:37:39.209 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T05:37:39.209 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T05:37:39.209 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T05:37:39.209 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T05:37:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T05:37:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T05:37:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T05:37:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T05:37:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T05:37:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T05:37:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T05:37:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T05:37:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T05:37:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T05:37:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T05:37:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T05:37:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T05:37:39.210 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T05:37:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T05:37:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T05:37:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T05:37:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T05:37:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T05:37:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T05:37:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T05:37:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T05:37:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T05:37:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T05:37:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T05:37:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T05:37:39.211 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-05-08T05:37:39.211 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T05:37:39.212 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:37:39.212 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-05-08T05:37:39.212 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-05-08T05:37:39.212 INFO:teuthology.orchestra.run.smithi112.stdout:Install 148 Packages 2024-05-08T05:37:39.212 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 8 Packages 2024-05-08T05:37:39.212 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:37:39.212 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 226 M 2024-05-08T05:37:39.213 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-05-08T05:37:39.850 INFO:teuthology.orchestra.run.smithi112.stdout:(1/156): ceph-19.0.0-3538.gbee299f4.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-05-08T05:37:40.084 INFO:teuthology.orchestra.run.smithi112.stdout:(2/156): ceph-fuse-19.0.0-3538.gbee299f4.el9.x8 3.7 MB/s | 890 kB 00:00 2024-05-08T05:37:40.151 INFO:teuthology.orchestra.run.smithi112.stdout:(3/156): ceph-immutable-object-cache-19.0.0-353 2.1 MB/s | 145 kB 00:00 2024-05-08T05:37:40.434 INFO:teuthology.orchestra.run.smithi112.stdout:(4/156): ceph-mds-19.0.0-3538.gbee299f4.el9.x86 8.4 MB/s | 2.4 MB 00:00 2024-05-08T05:37:40.609 INFO:teuthology.orchestra.run.smithi112.stdout:(5/156): ceph-mgr-19.0.0-3538.gbee299f4.el9.x86 8.7 MB/s | 1.5 MB 00:00 2024-05-08T05:37:40.960 INFO:teuthology.orchestra.run.smithi112.stdout:(6/156): ceph-base-19.0.0-3538.gbee299f4.el9.x8 3.9 MB/s | 5.4 MB 00:01 2024-05-08T05:37:41.069 INFO:teuthology.orchestra.run.smithi112.stdout:(7/156): ceph-mon-19.0.0-3538.gbee299f4.el9.x86 10 MB/s | 4.7 MB 00:00 2024-05-08T05:37:42.639 INFO:teuthology.orchestra.run.smithi112.stdout:(8/156): ceph-common-19.0.0-3538.gbee299f4.el9. 6.9 MB/s | 21 MB 00:03 2024-05-08T05:37:42.683 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-05-08T05:37:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:(9/156): ceph-radosgw-19.0.0-3538.gbee299f4.el9 6.1 MB/s | 11 MB 00:01 2024-05-08T05:37:42.861 INFO:teuthology.orchestra.run.smithi112.stdout:(10/156): ceph-selinux-19.0.0-3538.gbee299f4.el 114 kB/s | 25 kB 00:00 2024-05-08T05:37:42.938 INFO:teuthology.orchestra.run.smithi112.stdout:(11/156): cephfs-mirror-19.0.0-3538.gbee299f4.e 3.0 MB/s | 225 kB 00:00 2024-05-08T05:37:43.187 INFO:teuthology.orchestra.run.smithi112.stdout:(12/156): ceph-osd-19.0.0-3538.gbee299f4.el9.x8 7.6 MB/s | 17 MB 00:02 2024-05-08T05:37:43.228 INFO:teuthology.orchestra.run.smithi112.stdout:(13/156): cephfs-shell-19.0.0-3538.gbee299f4.el 93 kB/s | 27 kB 00:00 2024-05-08T05:37:43.279 INFO:teuthology.orchestra.run.smithi112.stdout:(14/156): libcephfs-devel-19.0.0-3538.gbee299f4 350 kB/s | 32 kB 00:00 2024-05-08T05:37:43.354 INFO:teuthology.orchestra.run.smithi112.stdout:(15/156): libcephfs2-19.0.0-3538.gbee299f4.el9. 5.9 MB/s | 747 kB 00:00 2024-05-08T05:37:43.388 INFO:teuthology.orchestra.run.smithi112.stdout:(16/156): libcephsqlite-19.0.0-3538.gbee299f4.e 1.5 MB/s | 163 kB 00:00 2024-05-08T05:37:43.430 INFO:teuthology.orchestra.run.smithi112.stdout:(17/156): librados-devel-19.0.0-3538.gbee299f4. 1.7 MB/s | 127 kB 00:00 2024-05-08T05:37:43.539 INFO:teuthology.orchestra.run.smithi112.stdout:(18/156): libradosstriper1-19.0.0-3538.gbee299f 3.3 MB/s | 500 kB 00:00 2024-05-08T05:37:43.639 INFO:teuthology.orchestra.run.smithi112.stdout:(19/156): python3-ceph-argparse-19.0.0-3538.gbe 456 kB/s | 45 kB 00:00 2024-05-08T05:37:43.739 INFO:teuthology.orchestra.run.smithi112.stdout:(20/156): python3-ceph-common-19.0.0-3538.gbee2 1.3 MB/s | 135 kB 00:00 2024-05-08T05:37:43.848 INFO:teuthology.orchestra.run.smithi112.stdout:(21/156): python3-cephfs-19.0.0-3538.gbee299f4. 1.5 MB/s | 161 kB 00:00 2024-05-08T05:37:43.973 INFO:teuthology.orchestra.run.smithi112.stdout:(22/156): python3-rados-19.0.0-3538.gbee299f4.e 2.5 MB/s | 321 kB 00:00 2024-05-08T05:37:44.081 INFO:teuthology.orchestra.run.smithi112.stdout:(23/156): librgw2-19.0.0-3538.gbee299f4.el9.x86 8.2 MB/s | 5.4 MB 00:00 2024-05-08T05:37:44.115 INFO:teuthology.orchestra.run.smithi112.stdout:(24/156): python3-rbd-19.0.0-3538.gbee299f4.el9 2.1 MB/s | 299 kB 00:00 2024-05-08T05:37:44.165 INFO:teuthology.orchestra.run.smithi112.stdout:(25/156): python3-rgw-19.0.0-3538.gbee299f4.el9 1.2 MB/s | 100 kB 00:00 2024-05-08T05:37:44.199 INFO:teuthology.orchestra.run.smithi112.stdout:(26/156): rbd-fuse-19.0.0-3538.gbee299f4.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-08T05:37:44.251 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-05-08T05:37:44.324 INFO:teuthology.orchestra.run.smithi112.stdout:(27/156): rbd-nbd-19.0.0-3538.gbee299f4.el9.x86 1.3 MB/s | 171 kB 00:00 2024-05-08T05:37:44.425 INFO:teuthology.orchestra.run.smithi112.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3538.g 344 kB/s | 34 kB 00:00 2024-05-08T05:37:44.525 INFO:teuthology.orchestra.run.smithi112.stdout:(29/156): rbd-mirror-19.0.0-3538.gbee299f4.el9. 8.6 MB/s | 3.1 MB 00:00 2024-05-08T05:37:44.558 INFO:teuthology.orchestra.run.smithi112.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3538.gbee299f 1.0 MB/s | 142 kB 00:00 2024-05-08T05:37:44.992 INFO:teuthology.orchestra.run.smithi112.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3538.gbee29 7.6 MB/s | 3.5 MB 00:00 2024-05-08T05:37:45.093 INFO:teuthology.orchestra.run.smithi112.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3538.gbe 2.7 MB/s | 274 kB 00:00 2024-05-08T05:37:45.399 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-05-08T05:37:45.399 INFO:teuthology.orchestra.run.smithi112.stdout:(33/156): ceph-mgr-rook-19.0.0-3538.gbee299f4.e 496 kB/s | 49 kB 00:00 2024-05-08T05:37:45.399 INFO:teuthology.orchestra.run.smithi112.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3538.gb 153 kB/s | 16 kB 00:00 2024-05-08T05:37:45.444 INFO:teuthology.orchestra.run.smithi112.stdout:(35/156): ceph-mgr-diskprediction-local-19.0.0- 8.4 MB/s | 7.4 MB 00:00 2024-05-08T05:37:45.486 INFO:teuthology.orchestra.run.smithi112.stdout:(36/156): ceph-volume-19.0.0-3538.gbee299f4.el9 1.5 MB/s | 273 kB 00:00 2024-05-08T05:37:45.586 INFO:teuthology.orchestra.run.smithi112.stdout:(37/156): cephadm-19.0.0-3538.gbee299f4.el9.noa 5.3 MB/s | 763 kB 00:00 2024-05-08T05:37:45.636 INFO:teuthology.orchestra.run.smithi112.stdout:(38/156): cephfs-top-19.0.0-3538.gbee299f4.el9. 122 kB/s | 18 kB 00:00 2024-05-08T05:37:45.795 INFO:teuthology.orchestra.run.smithi112.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 387 kB/s | 80 kB 00:00 2024-05-08T05:37:45.895 INFO:teuthology.orchestra.run.smithi112.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-05-08T05:37:45.937 INFO:teuthology.orchestra.run.smithi112.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-05-08T05:37:45.996 INFO:teuthology.orchestra.run.smithi112.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-05-08T05:37:46.054 INFO:teuthology.orchestra.run.smithi112.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-08T05:37:46.105 INFO:teuthology.orchestra.run.smithi112.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 7.4 MB/s | 1.2 MB 00:00 2024-05-08T05:37:46.138 INFO:teuthology.orchestra.run.smithi112.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-05-08T05:37:46.172 INFO:teuthology.orchestra.run.smithi112.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.9 MB/s | 125 kB 00:00 2024-05-08T05:37:46.206 INFO:teuthology.orchestra.run.smithi112.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-05-08T05:37:46.789 INFO:teuthology.orchestra.run.smithi112.stdout:(48/156): ceph-test-19.0.0-3538.gbee299f4.el9.x 12 MB/s | 49 MB 00:03 2024-05-08T05:37:47.815 INFO:teuthology.orchestra.run.smithi112.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 104 kB/s | 107 kB 00:01 2024-05-08T05:37:47.890 INFO:teuthology.orchestra.run.smithi112.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 401 kB/s | 689 kB 00:01 2024-05-08T05:37:48.015 INFO:teuthology.orchestra.run.smithi112.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 521 kB/s | 943 kB 00:01 2024-05-08T05:37:48.657 INFO:teuthology.orchestra.run.smithi112.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 26 kB/s | 21 kB 00:00 2024-05-08T05:37:48.882 INFO:teuthology.orchestra.run.smithi112.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 38 kB/s | 33 kB 00:00 2024-05-08T05:37:49.116 INFO:teuthology.orchestra.run.smithi112.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 256 kB/s | 313 kB 00:01 2024-05-08T05:37:49.716 INFO:teuthology.orchestra.run.smithi112.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 22 kB/s | 18 kB 00:00 2024-05-08T05:37:50.208 INFO:teuthology.orchestra.run.smithi112.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 10 kB/s | 11 kB 00:01 2024-05-08T05:37:50.384 INFO:teuthology.orchestra.run.smithi112.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.7 MB/s | 3.0 MB 00:01 2024-05-08T05:37:50.536 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-05-08T05:37:50.842 INFO:teuthology.orchestra.run.smithi112.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 142 kB/s | 160 kB 00:01 2024-05-08T05:37:51.535 INFO:teuthology.orchestra.run.smithi112.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 34 kB/s | 45 kB 00:01 2024-05-08T05:37:51.557 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-05-08T05:37:52.051 INFO:teuthology.orchestra.run.smithi112.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 397 kB/s | 662 kB 00:01 2024-05-08T05:37:52.160 INFO:teuthology.orchestra.run.smithi112.stdout:(61/156): libtool-2.4.6-45.el9.x86_64.rpm 442 kB/s | 582 kB 00:01 2024-05-08T05:37:52.760 INFO:teuthology.orchestra.run.smithi112.stdout:(62/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 21 kB/s | 25 kB 00:01 2024-05-08T05:37:52.812 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:37:52.813 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:37:52.823 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T05:37:52.824 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T05:37:52.825 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T05:37:52.826 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T05:37:52.827 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T05:37:52.830 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T05:37:52.830 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T05:37:52.892 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-05-08T05:37:52.902 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-05-08T05:37:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-05-08T05:37:52.902 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-05-08T05:37:52.902 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-05-08T05:37:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T05:37:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 6.6 k 2024-05-08T05:37:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:37:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 890 k 2024-05-08T05:37:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 145 k 2024-05-08T05:37:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 1.5 M 2024-05-08T05:37:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 142 k 2024-05-08T05:37:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 3.5 M 2024-05-08T05:37:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 7.4 M 2024-05-08T05:37:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 49 k 2024-05-08T05:37:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 11 M 2024-05-08T05:37:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 49 M 2024-05-08T05:37:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 273 k 2024-05-08T05:37:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 763 k 2024-05-08T05:37:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 225 k 2024-05-08T05:37:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 27 k 2024-05-08T05:37:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 18 k 2024-05-08T05:37:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T05:37:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T05:37:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T05:37:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 32 k 2024-05-08T05:37:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 747 k 2024-05-08T05:37:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 127 k 2024-05-08T05:37:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T05:37:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T05:37:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T05:37:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T05:37:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T05:37:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T05:37:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 161 k 2024-05-08T05:37:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 321 k 2024-05-08T05:37:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 299 k 2024-05-08T05:37:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 100 k 2024-05-08T05:37:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 85 k 2024-05-08T05:37:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:37:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 171 k 2024-05-08T05:37:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T05:37:52.905 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-05-08T05:37:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T05:37:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T05:37:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T05:37:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T05:37:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.4 M 2024-05-08T05:37:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:37:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T05:37:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T05:37:52.906 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-05-08T05:37:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T05:37:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T05:37:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T05:37:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 21 M 2024-05-08T05:37:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 34 k 2024-05-08T05:37:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 2.4 M 2024-05-08T05:37:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 274 k 2024-05-08T05:37:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 4.7 M 2024-05-08T05:37:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 17 M 2024-05-08T05:37:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 16 k 2024-05-08T05:37:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 25 k 2024-05-08T05:37:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T05:37:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T05:37:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T05:37:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T05:37:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T05:37:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T05:37:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T05:37:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T05:37:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 163 k 2024-05-08T05:37:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T05:37:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T05:37:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T05:37:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T05:37:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T05:37:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 500 k 2024-05-08T05:37:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T05:37:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:37:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T05:37:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T05:37:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T05:37:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T05:37:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T05:37:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T05:37:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T05:37:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T05:37:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T05:37:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T05:37:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T05:37:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T05:37:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T05:37:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T05:37:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T05:37:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T05:37:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T05:37:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 45 k 2024-05-08T05:37:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 135 k 2024-05-08T05:37:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T05:37:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T05:37:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T05:37:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T05:37:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T05:37:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T05:37:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T05:37:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T05:37:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T05:37:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T05:37:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T05:37:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T05:37:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T05:37:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T05:37:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T05:37:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T05:37:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T05:37:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T05:37:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T05:37:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T05:37:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T05:37:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T05:37:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T05:37:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T05:37:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T05:37:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T05:37:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T05:37:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T05:37:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T05:37:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T05:37:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T05:37:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T05:37:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T05:37:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T05:37:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T05:37:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T05:37:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T05:37:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T05:37:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T05:37:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T05:37:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T05:37:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T05:37:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T05:37:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T05:37:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T05:37:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T05:37:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T05:37:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T05:37:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T05:37:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T05:37:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T05:37:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T05:37:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T05:37:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T05:37:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T05:37:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T05:37:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T05:37:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T05:37:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T05:37:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T05:37:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T05:37:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T05:37:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T05:37:52.914 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-05-08T05:37:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T05:37:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-08T05:37:52.915 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-05-08T05:37:52.915 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-05-08T05:37:52.915 INFO:teuthology.orchestra.run.smithi150.stdout:Install 148 Packages 2024-05-08T05:37:52.915 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 8 Packages 2024-05-08T05:37:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-08T05:37:52.915 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 226 M 2024-05-08T05:37:52.916 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-05-08T05:37:53.060 INFO:teuthology.orchestra.run.smithi112.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 47 kB/s | 47 kB 00:01 2024-05-08T05:37:53.444 INFO:teuthology.orchestra.run.smithi112.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 228 kB/s | 292 kB 00:01 2024-05-08T05:37:53.679 INFO:teuthology.orchestra.run.smithi150.stdout:(1/156): ceph-19.0.0-3538.gbee299f4.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-05-08T05:37:53.869 INFO:teuthology.orchestra.run.smithi112.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 170 kB/s | 188 kB 00:01 2024-05-08T05:37:53.871 INFO:teuthology.orchestra.run.smithi150.stdout:(2/156): ceph-fuse-19.0.0-3538.gbee299f4.el9.x8 4.5 MB/s | 890 kB 00:00 2024-05-08T05:37:53.921 INFO:teuthology.orchestra.run.smithi150.stdout:(3/156): ceph-immutable-object-cache-19.0.0-353 2.8 MB/s | 145 kB 00:00 2024-05-08T05:37:54.205 INFO:teuthology.orchestra.run.smithi150.stdout:(4/156): ceph-base-19.0.0-3538.gbee299f4.el9.x8 7.2 MB/s | 5.4 MB 00:00 2024-05-08T05:37:54.255 INFO:teuthology.orchestra.run.smithi150.stdout:(5/156): ceph-mds-19.0.0-3538.gbee299f4.el9.x86 7.1 MB/s | 2.4 MB 00:00 2024-05-08T05:37:54.364 INFO:teuthology.orchestra.run.smithi150.stdout:(6/156): ceph-mgr-19.0.0-3538.gbee299f4.el9.x86 9.6 MB/s | 1.5 MB 00:00 2024-05-08T05:37:54.798 INFO:teuthology.orchestra.run.smithi150.stdout:(7/156): ceph-mon-19.0.0-3538.gbee299f4.el9.x86 8.7 MB/s | 4.7 MB 00:00 2024-05-08T05:37:55.028 INFO:teuthology.orchestra.run.smithi112.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 18 kB/s | 35 kB 00:01 2024-05-08T05:37:55.136 INFO:teuthology.orchestra.run.smithi112.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 2.8 MB/s | 4.7 MB 00:01 2024-05-08T05:37:55.662 INFO:teuthology.orchestra.run.smithi112.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 2.5 MB/s | 4.4 MB 00:01 2024-05-08T05:37:55.682 INFO:teuthology.orchestra.run.smithi150.stdout:(8/156): ceph-osd-19.0.0-3538.gbee299f4.el9.x86 13 MB/s | 17 MB 00:01 2024-05-08T05:37:55.724 INFO:teuthology.orchestra.run.smithi150.stdout:(9/156): ceph-selinux-19.0.0-3538.gbee299f4.el9 608 kB/s | 25 kB 00:00 2024-05-08T05:37:56.307 INFO:teuthology.orchestra.run.smithi150.stdout:(10/156): ceph-radosgw-19.0.0-3538.gbee299f4.el 7.0 MB/s | 11 MB 00:01 2024-05-08T05:37:56.424 INFO:teuthology.orchestra.run.smithi150.stdout:(11/156): cephfs-mirror-19.0.0-3538.gbee299f4.e 1.9 MB/s | 225 kB 00:00 2024-05-08T05:37:56.491 INFO:teuthology.orchestra.run.smithi150.stdout:(12/156): cephfs-shell-19.0.0-3538.gbee299f4.el 406 kB/s | 27 kB 00:00 2024-05-08T05:37:56.545 INFO:teuthology.orchestra.run.smithi112.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 1.0 MB/s | 1.6 MB 00:01 2024-05-08T05:37:56.549 INFO:teuthology.orchestra.run.smithi150.stdout:(13/156): libcephfs-devel-19.0.0-3538.gbee299f4 545 kB/s | 32 kB 00:00 2024-05-08T05:37:56.737 INFO:teuthology.orchestra.run.smithi112.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 570 kB/s | 911 kB 00:01 2024-05-08T05:37:56.738 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-05-08T05:37:56.775 INFO:teuthology.orchestra.run.smithi150.stdout:(14/156): libcephfs2-19.0.0-3538.gbee299f4.el9. 3.2 MB/s | 747 kB 00:00 2024-05-08T05:37:56.850 INFO:teuthology.orchestra.run.smithi150.stdout:(15/156): libcephsqlite-19.0.0-3538.gbee299f4.e 2.1 MB/s | 163 kB 00:00 2024-05-08T05:37:56.925 INFO:teuthology.orchestra.run.smithi150.stdout:(16/156): librados-devel-19.0.0-3538.gbee299f4. 1.7 MB/s | 127 kB 00:00 2024-05-08T05:37:57.071 INFO:teuthology.orchestra.run.smithi112.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 95 kB/s | 133 kB 00:01 2024-05-08T05:37:57.101 INFO:teuthology.orchestra.run.smithi150.stdout:(17/156): libradosstriper1-19.0.0-3538.gbee299f 2.8 MB/s | 500 kB 00:00 2024-05-08T05:37:58.062 INFO:teuthology.orchestra.run.smithi112.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 88 kB/s | 87 kB 00:00 2024-05-08T05:37:58.168 INFO:teuthology.orchestra.run.smithi150.stdout:(18/156): ceph-common-19.0.0-3538.gbee299f4.el9 4.5 MB/s | 21 MB 00:04 2024-05-08T05:37:58.230 INFO:teuthology.orchestra.run.smithi112.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 693 kB/s | 1.0 MB 00:01 2024-05-08T05:37:58.471 INFO:teuthology.orchestra.run.smithi112.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 3.0 MB/s | 5.8 MB 00:01 2024-05-08T05:37:58.610 INFO:teuthology.orchestra.run.smithi150.stdout:(19/156): ceph-test-19.0.0-3538.gbee299f4.el9.x 17 MB/s | 49 MB 00:02 2024-05-08T05:37:58.635 INFO:teuthology.orchestra.run.smithi150.stdout:(20/156): python3-ceph-argparse-19.0.0-3538.gbe 97 kB/s | 45 kB 00:00 2024-05-08T05:37:58.660 INFO:teuthology.orchestra.run.smithi150.stdout:(21/156): python3-ceph-common-19.0.0-3538.gbee2 2.6 MB/s | 135 kB 00:00 2024-05-08T05:37:58.735 INFO:teuthology.orchestra.run.smithi150.stdout:(22/156): librgw2-19.0.0-3538.gbee299f4.el9.x86 3.3 MB/s | 5.4 MB 00:01 2024-05-08T05:37:58.761 INFO:teuthology.orchestra.run.smithi150.stdout:(23/156): python3-cephfs-19.0.0-3538.gbee299f4. 1.3 MB/s | 161 kB 00:00 2024-05-08T05:37:58.786 INFO:teuthology.orchestra.run.smithi150.stdout:(24/156): python3-rados-19.0.0-3538.gbee299f4.e 2.5 MB/s | 321 kB 00:00 2024-05-08T05:37:58.811 INFO:teuthology.orchestra.run.smithi150.stdout:(25/156): python3-rgw-19.0.0-3538.gbee299f4.el9 1.9 MB/s | 100 kB 00:00 2024-05-08T05:37:58.836 INFO:teuthology.orchestra.run.smithi150.stdout:(26/156): rbd-fuse-19.0.0-3538.gbee299f4.el9.x8 1.7 MB/s | 85 kB 00:00 2024-05-08T05:37:58.887 INFO:teuthology.orchestra.run.smithi150.stdout:(27/156): rbd-nbd-19.0.0-3538.gbee299f4.el9.x86 3.3 MB/s | 171 kB 00:00 2024-05-08T05:37:58.929 INFO:teuthology.orchestra.run.smithi150.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3538.g 825 kB/s | 34 kB 00:00 2024-05-08T05:37:58.979 INFO:teuthology.orchestra.run.smithi150.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3538.gbee299f 2.8 MB/s | 142 kB 00:00 2024-05-08T05:37:59.056 INFO:teuthology.orchestra.run.smithi162.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:37:59.057 INFO:teuthology.orchestra.run.smithi162.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:37:59.068 INFO:teuthology.orchestra.run.smithi162.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T05:37:59.069 INFO:teuthology.orchestra.run.smithi162.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T05:37:59.070 INFO:teuthology.orchestra.run.smithi162.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T05:37:59.070 INFO:teuthology.orchestra.run.smithi162.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T05:37:59.072 INFO:teuthology.orchestra.run.smithi162.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T05:37:59.074 INFO:teuthology.orchestra.run.smithi162.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T05:37:59.075 INFO:teuthology.orchestra.run.smithi162.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T05:37:59.113 INFO:teuthology.orchestra.run.smithi150.stdout:(30/156): python3-rbd-19.0.0-3538.gbee299f4.el9 793 kB/s | 299 kB 00:00 2024-05-08T05:37:59.136 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-05-08T05:37:59.163 INFO:teuthology.orchestra.run.smithi162.stdout:======================================================================================= 2024-05-08T05:37:59.163 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-05-08T05:37:59.163 INFO:teuthology.orchestra.run.smithi162.stdout:======================================================================================= 2024-05-08T05:37:59.163 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-05-08T05:37:59.163 INFO:teuthology.orchestra.run.smithi162.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T05:37:59.164 INFO:teuthology.orchestra.run.smithi162.stdout: ceph x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 6.6 k 2024-05-08T05:37:59.164 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:37:59.164 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 890 k 2024-05-08T05:37:59.164 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 145 k 2024-05-08T05:37:59.164 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 1.5 M 2024-05-08T05:37:59.164 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 142 k 2024-05-08T05:37:59.164 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 3.5 M 2024-05-08T05:37:59.164 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 7.4 M 2024-05-08T05:37:59.164 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 49 k 2024-05-08T05:37:59.164 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 11 M 2024-05-08T05:37:59.164 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 49 M 2024-05-08T05:37:59.164 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 273 k 2024-05-08T05:37:59.164 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 763 k 2024-05-08T05:37:59.164 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 225 k 2024-05-08T05:37:59.165 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-shell x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 27 k 2024-05-08T05:37:59.165 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-top noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 18 k 2024-05-08T05:37:59.165 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T05:37:59.165 INFO:teuthology.orchestra.run.smithi162.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T05:37:59.165 INFO:teuthology.orchestra.run.smithi162.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T05:37:59.165 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 32 k 2024-05-08T05:37:59.165 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 747 k 2024-05-08T05:37:59.166 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 127 k 2024-05-08T05:37:59.166 INFO:teuthology.orchestra.run.smithi162.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T05:37:59.166 INFO:teuthology.orchestra.run.smithi162.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T05:37:59.166 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T05:37:59.166 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T05:37:59.166 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T05:37:59.167 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T05:37:59.167 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 161 k 2024-05-08T05:37:59.167 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 321 k 2024-05-08T05:37:59.167 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 299 k 2024-05-08T05:37:59.167 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 100 k 2024-05-08T05:37:59.167 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 85 k 2024-05-08T05:37:59.168 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:37:59.168 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 171 k 2024-05-08T05:37:59.168 INFO:teuthology.orchestra.run.smithi162.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T05:37:59.168 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrading: 2024-05-08T05:37:59.168 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T05:37:59.169 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T05:37:59.169 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T05:37:59.169 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T05:37:59.169 INFO:teuthology.orchestra.run.smithi162.stdout: librados2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.4 M 2024-05-08T05:37:59.169 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:37:59.169 INFO:teuthology.orchestra.run.smithi162.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T05:37:59.169 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T05:37:59.169 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-05-08T05:37:59.170 INFO:teuthology.orchestra.run.smithi162.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T05:37:59.170 INFO:teuthology.orchestra.run.smithi162.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T05:37:59.170 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T05:37:59.170 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 21 M 2024-05-08T05:37:59.170 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 34 k 2024-05-08T05:37:59.170 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 2.4 M 2024-05-08T05:37:59.170 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 274 k 2024-05-08T05:37:59.170 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 4.7 M 2024-05-08T05:37:59.170 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 17 M 2024-05-08T05:37:59.170 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 16 k 2024-05-08T05:37:59.170 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 25 k 2024-05-08T05:37:59.171 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T05:37:59.171 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T05:37:59.171 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T05:37:59.171 INFO:teuthology.orchestra.run.smithi162.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T05:37:59.171 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T05:37:59.171 INFO:teuthology.orchestra.run.smithi162.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T05:37:59.171 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T05:37:59.171 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T05:37:59.171 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 163 k 2024-05-08T05:37:59.171 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T05:37:59.171 INFO:teuthology.orchestra.run.smithi162.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T05:37:59.171 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T05:37:59.172 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T05:37:59.172 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T05:37:59.172 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 500 k 2024-05-08T05:37:59.172 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T05:37:59.172 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:37:59.172 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T05:37:59.172 INFO:teuthology.orchestra.run.smithi162.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T05:37:59.172 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T05:37:59.172 INFO:teuthology.orchestra.run.smithi162.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T05:37:59.172 INFO:teuthology.orchestra.run.smithi162.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T05:37:59.172 INFO:teuthology.orchestra.run.smithi162.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T05:37:59.173 INFO:teuthology.orchestra.run.smithi162.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T05:37:59.173 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T05:37:59.173 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T05:37:59.173 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T05:37:59.173 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T05:37:59.173 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T05:37:59.173 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T05:37:59.173 INFO:teuthology.orchestra.run.smithi162.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T05:37:59.173 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T05:37:59.173 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T05:37:59.173 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T05:37:59.173 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 45 k 2024-05-08T05:37:59.174 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 135 k 2024-05-08T05:37:59.174 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T05:37:59.174 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T05:37:59.174 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T05:37:59.174 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T05:37:59.174 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T05:37:59.174 INFO:teuthology.orchestra.run.smithi162.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T05:37:59.174 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T05:37:59.174 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T05:37:59.174 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T05:37:59.174 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T05:37:59.174 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T05:37:59.175 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T05:37:59.175 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T05:37:59.175 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T05:37:59.175 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T05:37:59.175 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T05:37:59.175 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T05:37:59.175 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T05:37:59.175 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T05:37:59.175 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T05:37:59.175 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T05:37:59.175 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T05:37:59.175 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T05:37:59.175 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T05:37:59.175 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T05:37:59.176 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T05:37:59.176 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T05:37:59.176 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T05:37:59.176 INFO:teuthology.orchestra.run.smithi162.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T05:37:59.176 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T05:37:59.176 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T05:37:59.176 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T05:37:59.176 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T05:37:59.176 INFO:teuthology.orchestra.run.smithi162.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T05:37:59.176 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T05:37:59.177 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T05:37:59.177 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T05:37:59.177 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T05:37:59.177 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T05:37:59.177 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T05:37:59.177 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T05:37:59.177 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T05:37:59.177 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T05:37:59.177 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T05:37:59.178 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T05:37:59.178 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T05:37:59.178 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T05:37:59.178 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T05:37:59.178 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T05:37:59.178 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T05:37:59.178 INFO:teuthology.orchestra.run.smithi162.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T05:37:59.178 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T05:37:59.178 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T05:37:59.178 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T05:37:59.178 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T05:37:59.179 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T05:37:59.179 INFO:teuthology.orchestra.run.smithi162.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T05:37:59.179 INFO:teuthology.orchestra.run.smithi162.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T05:37:59.179 INFO:teuthology.orchestra.run.smithi162.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T05:37:59.179 INFO:teuthology.orchestra.run.smithi162.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T05:37:59.179 INFO:teuthology.orchestra.run.smithi162.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T05:37:59.179 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T05:37:59.179 INFO:teuthology.orchestra.run.smithi162.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T05:37:59.179 INFO:teuthology.orchestra.run.smithi162.stdout:Installing weak dependencies: 2024-05-08T05:37:59.179 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T05:37:59.179 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:37:59.179 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-05-08T05:37:59.179 INFO:teuthology.orchestra.run.smithi162.stdout:======================================================================================= 2024-05-08T05:37:59.179 INFO:teuthology.orchestra.run.smithi162.stdout:Install 148 Packages 2024-05-08T05:37:59.180 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrade 8 Packages 2024-05-08T05:37:59.180 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:37:59.180 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 226 M 2024-05-08T05:37:59.180 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-05-08T05:37:59.221 INFO:teuthology.orchestra.run.smithi150.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3538.gbee29 15 MB/s | 3.5 MB 00:00 2024-05-08T05:37:59.280 INFO:teuthology.orchestra.run.smithi150.stdout:(32/156): rbd-mirror-19.0.0-3538.gbee299f4.el9. 6.6 MB/s | 3.1 MB 00:00 2024-05-08T05:37:59.305 INFO:teuthology.orchestra.run.smithi150.stdout:(33/156): ceph-mgr-modules-core-19.0.0-3538.gbe 3.2 MB/s | 274 kB 00:00 2024-05-08T05:37:59.331 INFO:teuthology.orchestra.run.smithi150.stdout:(34/156): ceph-mgr-rook-19.0.0-3538.gbee299f4.e 984 kB/s | 49 kB 00:00 2024-05-08T05:37:59.356 INFO:teuthology.orchestra.run.smithi150.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3538.gb 327 kB/s | 16 kB 00:00 2024-05-08T05:37:59.406 INFO:teuthology.orchestra.run.smithi150.stdout:(36/156): ceph-volume-19.0.0-3538.gbee299f4.el9 3.5 MB/s | 273 kB 00:00 2024-05-08T05:37:59.440 INFO:teuthology.orchestra.run.smithi150.stdout:(37/156): cephadm-19.0.0-3538.gbee299f4.el9.noa 8.9 MB/s | 763 kB 00:00 2024-05-08T05:37:59.465 INFO:teuthology.orchestra.run.smithi150.stdout:(38/156): cephfs-top-19.0.0-3538.gbee299f4.el9. 312 kB/s | 18 kB 00:00 2024-05-08T05:37:59.497 INFO:teuthology.orchestra.run.smithi112.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 193 kB/s | 244 kB 00:01 2024-05-08T05:37:59.722 INFO:teuthology.orchestra.run.smithi112.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 209 kB/s | 248 kB 00:01 2024-05-08T05:37:59.997 INFO:teuthology.orchestra.run.smithi112.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.1 MB/s | 6.0 MB 00:01 2024-05-08T05:38:00.132 INFO:teuthology.orchestra.run.smithi150.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 7.3 MB/s | 7.4 MB 00:01 2024-05-08T05:38:00.356 INFO:teuthology.orchestra.run.smithi112.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 30 kB/s | 26 kB 00:00 2024-05-08T05:38:00.565 INFO:teuthology.orchestra.run.smithi162.stdout:(1/156): ceph-19.0.0-3538.gbee299f4.el9.x86_64. 9.3 kB/s | 6.6 kB 00:00 2024-05-08T05:38:00.631 INFO:teuthology.orchestra.run.smithi112.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 21 kB/s | 19 kB 00:00 2024-05-08T05:38:00.766 INFO:teuthology.orchestra.run.smithi150.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 61 kB/s | 80 kB 00:01 2024-05-08T05:38:00.898 INFO:teuthology.orchestra.run.smithi162.stdout:(2/156): ceph-fuse-19.0.0-3538.gbee299f4.el9.x8 2.6 MB/s | 890 kB 00:00 2024-05-08T05:38:00.965 INFO:teuthology.orchestra.run.smithi162.stdout:(3/156): ceph-immutable-object-cache-19.0.0-353 2.1 MB/s | 145 kB 00:00 2024-05-08T05:38:01.164 INFO:teuthology.orchestra.run.smithi112.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 148 kB/s | 172 kB 00:01 2024-05-08T05:38:01.174 INFO:teuthology.orchestra.run.smithi150.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 470 kB/s | 803 kB 00:01 2024-05-08T05:38:01.199 INFO:teuthology.orchestra.run.smithi150.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 183 kB/s | 195 kB 00:01 2024-05-08T05:38:01.206 INFO:teuthology.orchestra.run.smithi112.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 41 kB/s | 35 kB 00:00 2024-05-08T05:38:01.357 INFO:teuthology.orchestra.run.smithi162.stdout:(4/156): ceph-base-19.0.0-3538.gbee299f4.el9.x8 3.6 MB/s | 5.4 MB 00:01 2024-05-08T05:38:01.432 INFO:teuthology.orchestra.run.smithi162.stdout:(5/156): ceph-mds-19.0.0-3538.gbee299f4.el9.x86 5.1 MB/s | 2.4 MB 00:00 2024-05-08T05:38:01.991 INFO:teuthology.orchestra.run.smithi162.stdout:(6/156): ceph-common-19.0.0-3538.gbee299f4.el9. 9.9 MB/s | 21 MB 00:02 2024-05-08T05:38:02.032 INFO:teuthology.orchestra.run.smithi162.stdout:(7/156): ceph-mgr-19.0.0-3538.gbee299f4.el9.x86 2.2 MB/s | 1.5 MB 00:00 2024-05-08T05:38:02.150 INFO:teuthology.orchestra.run.smithi150.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 112 kB/s | 106 kB 00:00 2024-05-08T05:38:02.282 INFO:teuthology.orchestra.run.smithi112.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.1 MB/s | 5.1 MB 00:01 2024-05-08T05:38:02.315 INFO:teuthology.orchestra.run.smithi112.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 218 kB/s | 250 kB 00:01 2024-05-08T05:38:02.374 INFO:teuthology.orchestra.run.smithi112.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 190 kB/s | 222 kB 00:01 2024-05-08T05:38:02.483 INFO:teuthology.orchestra.run.smithi150.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 959 kB/s | 1.2 MB 00:01 2024-05-08T05:38:02.491 INFO:teuthology.orchestra.run.smithi162.stdout:(8/156): ceph-mon-19.0.0-3538.gbee299f4.el9.x86 4.4 MB/s | 4.7 MB 00:01 2024-05-08T05:38:02.567 INFO:teuthology.orchestra.run.smithi162.stdout:(9/156): ceph-selinux-19.0.0-3538.gbee299f4.el9 336 kB/s | 25 kB 00:00 2024-05-08T05:38:02.917 INFO:teuthology.orchestra.run.smithi162.stdout:(10/156): ceph-osd-19.0.0-3538.gbee299f4.el9.x8 18 MB/s | 17 MB 00:00 2024-05-08T05:38:02.967 INFO:teuthology.orchestra.run.smithi150.stdout:(45/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 115 kB/s | 253 kB 00:02 2024-05-08T05:38:02.983 INFO:teuthology.orchestra.run.smithi162.stdout:(11/156): cephfs-mirror-19.0.0-3538.gbee299f4.e 3.3 MB/s | 225 kB 00:00 2024-05-08T05:38:03.042 INFO:teuthology.orchestra.run.smithi162.stdout:(12/156): cephfs-shell-19.0.0-3538.gbee299f4.el 464 kB/s | 27 kB 00:00 2024-05-08T05:38:03.117 INFO:teuthology.orchestra.run.smithi162.stdout:(13/156): libcephfs-devel-19.0.0-3538.gbee299f4 423 kB/s | 32 kB 00:00 2024-05-08T05:38:03.209 INFO:teuthology.orchestra.run.smithi162.stdout:(14/156): libcephfs2-19.0.0-3538.gbee299f4.el9. 8.0 MB/s | 747 kB 00:00 2024-05-08T05:38:03.242 INFO:teuthology.orchestra.run.smithi150.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 127 kB/s | 135 kB 00:01 2024-05-08T05:38:03.275 INFO:teuthology.orchestra.run.smithi112.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 44 kB/s | 42 kB 00:00 2024-05-08T05:38:03.293 INFO:teuthology.orchestra.run.smithi162.stdout:(15/156): libcephsqlite-19.0.0-3538.gbee299f4.e 1.9 MB/s | 163 kB 00:00 2024-05-08T05:38:03.308 INFO:teuthology.orchestra.run.smithi112.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 75 kB/s | 77 kB 00:01 2024-05-08T05:38:03.368 INFO:teuthology.orchestra.run.smithi162.stdout:(16/156): librados-devel-19.0.0-3538.gbee299f4. 1.6 MB/s | 127 kB 00:00 2024-05-08T05:38:03.452 INFO:teuthology.orchestra.run.smithi162.stdout:(17/156): libradosstriper1-19.0.0-3538.gbee299f 5.9 MB/s | 500 kB 00:00 2024-05-08T05:38:03.509 INFO:teuthology.orchestra.run.smithi150.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-05-08T05:38:03.534 INFO:teuthology.orchestra.run.smithi150.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 122 kB/s | 125 kB 00:01 2024-05-08T05:38:03.610 INFO:teuthology.orchestra.run.smithi150.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 9.3 MB/s | 943 kB 00:00 2024-05-08T05:38:03.630 INFO:teuthology.orchestra.run.smithi112.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 215 kB/s | 269 kB 00:01 2024-05-08T05:38:03.668 INFO:teuthology.orchestra.run.smithi150.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 370 kB/s | 21 kB 00:00 2024-05-08T05:38:03.718 INFO:teuthology.orchestra.run.smithi150.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 582 kB/s | 107 kB 00:00 2024-05-08T05:38:03.744 INFO:teuthology.orchestra.run.smithi150.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-05-08T05:38:03.777 INFO:teuthology.orchestra.run.smithi150.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-05-08T05:38:03.836 INFO:teuthology.orchestra.run.smithi150.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-05-08T05:38:03.886 INFO:teuthology.orchestra.run.smithi150.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-05-08T05:38:03.903 INFO:teuthology.orchestra.run.smithi150.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-05-08T05:38:03.945 INFO:teuthology.orchestra.run.smithi150.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-05-08T05:38:03.962 INFO:teuthology.orchestra.run.smithi150.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-05-08T05:38:04.012 INFO:teuthology.orchestra.run.smithi150.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-05-08T05:38:04.046 INFO:teuthology.orchestra.run.smithi150.stdout:(60/156): python3-urllib3-1.26.5-5.el9.noarch.r 199 kB/s | 215 kB 00:01 2024-05-08T05:38:04.052 INFO:teuthology.orchestra.run.smithi162.stdout:(18/156): librgw2-19.0.0-3538.gbee299f4.el9.x86 8.9 MB/s | 5.4 MB 00:00 2024-05-08T05:38:04.088 INFO:teuthology.orchestra.run.smithi150.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-05-08T05:38:04.113 INFO:teuthology.orchestra.run.smithi150.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-05-08T05:38:04.118 INFO:teuthology.orchestra.run.smithi162.stdout:(19/156): python3-ceph-argparse-19.0.0-3538.gbe 679 kB/s | 45 kB 00:00 2024-05-08T05:38:04.147 INFO:teuthology.orchestra.run.smithi150.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-05-08T05:38:04.172 INFO:teuthology.orchestra.run.smithi150.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-05-08T05:38:04.177 INFO:teuthology.orchestra.run.smithi162.stdout:(20/156): python3-ceph-common-19.0.0-3538.gbee2 2.3 MB/s | 135 kB 00:00 2024-05-08T05:38:04.206 INFO:teuthology.orchestra.run.smithi150.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 600 kB/s | 35 kB 00:00 2024-05-08T05:38:04.231 INFO:teuthology.orchestra.run.smithi150.stdout:(66/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 257 kB/s | 47 kB 00:00 2024-05-08T05:38:04.236 INFO:teuthology.orchestra.run.smithi162.stdout:(21/156): python3-cephfs-19.0.0-3538.gbee299f4. 2.7 MB/s | 161 kB 00:00 2024-05-08T05:38:04.303 INFO:teuthology.orchestra.run.smithi162.stdout:(22/156): python3-rados-19.0.0-3538.gbee299f4.e 4.7 MB/s | 321 kB 00:00 2024-05-08T05:38:04.403 INFO:teuthology.orchestra.run.smithi162.stdout:(23/156): python3-rbd-19.0.0-3538.gbee299f4.el9 2.9 MB/s | 299 kB 00:00 2024-05-08T05:38:04.406 INFO:teuthology.orchestra.run.smithi150.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-05-08T05:38:04.483 INFO:teuthology.orchestra.run.smithi112.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 131 kB/s | 159 kB 00:01 2024-05-08T05:38:04.490 INFO:teuthology.orchestra.run.smithi150.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-05-08T05:38:04.545 INFO:teuthology.orchestra.run.smithi162.stdout:(24/156): ceph-radosgw-19.0.0-3538.gbee299f4.el 4.2 MB/s | 11 MB 00:02 2024-05-08T05:38:04.570 INFO:teuthology.orchestra.run.smithi162.stdout:(25/156): python3-rgw-19.0.0-3538.gbee299f4.el9 597 kB/s | 100 kB 00:00 2024-05-08T05:38:04.578 INFO:teuthology.orchestra.run.smithi150.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 1.5 MB/s | 133 kB 00:00 2024-05-08T05:38:04.587 INFO:teuthology.orchestra.run.smithi162.stdout:(26/156): rbd-fuse-19.0.0-3538.gbee299f4.el9.x8 2.0 MB/s | 85 kB 00:00 2024-05-08T05:38:04.617 INFO:teuthology.orchestra.run.smithi112.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 213 kB/s | 279 kB 00:01 2024-05-08T05:38:04.679 INFO:teuthology.orchestra.run.smithi162.stdout:(27/156): rbd-nbd-19.0.0-3538.gbee299f4.el9.x86 1.8 MB/s | 171 kB 00:00 2024-05-08T05:38:04.740 INFO:teuthology.orchestra.run.smithi150.stdout:(70/156): openblas-openmp-0.3.21-2.el9.x86_64.r 8.3 MB/s | 4.7 MB 00:00 2024-05-08T05:38:04.746 INFO:teuthology.orchestra.run.smithi162.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3538.g 513 kB/s | 34 kB 00:00 2024-05-08T05:38:04.805 INFO:teuthology.orchestra.run.smithi162.stdout:(29/156): rbd-mirror-19.0.0-3538.gbee299f4.el9. 13 MB/s | 3.1 MB 00:00 2024-05-08T05:38:04.816 INFO:teuthology.orchestra.run.smithi150.stdout:(71/156): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-05-08T05:38:04.830 INFO:teuthology.orchestra.run.smithi162.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3538.gbee299f 1.7 MB/s | 142 kB 00:00 2024-05-08T05:38:04.883 INFO:teuthology.orchestra.run.smithi150.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-05-08T05:38:04.908 INFO:teuthology.orchestra.run.smithi150.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 953 kB/s | 87 kB 00:00 2024-05-08T05:38:04.950 INFO:teuthology.orchestra.run.smithi150.stdout:(74/156): postgresql-13.14-1.el9.x86_64.rpm 2.2 MB/s | 1.6 MB 00:00 2024-05-08T05:38:04.975 INFO:teuthology.orchestra.run.smithi150.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-05-08T05:38:05.034 INFO:teuthology.orchestra.run.smithi150.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-05-08T05:38:05.047 INFO:teuthology.orchestra.run.smithi162.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3538.gbee29 15 MB/s | 3.5 MB 00:00 2024-05-08T05:38:05.059 INFO:teuthology.orchestra.run.smithi150.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-05-08T05:38:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 39 kB/s | 51 kB 00:01 2024-05-08T05:38:05.093 INFO:teuthology.orchestra.run.smithi150.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-05-08T05:38:05.114 INFO:teuthology.orchestra.run.smithi162.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3538.gbe 4.0 MB/s | 274 kB 00:00 2024-05-08T05:38:05.152 INFO:teuthology.orchestra.run.smithi150.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-05-08T05:38:05.177 INFO:teuthology.orchestra.run.smithi150.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 415 kB/s | 35 kB 00:00 2024-05-08T05:38:05.181 INFO:teuthology.orchestra.run.smithi162.stdout:(33/156): ceph-mgr-rook-19.0.0-3538.gbee299f4.e 740 kB/s | 49 kB 00:00 2024-05-08T05:38:05.235 INFO:teuthology.orchestra.run.smithi150.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-05-08T05:38:05.239 INFO:teuthology.orchestra.run.smithi162.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3538.gb 282 kB/s | 16 kB 00:00 2024-05-08T05:38:05.294 INFO:teuthology.orchestra.run.smithi150.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-05-08T05:38:05.315 INFO:teuthology.orchestra.run.smithi162.stdout:(35/156): ceph-volume-19.0.0-3538.gbee299f4.el9 3.6 MB/s | 273 kB 00:00 2024-05-08T05:38:05.411 INFO:teuthology.orchestra.run.smithi150.stdout:(83/156): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-05-08T05:38:05.423 INFO:teuthology.orchestra.run.smithi162.stdout:(36/156): cephadm-19.0.0-3538.gbee299f4.el9.noa 6.9 MB/s | 763 kB 00:00 2024-05-08T05:38:05.436 INFO:teuthology.orchestra.run.smithi150.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 544 kB/s | 77 kB 00:00 2024-05-08T05:38:05.478 INFO:teuthology.orchestra.run.smithi150.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 632 kB/s | 42 kB 00:00 2024-05-08T05:38:05.482 INFO:teuthology.orchestra.run.smithi162.stdout:(37/156): cephfs-top-19.0.0-3538.gbee299f4.el9. 312 kB/s | 18 kB 00:00 2024-05-08T05:38:05.484 INFO:teuthology.orchestra.run.smithi112.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 54 kB/s | 54 kB 00:00 2024-05-08T05:38:05.504 INFO:teuthology.orchestra.run.smithi150.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-05-08T05:38:05.554 INFO:teuthology.orchestra.run.smithi150.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-05-08T05:38:05.579 INFO:teuthology.orchestra.run.smithi150.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-05-08T05:38:05.621 INFO:teuthology.orchestra.run.smithi150.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 762 kB/s | 51 kB 00:00 2024-05-08T05:38:05.634 INFO:teuthology.orchestra.run.smithi162.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-05-08T05:38:05.646 INFO:teuthology.orchestra.run.smithi150.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 802 kB/s | 54 kB 00:00 2024-05-08T05:38:05.705 INFO:teuthology.orchestra.run.smithi150.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 716 kB/s | 42 kB 00:00 2024-05-08T05:38:05.764 INFO:teuthology.orchestra.run.smithi150.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 754 kB/s | 44 kB 00:00 2024-05-08T05:38:05.801 INFO:teuthology.orchestra.run.smithi162.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.7 MB/s | 803 kB 00:00 2024-05-08T05:38:05.831 INFO:teuthology.orchestra.run.smithi150.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-05-08T05:38:05.881 INFO:teuthology.orchestra.run.smithi150.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-05-08T05:38:05.948 INFO:teuthology.orchestra.run.smithi150.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 711 kB/s | 47 kB 00:00 2024-05-08T05:38:05.952 INFO:teuthology.orchestra.run.smithi112.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 47 kB/s | 42 kB 00:00 2024-05-08T05:38:06.008 INFO:teuthology.orchestra.run.smithi162.stdout:(40/156): ceph-mgr-diskprediction-local-19.0.0- 6.3 MB/s | 7.4 MB 00:01 2024-05-08T05:38:06.023 INFO:teuthology.orchestra.run.smithi150.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-05-08T05:38:06.067 INFO:teuthology.orchestra.run.smithi162.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 731 kB/s | 195 kB 00:00 2024-05-08T05:38:06.091 INFO:teuthology.orchestra.run.smithi150.stdout:(97/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.4 MB/s | 5.1 MB 00:00 2024-05-08T05:38:06.116 INFO:teuthology.orchestra.run.smithi150.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 630 kB/s | 58 kB 00:00 2024-05-08T05:38:06.157 INFO:teuthology.orchestra.run.smithi150.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-05-08T05:38:06.174 INFO:teuthology.orchestra.run.smithi150.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-08T05:38:06.208 INFO:teuthology.orchestra.run.smithi150.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-05-08T05:38:06.208 INFO:teuthology.orchestra.run.smithi162.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-05-08T05:38:06.267 INFO:teuthology.orchestra.run.smithi162.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-08T05:38:06.326 INFO:teuthology.orchestra.run.smithi162.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 4.7 MB/s | 1.2 MB 00:00 2024-05-08T05:38:06.351 INFO:teuthology.orchestra.run.smithi162.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-05-08T05:38:06.410 INFO:teuthology.orchestra.run.smithi162.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-08T05:38:06.475 INFO:teuthology.orchestra.run.smithi150.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-05-08T05:38:06.476 INFO:teuthology.orchestra.run.smithi112.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 45 kB/s | 44 kB 00:00 2024-05-08T05:38:06.492 INFO:teuthology.orchestra.run.smithi150.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 70 kB/s | 22 kB 00:00 2024-05-08T05:38:06.626 INFO:teuthology.orchestra.run.smithi162.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 780 kB/s | 215 kB 00:00 2024-05-08T05:38:07.142 INFO:teuthology.orchestra.run.smithi150.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 475 kB/s | 309 kB 00:00 2024-05-08T05:38:07.152 INFO:teuthology.orchestra.run.smithi162.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 928 kB/s | 689 kB 00:00 2024-05-08T05:38:07.192 INFO:teuthology.orchestra.run.smithi150.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 765 kB/s | 548 kB 00:00 2024-05-08T05:38:07.210 INFO:teuthology.orchestra.run.smithi112.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.3 MB/s | 16 MB 00:02 2024-05-08T05:38:07.234 INFO:teuthology.orchestra.run.smithi150.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-05-08T05:38:07.244 INFO:teuthology.orchestra.run.smithi112.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 171 kB/s | 221 kB 00:01 2024-05-08T05:38:07.260 INFO:teuthology.orchestra.run.smithi162.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 988 kB/s | 107 kB 00:00 2024-05-08T05:38:07.377 INFO:teuthology.orchestra.run.smithi162.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 184 kB/s | 21 kB 00:00 2024-05-08T05:38:07.486 INFO:teuthology.orchestra.run.smithi162.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-08T05:38:07.520 INFO:teuthology.orchestra.run.smithi162.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-05-08T05:38:07.526 INFO:teuthology.orchestra.run.smithi150.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 89 kB/s | 26 kB 00:00 2024-05-08T05:38:07.595 INFO:teuthology.orchestra.run.smithi162.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-05-08T05:38:07.635 INFO:teuthology.orchestra.run.smithi150.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-05-08T05:38:07.695 INFO:teuthology.orchestra.run.smithi162.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-05-08T05:38:07.744 INFO:teuthology.orchestra.run.smithi150.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 622 kB/s | 67 kB 00:00 2024-05-08T05:38:07.795 INFO:teuthology.orchestra.run.smithi162.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-05-08T05:38:07.811 INFO:teuthology.orchestra.run.smithi150.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-05-08T05:38:07.869 INFO:teuthology.orchestra.run.smithi150.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-05-08T05:38:07.912 INFO:teuthology.orchestra.run.smithi162.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.6 MB/s | 3.0 MB 00:00 2024-05-08T05:38:07.928 INFO:teuthology.orchestra.run.smithi150.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-05-08T05:38:07.938 INFO:teuthology.orchestra.run.smithi162.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-05-08T05:38:07.985 INFO:teuthology.orchestra.run.smithi112.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 24 kB/s | 36 kB 00:01 2024-05-08T05:38:08.021 INFO:teuthology.orchestra.run.smithi162.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-05-08T05:38:08.028 INFO:teuthology.orchestra.run.smithi150.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 435 kB/s | 43 kB 00:00 2024-05-08T05:38:08.062 INFO:teuthology.orchestra.run.smithi150.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-05-08T05:38:08.129 INFO:teuthology.orchestra.run.smithi150.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-05-08T05:38:08.138 INFO:teuthology.orchestra.run.smithi162.stdout:(59/156): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-05-08T05:38:08.162 INFO:teuthology.orchestra.run.smithi150.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-05-08T05:38:08.163 INFO:teuthology.orchestra.run.smithi162.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-05-08T05:38:08.238 INFO:teuthology.orchestra.run.smithi150.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-05-08T05:38:08.238 INFO:teuthology.orchestra.run.smithi162.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 253 kB/s | 25 kB 00:00 2024-05-08T05:38:08.272 INFO:teuthology.orchestra.run.smithi162.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-05-08T05:38:08.280 INFO:teuthology.orchestra.run.smithi150.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-05-08T05:38:08.347 INFO:teuthology.orchestra.run.smithi150.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-05-08T05:38:08.347 INFO:teuthology.orchestra.run.smithi162.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-08T05:38:08.381 INFO:teuthology.orchestra.run.smithi150.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 351 kB/s | 35 kB 00:00 2024-05-08T05:38:08.381 INFO:teuthology.orchestra.run.smithi162.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-08T05:38:08.419 INFO:teuthology.orchestra.run.smithi112.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 260 kB/s | 305 kB 00:01 2024-05-08T05:38:08.447 INFO:teuthology.orchestra.run.smithi150.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-08T05:38:08.448 INFO:teuthology.orchestra.run.smithi162.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-05-08T05:38:08.478 INFO:teuthology.orchestra.run.smithi112.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 38 kB/s | 47 kB 00:01 2024-05-08T05:38:08.564 INFO:teuthology.orchestra.run.smithi150.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-05-08T05:38:08.631 INFO:teuthology.orchestra.run.smithi150.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-05-08T05:38:08.657 INFO:teuthology.orchestra.run.smithi150.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-05-08T05:38:08.724 INFO:teuthology.orchestra.run.smithi150.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-05-08T05:38:08.749 INFO:teuthology.orchestra.run.smithi150.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-05-08T05:38:08.816 INFO:teuthology.orchestra.run.smithi150.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-05-08T05:38:08.841 INFO:teuthology.orchestra.run.smithi150.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-05-08T05:38:08.895 INFO:teuthology.orchestra.run.smithi112.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 64 kB/s | 58 kB 00:00 2024-05-08T05:38:08.941 INFO:teuthology.orchestra.run.smithi150.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-05-08T05:38:08.975 INFO:teuthology.orchestra.run.smithi150.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-05-08T05:38:09.042 INFO:teuthology.orchestra.run.smithi150.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 786 kB/s | 79 kB 00:00 2024-05-08T05:38:09.076 INFO:teuthology.orchestra.run.smithi150.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 577 kB/s | 58 kB 00:00 2024-05-08T05:38:09.151 INFO:teuthology.orchestra.run.smithi150.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-05-08T05:38:09.168 INFO:teuthology.orchestra.run.smithi150.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-05-08T05:38:09.251 INFO:teuthology.orchestra.run.smithi150.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 898 kB/s | 90 kB 00:00 2024-05-08T05:38:09.277 INFO:teuthology.orchestra.run.smithi150.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-05-08T05:38:09.352 INFO:teuthology.orchestra.run.smithi150.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-05-08T05:38:09.369 INFO:teuthology.orchestra.run.smithi150.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-05-08T05:38:09.395 INFO:teuthology.orchestra.run.smithi112.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 70 kB/s | 64 kB 00:00 2024-05-08T05:38:09.444 INFO:teuthology.orchestra.run.smithi150.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 644 kB/s | 59 kB 00:00 2024-05-08T05:38:09.445 INFO:teuthology.orchestra.run.smithi112.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 453 kB/s | 22 kB 00:00 2024-05-08T05:38:09.478 INFO:teuthology.orchestra.run.smithi150.stdout:(140/156): protobuf-compiler-3.14.0-13.el9.x86_ 264 kB/s | 863 kB 00:03 2024-05-08T05:38:09.503 INFO:teuthology.orchestra.run.smithi150.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 267 kB/s | 36 kB 00:00 2024-05-08T05:38:09.545 INFO:teuthology.orchestra.run.smithi150.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-05-08T05:38:09.554 INFO:teuthology.orchestra.run.smithi112.stdout:(102/156): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-05-08T05:38:09.587 INFO:teuthology.orchestra.run.smithi150.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-05-08T05:38:09.654 INFO:teuthology.orchestra.run.smithi150.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-05-08T05:38:09.679 INFO:teuthology.orchestra.run.smithi150.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-05-08T05:38:09.763 INFO:teuthology.orchestra.run.smithi150.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-08T05:38:09.779 INFO:teuthology.orchestra.run.smithi112.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 51 kB/s | 45 kB 00:00 2024-05-08T05:38:09.838 INFO:teuthology.orchestra.run.smithi150.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-05-08T05:38:09.922 INFO:teuthology.orchestra.run.smithi150.stdout:(148/156): python3-websocket-client-1.2.3-2.el9 214 kB/s | 90 kB 00:00 2024-05-08T05:38:10.156 INFO:teuthology.orchestra.run.smithi150.stdout:(149/156): librbd1-19.0.0-3538.gbee299f4.el9.x8 9.9 MB/s | 3.1 MB 00:00 2024-05-08T05:38:10.171 INFO:teuthology.orchestra.run.smithi112.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 889 kB/s | 548 kB 00:00 2024-05-08T05:38:10.274 INFO:teuthology.orchestra.run.smithi112.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 196 kB/s | 19 kB 00:00 2024-05-08T05:38:10.306 INFO:teuthology.orchestra.run.smithi150.stdout:(150/156): librados2-19.0.0-3538.gbee299f4.el9. 6.3 MB/s | 3.4 MB 00:00 2024-05-08T05:38:10.422 INFO:teuthology.orchestra.run.smithi112.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 542 kB/s | 309 kB 00:00 2024-05-08T05:38:10.464 INFO:teuthology.orchestra.run.smithi112.stdout:(107/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 46 kB/s | 94 kB 00:02 2024-05-08T05:38:10.522 INFO:teuthology.orchestra.run.smithi112.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 261 kB/s | 26 kB 00:00 2024-05-08T05:38:10.631 INFO:teuthology.orchestra.run.smithi112.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 625 kB/s | 67 kB 00:00 2024-05-08T05:38:10.740 INFO:teuthology.orchestra.run.smithi162.stdout:(66/156): ceph-test-19.0.0-3538.gbee299f4.el9.x 6.0 MB/s | 49 MB 00:08 2024-05-08T05:38:10.752 INFO:teuthology.orchestra.run.smithi112.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.2 MB/s | 4.4 MB 00:00 2024-05-08T05:38:10.790 INFO:teuthology.orchestra.run.smithi112.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 958 kB/s | 151 kB 00:00 2024-05-08T05:38:10.815 INFO:teuthology.orchestra.run.smithi112.stdout:(112/156): liboath-2.6.7-2.el9.x86_64.rpm 139 kB/s | 49 kB 00:00 2024-05-08T05:38:10.882 INFO:teuthology.orchestra.run.smithi112.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.4 MB/s | 839 kB 00:00 2024-05-08T05:38:10.984 INFO:teuthology.orchestra.run.smithi112.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 324 kB/s | 32 kB 00:00 2024-05-08T05:38:11.016 INFO:teuthology.orchestra.run.smithi112.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-05-08T05:38:11.058 INFO:teuthology.orchestra.run.smithi112.stdout:(116/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 179 kB/s | 43 kB 00:00 2024-05-08T05:38:11.084 INFO:teuthology.orchestra.run.smithi112.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-05-08T05:38:11.125 INFO:teuthology.orchestra.run.smithi112.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-05-08T05:38:11.201 INFO:teuthology.orchestra.run.smithi112.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-05-08T05:38:11.226 INFO:teuthology.orchestra.run.smithi112.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 354 kB/s | 35 kB 00:00 2024-05-08T05:38:11.301 INFO:teuthology.orchestra.run.smithi112.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-08T05:38:11.343 INFO:teuthology.orchestra.run.smithi112.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.1 MB/s | 303 kB 00:00 2024-05-08T05:38:11.393 INFO:teuthology.orchestra.run.smithi112.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-05-08T05:38:11.435 INFO:teuthology.orchestra.run.smithi112.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-05-08T05:38:11.489 INFO:teuthology.orchestra.run.smithi150.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 1.5 MB/s | 2.3 MB 00:01 2024-05-08T05:38:11.502 INFO:teuthology.orchestra.run.smithi112.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.4 MB/s | 2.0 MB 00:00 2024-05-08T05:38:11.528 INFO:teuthology.orchestra.run.smithi112.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 132 kB/s | 18 kB 00:00 2024-05-08T05:38:11.558 INFO:teuthology.orchestra.run.smithi150.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 34 kB/s | 42 kB 00:01 2024-05-08T05:38:11.581 INFO:teuthology.orchestra.run.smithi162.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-05-08T05:38:11.600 INFO:teuthology.orchestra.run.smithi112.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 141 kB/s | 23 kB 00:00 2024-05-08T05:38:11.628 INFO:teuthology.orchestra.run.smithi112.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 156 kB/s | 19 kB 00:00 2024-05-08T05:38:11.648 INFO:teuthology.orchestra.run.smithi162.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 1.5 MB/s | 4.7 MB 00:03 2024-05-08T05:38:11.653 INFO:teuthology.orchestra.run.smithi112.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2024-05-08T05:38:11.699 INFO:teuthology.orchestra.run.smithi162.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-05-08T05:38:11.754 INFO:teuthology.orchestra.run.smithi112.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 468 kB/s | 46 kB 00:00 2024-05-08T05:38:11.779 INFO:teuthology.orchestra.run.smithi112.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 635 kB/s | 79 kB 00:00 2024-05-08T05:38:11.854 INFO:teuthology.orchestra.run.smithi112.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 580 kB/s | 58 kB 00:00 2024-05-08T05:38:11.896 INFO:teuthology.orchestra.run.smithi112.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-05-08T05:38:11.930 INFO:teuthology.orchestra.run.smithi112.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-05-08T05:38:11.949 INFO:teuthology.orchestra.run.smithi162.stdout:(70/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 4.4 MB 00:03 2024-05-08T05:38:11.955 INFO:teuthology.orchestra.run.smithi112.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-05-08T05:38:11.974 INFO:teuthology.orchestra.run.smithi162.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 409 kB/s | 133 kB 00:00 2024-05-08T05:38:11.998 INFO:teuthology.orchestra.run.smithi150.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.0 MB/s | 1.8 MB 00:01 2024-05-08T05:38:12.028 INFO:teuthology.orchestra.run.smithi112.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 686 kB/s | 90 kB 00:00 2024-05-08T05:38:12.056 INFO:teuthology.orchestra.run.smithi112.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 226 kB/s | 28 kB 00:00 2024-05-08T05:38:12.081 INFO:teuthology.orchestra.run.smithi112.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 246 kB/s | 31 kB 00:00 2024-05-08T05:38:12.140 INFO:teuthology.orchestra.run.smithi112.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-08T05:38:12.165 INFO:teuthology.orchestra.run.smithi112.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 547 kB/s | 59 kB 00:00 2024-05-08T05:38:12.166 INFO:teuthology.orchestra.run.smithi162.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 456 kB/s | 87 kB 00:00 2024-05-08T05:38:12.191 INFO:teuthology.orchestra.run.smithi112.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-05-08T05:38:12.241 INFO:teuthology.orchestra.run.smithi112.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 510 kB/s | 51 kB 00:00 2024-05-08T05:38:12.283 INFO:teuthology.orchestra.run.smithi112.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-08T05:38:12.308 INFO:teuthology.orchestra.run.smithi112.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 768 kB/s | 90 kB 00:00 2024-05-08T05:38:12.366 INFO:teuthology.orchestra.run.smithi112.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-08T05:38:12.392 INFO:teuthology.orchestra.run.smithi112.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-05-08T05:38:12.456 INFO:teuthology.orchestra.run.smithi112.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-05-08T05:38:12.567 INFO:teuthology.orchestra.run.smithi112.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-05-08T05:38:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 226 kB/s | 248 kB 00:01 2024-05-08T05:38:12.759 INFO:teuthology.orchestra.run.smithi112.stdout:(149/156): librados2-19.0.0-3538.gbee299f4.el9. 9.3 MB/s | 3.4 MB 00:00 2024-05-08T05:38:12.882 INFO:teuthology.orchestra.run.smithi150.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 1.0 MB/s | 1.3 MB 00:01 2024-05-08T05:38:12.951 INFO:teuthology.orchestra.run.smithi112.stdout:(150/156): librbd1-19.0.0-3538.gbee299f4.el9.x8 6.4 MB/s | 3.1 MB 00:00 2024-05-08T05:38:13.018 INFO:teuthology.orchestra.run.smithi112.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 5.1 MB/s | 2.3 MB 00:00 2024-05-08T05:38:13.110 INFO:teuthology.orchestra.run.smithi112.stdout:(152/156): NetworkManager-libnm-1.47.5-1.el9.x8 5.2 MB/s | 1.8 MB 00:00 2024-05-08T05:38:13.135 INFO:teuthology.orchestra.run.smithi112.stdout:(153/156): NetworkManager-team-1.47.5-1.el9.x86 228 kB/s | 42 kB 00:00 2024-05-08T05:38:13.169 INFO:teuthology.orchestra.run.smithi112.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 1.6 MB/s | 248 kB 00:00 2024-05-08T05:38:13.261 INFO:teuthology.orchestra.run.smithi112.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-05-08T05:38:13.366 INFO:teuthology.orchestra.run.smithi162.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 729 kB/s | 1.0 MB 00:01 2024-05-08T05:38:13.515 INFO:teuthology.orchestra.run.smithi150.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.6 MB/s | 2.4 MB 00:01 2024-05-08T05:38:13.518 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:38:13.518 INFO:teuthology.orchestra.run.smithi150.stdout:Total 11 MB/s | 226 MB 00:20 2024-05-08T05:38:13.527 INFO:teuthology.orchestra.run.smithi112.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.1 MB/s | 2.4 MB 00:00 2024-05-08T05:38:13.534 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:38:13.535 INFO:teuthology.orchestra.run.smithi112.stdout:Total 6.6 MB/s | 226 MB 00:34 2024-05-08T05:38:13.720 INFO:teuthology.orchestra.run.smithi162.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 691 kB/s | 244 kB 00:00 2024-05-08T05:38:14.075 INFO:teuthology.orchestra.run.smithi162.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 699 kB/s | 248 kB 00:00 2024-05-08T05:38:14.184 INFO:teuthology.orchestra.run.smithi162.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-05-08T05:38:14.292 INFO:teuthology.orchestra.run.smithi162.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-05-08T05:38:14.576 INFO:teuthology.orchestra.run.smithi162.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 608 kB/s | 172 kB 00:00 2024-05-08T05:38:14.676 INFO:teuthology.orchestra.run.smithi162.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 347 kB/s | 35 kB 00:00 2024-05-08T05:38:14.698 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-05-08T05:38:14.851 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-05-08T05:38:14.851 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-05-08T05:38:15.527 INFO:teuthology.orchestra.run.smithi162.stdout:(80/156): postgresql-server-13.14-1.el9.x86_64. 1.5 MB/s | 5.8 MB 00:03 2024-05-08T05:38:15.868 INFO:teuthology.orchestra.run.smithi162.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 734 kB/s | 250 kB 00:00 2024-05-08T05:38:16.210 INFO:teuthology.orchestra.run.smithi162.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 650 kB/s | 222 kB 00:00 2024-05-08T05:38:16.402 INFO:teuthology.orchestra.run.smithi162.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 403 kB/s | 77 kB 00:00 2024-05-08T05:38:16.411 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-05-08T05:38:16.503 INFO:teuthology.orchestra.run.smithi162.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 423 kB/s | 42 kB 00:00 2024-05-08T05:38:16.645 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-05-08T05:38:16.645 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-05-08T05:38:16.660 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-05-08T05:38:16.660 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-05-08T05:38:16.919 INFO:teuthology.orchestra.run.smithi162.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 646 kB/s | 269 kB 00:00 2024-05-08T05:38:17.195 INFO:teuthology.orchestra.run.smithi162.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 577 kB/s | 159 kB 00:00 2024-05-08T05:38:17.620 INFO:teuthology.orchestra.run.smithi162.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 656 kB/s | 279 kB 00:00 2024-05-08T05:38:17.720 INFO:teuthology.orchestra.run.smithi162.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 510 kB/s | 51 kB 00:00 2024-05-08T05:38:17.829 INFO:teuthology.orchestra.run.smithi162.stdout:(89/156): python3-requests-oauthlib-1.3.0-12.el 496 kB/s | 54 kB 00:00 2024-05-08T05:38:19.375 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-05-08T05:38:19.376 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-05-08T05:38:20.062 INFO:teuthology.orchestra.run.smithi162.stdout:(90/156): python3-babel-2.9.1-2.el9.noarch.rpm 773 kB/s | 6.0 MB 00:07 2024-05-08T05:38:20.163 INFO:teuthology.orchestra.run.smithi162.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 418 kB/s | 42 kB 00:00 2024-05-08T05:38:20.263 INFO:teuthology.orchestra.run.smithi162.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-05-08T05:38:20.621 INFO:teuthology.orchestra.run.smithi162.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 616 kB/s | 221 kB 00:00 2024-05-08T05:38:20.721 INFO:teuthology.orchestra.run.smithi162.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 359 kB/s | 36 kB 00:00 2024-05-08T05:38:20.822 INFO:teuthology.orchestra.run.smithi162.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-05-08T05:38:21.240 INFO:teuthology.orchestra.run.smithi162.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 732 kB/s | 305 kB 00:00 2024-05-08T05:38:21.339 INFO:teuthology.orchestra.run.smithi162.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 578 kB/s | 58 kB 00:00 2024-05-08T05:38:21.532 INFO:teuthology.orchestra.run.smithi162.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 489 kB/s | 94 kB 00:00 2024-05-08T05:38:21.706 INFO:teuthology.orchestra.run.smithi162.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 364 kB/s | 64 kB 00:00 2024-05-08T05:38:21.798 INFO:teuthology.orchestra.run.smithi162.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 488 kB/s | 45 kB 00:00 2024-05-08T05:38:21.865 INFO:teuthology.orchestra.run.smithi162.stdout:(101/156): python3-numpy-1.20.1-5.el9.x86_64.rp 726 kB/s | 5.1 MB 00:07 2024-05-08T05:38:22.640 INFO:teuthology.orchestra.run.smithi162.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 26 kB/s | 22 kB 00:00 2024-05-08T05:38:22.927 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-05-08T05:38:23.020 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T05:38:23.058 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T05:38:23.111 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T05:38:23.145 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T05:38:23.149 INFO:teuthology.orchestra.run.smithi162.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 673 kB/s | 863 kB 00:01 2024-05-08T05:38:23.171 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T05:38:23.214 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T05:38:23.545 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T05:38:23.656 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:38:23.761 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:38:23.823 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:38:23.898 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:38:23.928 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:38:24.074 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T05:38:24.124 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T05:38:24.152 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:38:24.248 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:38:24.270 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:38:24.316 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:38:24.363 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:38:24.386 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:38:24.555 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:38:24.681 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:38:24.835 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:38:24.895 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:38:24.996 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:38:25.080 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:38:25.268 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:38:25.378 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T05:38:25.410 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T05:38:25.444 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T05:38:25.508 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T05:38:25.544 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T05:38:25.587 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T05:38:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T05:38:25.648 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T05:38:25.716 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T05:38:25.769 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T05:38:25.800 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T05:38:25.844 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T05:38:25.921 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T05:38:25.955 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 32/164 2024-05-08T05:38:25.998 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 33/164 2024-05-08T05:38:26.027 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 34/164 2024-05-08T05:38:26.036 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-05-08T05:38:26.106 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T05:38:26.150 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T05:38:26.169 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T05:38:26.225 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T05:38:26.250 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T05:38:26.356 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T05:38:26.364 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T05:38:26.410 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T05:38:26.422 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T05:38:26.462 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T05:38:26.564 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T05:38:26.930 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T05:38:26.978 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T05:38:27.014 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T05:38:27.044 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T05:38:27.068 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:38:27.086 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T05:38:27.124 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 44/164 2024-05-08T05:38:27.166 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T05:38:27.229 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:38:27.261 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:38:27.314 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T05:38:27.369 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T05:38:27.386 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:38:27.418 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:38:27.461 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T05:38:27.525 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T05:38:27.597 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T05:38:27.637 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T05:38:27.673 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T05:38:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T05:38:27.720 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T05:38:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T05:38:27.764 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T05:38:27.782 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:38:27.807 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T05:38:27.849 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T05:38:27.923 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:38:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:38:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T05:38:28.030 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T05:38:28.060 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T05:38:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:38:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:38:28.101 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T05:38:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:38:28.197 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T05:38:28.244 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T05:38:28.284 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T05:38:28.309 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T05:38:28.335 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T05:38:28.408 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T05:38:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:38:28.620 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T05:38:28.658 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T05:38:28.685 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T05:38:28.728 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T05:38:28.768 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T05:38:28.799 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T05:38:28.817 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:38:28.829 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T05:38:28.863 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T05:38:28.905 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T05:38:28.947 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T05:38:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T05:38:28.990 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:38:29.068 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T05:38:29.078 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:38:29.120 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T05:38:29.151 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T05:38:29.198 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T05:38:29.273 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:38:29.297 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T05:38:29.344 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T05:38:29.385 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:38:29.723 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:38:29.930 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T05:38:29.971 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T05:38:30.027 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T05:38:30.070 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T05:38:30.125 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T05:38:30.134 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T05:38:30.158 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T05:38:30.188 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T05:38:30.252 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T05:38:30.258 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T05:38:30.300 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T05:38:30.316 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T05:38:30.339 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T05:38:30.404 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T05:38:30.436 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T05:38:30.455 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T05:38:30.520 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T05:38:30.566 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T05:38:30.614 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T05:38:30.622 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T05:38:30.642 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T05:38:30.719 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T05:38:30.731 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T05:38:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 32/164 2024-05-08T05:38:30.814 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 33/164 2024-05-08T05:38:30.850 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 34/164 2024-05-08T05:38:30.964 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T05:38:31.029 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T05:38:31.127 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T05:38:31.295 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T05:38:31.365 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T05:38:31.365 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T05:38:31.474 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T05:38:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T05:38:32.154 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T05:38:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T05:38:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T05:38:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 44/164 2024-05-08T05:38:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T05:38:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T05:38:32.693 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T05:38:32.695 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T05:38:32.771 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T05:38:32.799 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T05:38:32.818 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T05:38:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T05:38:32.887 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T05:38:32.980 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T05:38:33.019 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T05:38:33.056 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T05:38:33.144 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T05:38:33.146 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T05:38:33.186 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:38:33.191 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T05:38:33.256 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T05:38:33.277 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:38:33.308 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 102/164 2024-05-08T05:38:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T05:38:33.344 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T05:38:33.538 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T05:38:33.590 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T05:38:33.627 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T05:38:33.683 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T05:38:33.813 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T05:38:33.827 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T05:38:33.860 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T05:38:33.909 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T05:38:33.968 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T05:38:33.981 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:38:34.001 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T05:38:34.036 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:38:34.058 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 106/164 2024-05-08T05:38:34.196 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T05:38:34.365 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T05:38:34.407 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T05:38:34.444 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T05:38:34.480 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T05:38:34.537 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T05:38:34.599 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T05:38:34.643 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T05:38:34.689 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T05:38:34.738 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T05:38:34.840 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T05:38:34.889 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T05:38:35.028 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T05:38:35.100 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T05:38:35.145 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T05:38:35.241 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T05:38:35.399 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T05:38:35.467 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T05:38:36.329 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:38:36.362 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:38:36.401 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T05:38:36.468 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T05:38:36.508 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T05:38:36.632 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T05:38:36.694 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T05:38:36.791 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T05:38:36.834 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T05:38:36.904 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:38:36.941 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:38:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:38:37.033 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T05:38:37.033 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T05:38:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-08T05:38:37.062 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T05:38:37.067 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:38:37.089 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T05:38:37.190 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T05:38:37.995 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T05:38:38.688 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T05:38:39.146 INFO:teuthology.orchestra.run.smithi162.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 781 kB/s | 16 MB 00:21 2024-05-08T05:38:40.373 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T05:38:40.505 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T05:38:40.550 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T05:38:40.594 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T05:38:40.971 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T05:38:41.017 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:38:41.171 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:38:41.207 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 102/164 2024-05-08T05:38:41.262 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T05:38:41.838 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T05:38:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:38:41.970 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:38:41.997 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 106/164 2024-05-08T05:38:42.203 INFO:teuthology.orchestra.run.smithi162.stdout:[MIRROR] grpc-data-1.46.7-10.el9.noarch.rpm: Curl error (7): Couldn't connect to server for https://ix-denver.mm.fcix.net/epel/9/Everything/x86_64/Packages/g/grpc-data-1.46.7-10.el9.noarch.rpm [Failed to connect to ix-denver.mm.fcix.net port 443: Network is unreachable] 2024-05-08T05:38:42.377 INFO:teuthology.orchestra.run.smithi162.stdout:[MIRROR] grpc-data-1.46.7-10.el9.noarch.rpm: Curl error (7): Couldn't connect to server for http://ix-denver.mm.fcix.net/epel/9/Everything/x86_64/Packages/g/grpc-data-1.46.7-10.el9.noarch.rpm [Failed to connect to ix-denver.mm.fcix.net port 80: Network is unreachable] 2024-05-08T05:38:42.487 INFO:teuthology.orchestra.run.smithi162.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 5.8 kB/s | 19 kB 00:03 2024-05-08T05:38:42.517 INFO:teuthology.orchestra.run.smithi162.stdout:[MIRROR] libarrow-9.0.0-11.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ix-denver.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-11.el9.x86_64.rpm [Failed to connect to ix-denver.mm.fcix.net port 443: Network is unreachable] 2024-05-08T05:38:42.528 INFO:teuthology.orchestra.run.smithi162.stdout:[MIRROR] libarrow-9.0.0-11.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ix-denver.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-11.el9.x86_64.rpm [Failed to connect to ix-denver.mm.fcix.net port 80: Network is unreachable] 2024-05-08T05:38:42.813 INFO:teuthology.orchestra.run.smithi162.stdout:(106/156): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-08T05:38:44.718 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:38:44.760 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:38:45.434 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:38:45.498 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:38:45.694 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:38:45.694 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T05:38:45.695 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T05:38:45.695 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:38:45.740 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:38:48.824 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:38:48.825 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-05-08T05:38:48.825 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-05-08T05:38:48.825 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-05-08T05:38:48.825 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-05-08T05:38:48.825 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-05-08T05:38:48.826 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-05-08T05:38:48.826 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-05-08T05:38:48.826 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-08T05:38:49.018 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:38:49.033 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:38:49.033 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:38:49.033 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T05:38:49.033 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-08T05:38:50.504 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:38:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:38:50.529 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:38:50.529 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T05:38:50.529 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:38:50.529 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:38:50.529 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-08T05:38:50.754 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:38:50.776 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:38:50.776 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:38:50.777 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T05:38:50.777 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:38:50.777 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:38:50.777 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-08T05:38:51.230 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:38:51.253 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:38:51.253 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:38:51.253 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T05:38:51.253 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:38:51.253 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:38:51.253 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-08T05:38:51.369 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T05:38:51.426 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T05:38:51.481 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T05:38:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T05:38:51.715 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T05:38:51.791 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T05:38:51.881 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T05:38:52.010 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T05:38:52.065 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T05:38:52.098 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T05:38:52.124 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T05:38:52.545 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:38:52.573 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:38:52.605 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 126/164 2024-05-08T05:38:52.666 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 127/164 2024-05-08T05:38:52.696 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:38:53.740 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:38:53.773 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:38:54.527 INFO:teuthology.orchestra.run.smithi162.stdout:[MIRROR] abseil-cpp-20211102.0-3.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ix-denver.mm.fcix.net/epel/9/Everything/x86_64/Packages/a/abseil-cpp-20211102.0-3.el9.x86_64.rpm [Failed to connect to ix-denver.mm.fcix.net port 443: Network is unreachable] 2024-05-08T05:38:54.533 INFO:teuthology.orchestra.run.smithi162.stdout:[MIRROR] abseil-cpp-20211102.0-3.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ix-denver.mm.fcix.net/epel/9/Everything/x86_64/Packages/a/abseil-cpp-20211102.0-3.el9.x86_64.rpm [Failed to connect to ix-denver.mm.fcix.net port 80: Network is unreachable] 2024-05-08T05:38:54.742 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:38:54.765 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:38:54.810 INFO:teuthology.orchestra.run.smithi162.stdout:(107/156): abseil-cpp-20211102.0-3.el9.x86_64.r 17 kB/s | 548 kB 00:32 2024-05-08T05:38:54.874 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:38:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 131/164 2024-05-08T05:38:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:38:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:38:55.098 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:38:55.098 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T05:38:55.098 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:38:55.099 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:38:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-08T05:38:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:38:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:38:55.252 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 134/164 2024-05-08T05:38:55.299 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 135/164 2024-05-08T05:38:56.224 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T05:38:56.262 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:38:56.285 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:38:56.285 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:38:56.285 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T05:38:56.285 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:38:56.285 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:38:56.286 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-08T05:38:56.329 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:38:56.351 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:38:56.351 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:38:56.351 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T05:38:56.351 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-08T05:38:56.405 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:38:56.427 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:38:56.427 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:38:56.428 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T05:38:56.428 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:38:56.428 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:38:56.428 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-08T05:38:56.731 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:38:56.753 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:38:56.753 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:38:56.753 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T05:38:56.753 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:38:56.753 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:38:56.754 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-08T05:38:58.652 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:38:58.653 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-05-08T05:38:58.653 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-05-08T05:38:58.653 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-05-08T05:38:58.653 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-05-08T05:38:58.653 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-05-08T05:38:58.653 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-05-08T05:38:58.653 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-05-08T05:38:58.653 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:38:58.894 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:38:58.939 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:38:58.940 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:38:58.940 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T05:38:58.940 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:39:00.544 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:39:00.628 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:39:00.628 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:00.628 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T05:39:00.628 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:39:00.628 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:39:00.628 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:39:00.878 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:39:00.973 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:39:00.973 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:00.973 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T05:39:00.973 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:39:00.973 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:39:00.974 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:39:01.343 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 141/164 2024-05-08T05:39:01.384 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 142/164 2024-05-08T05:39:01.424 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 143/164 2024-05-08T05:39:01.453 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:39:01.457 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T05:39:01.484 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T05:39:01.535 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:39:01.535 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:01.535 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T05:39:01.535 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:39:01.535 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:39:01.535 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:39:01.664 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 146/164 2024-05-08T05:39:01.670 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T05:39:01.705 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T05:39:01.733 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T05:39:01.736 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 148/164 2024-05-08T05:39:01.765 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T05:39:01.815 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T05:39:01.828 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T05:39:01.915 INFO:teuthology.orchestra.run.smithi162.stdout:[MIRROR] liboath-2.6.7-2.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ix-denver.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/liboath-2.6.7-2.el9.x86_64.rpm [Failed to connect to ix-denver.mm.fcix.net port 80: Network is unreachable] 2024-05-08T05:39:01.943 INFO:teuthology.orchestra.run.smithi162.stdout:[MIRROR] liboath-2.6.7-2.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ix-denver.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/liboath-2.6.7-2.el9.x86_64.rpm [Failed to connect to ix-denver.mm.fcix.net port 443: Network is unreachable] 2024-05-08T05:39:02.050 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T05:39:02.090 INFO:teuthology.orchestra.run.smithi162.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 6.7 kB/s | 49 kB 00:07 2024-05-08T05:39:02.108 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T05:39:02.193 INFO:teuthology.orchestra.run.smithi162.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 656 kB/s | 67 kB 00:00 2024-05-08T05:39:02.194 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T05:39:02.268 INFO:teuthology.orchestra.run.smithi162.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-05-08T05:39:02.310 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T05:39:02.376 INFO:teuthology.orchestra.run.smithi162.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 839 kB 00:00 2024-05-08T05:39:02.427 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:39:02.435 INFO:teuthology.orchestra.run.smithi162.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-05-08T05:39:02.468 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:39:02.477 INFO:teuthology.orchestra.run.smithi162.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-05-08T05:39:02.483 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T05:39:02.488 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:39:02.519 INFO:teuthology.orchestra.run.smithi162.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 775 kB/s | 32 kB 00:00 2024-05-08T05:39:02.532 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T05:39:02.561 INFO:teuthology.orchestra.run.smithi162.stdout:(115/156): python3-certifi-2021.10.8-2.el9.noar 360 kB/s | 15 kB 00:00 2024-05-08T05:39:02.587 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T05:39:02.611 INFO:teuthology.orchestra.run.smithi162.stdout:(116/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-05-08T05:39:02.614 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T05:39:02.661 INFO:teuthology.orchestra.run.smithi162.stdout:(117/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-05-08T05:39:02.712 INFO:teuthology.orchestra.run.smithi162.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-05-08T05:39:02.762 INFO:teuthology.orchestra.run.smithi162.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 709 kB/s | 35 kB 00:00 2024-05-08T05:39:02.812 INFO:teuthology.orchestra.run.smithi162.stdout:(120/156): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-05-08T05:39:02.912 INFO:teuthology.orchestra.run.smithi162.stdout:(121/156): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-05-08T05:39:02.962 INFO:teuthology.orchestra.run.smithi162.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-05-08T05:39:03.013 INFO:teuthology.orchestra.run.smithi162.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-05-08T05:39:03.063 INFO:teuthology.orchestra.run.smithi162.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-05-08T05:39:03.113 INFO:teuthology.orchestra.run.smithi162.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 464 kB/s | 23 kB 00:00 2024-05-08T05:39:03.164 INFO:teuthology.orchestra.run.smithi162.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-05-08T05:39:03.215 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:39:03.222 INFO:teuthology.orchestra.run.smithi162.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-05-08T05:39:03.274 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:39:03.282 INFO:teuthology.orchestra.run.smithi162.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-05-08T05:39:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 152/164 2024-05-08T05:39:03.331 INFO:teuthology.orchestra.run.smithi162.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 929 kB/s | 46 kB 00:00 2024-05-08T05:39:03.342 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 126/164 2024-05-08T05:39:03.373 INFO:teuthology.orchestra.run.smithi162.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-05-08T05:39:03.406 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T05:39:03.424 INFO:teuthology.orchestra.run.smithi162.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-08T05:39:03.447 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T05:39:03.452 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 127/164 2024-05-08T05:39:03.473 INFO:teuthology.orchestra.run.smithi162.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-05-08T05:39:03.486 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:39:03.524 INFO:teuthology.orchestra.run.smithi162.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 329 kB/s | 16 kB 00:00 2024-05-08T05:39:03.528 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T05:39:03.568 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T05:39:03.574 INFO:teuthology.orchestra.run.smithi162.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-05-08T05:39:03.594 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T05:39:03.594 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:39:03.624 INFO:teuthology.orchestra.run.smithi162.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-05-08T05:39:03.662 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:39:03.674 INFO:teuthology.orchestra.run.smithi162.stdout:(136/156): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-05-08T05:39:03.686 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T05:39:03.687 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:39:03.725 INFO:teuthology.orchestra.run.smithi162.stdout:(137/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-05-08T05:39:03.775 INFO:teuthology.orchestra.run.smithi162.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-05-08T05:39:03.779 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:39:03.804 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T05:39:03.809 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:39:03.809 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:39:03.825 INFO:teuthology.orchestra.run.smithi162.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-05-08T05:39:03.876 INFO:teuthology.orchestra.run.smithi162.stdout:(140/156): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-08T05:39:03.926 INFO:teuthology.orchestra.run.smithi162.stdout:(141/156): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-05-08T05:39:03.976 INFO:teuthology.orchestra.run.smithi162.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-05-08T05:39:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:39:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:39:04.026 INFO:teuthology.orchestra.run.smithi162.stdout:(143/156): python3-werkzeug-2.0.3-3.el9.1.noarc 8.3 MB/s | 427 kB 00:00 2024-05-08T05:39:04.065 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:39:04.076 INFO:teuthology.orchestra.run.smithi162.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 401 kB/s | 20 kB 00:00 2024-05-08T05:39:04.127 INFO:teuthology.orchestra.run.smithi162.stdout:(145/156): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-05-08T05:39:04.153 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:39:04.194 INFO:teuthology.orchestra.run.smithi162.stdout:(146/156): thrift-0.15.0-2.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-05-08T05:39:04.794 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:39:04.846 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:39:04.927 INFO:teuthology.orchestra.run.smithi162.stdout:(147/156): librados2-19.0.0-3538.gbee299f4.el9. 4.6 MB/s | 3.4 MB 00:00 2024-05-08T05:39:05.494 INFO:teuthology.orchestra.run.smithi162.stdout:(148/156): librbd1-19.0.0-3538.gbee299f4.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-05-08T05:39:05.936 INFO:teuthology.orchestra.run.smithi162.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 5.2 MB/s | 2.3 MB 00:00 2024-05-08T05:39:05.950 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:39:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:39:06.135 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:39:06.186 INFO:teuthology.orchestra.run.smithi162.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 7.3 MB/s | 1.8 MB 00:00 2024-05-08T05:39:06.236 INFO:teuthology.orchestra.run.smithi162.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 837 kB/s | 42 kB 00:00 2024-05-08T05:39:06.295 INFO:teuthology.orchestra.run.smithi162.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-05-08T05:39:06.330 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 131/164 2024-05-08T05:39:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:39:06.460 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:39:06.460 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:06.461 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T05:39:06.461 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:39:06.461 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:39:06.461 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:39:06.478 INFO:teuthology.orchestra.run.smithi162.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 1.3 MB 00:00 2024-05-08T05:39:06.516 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:39:06.555 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:39:06.667 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 134/164 2024-05-08T05:39:06.732 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 135/164 2024-05-08T05:39:06.787 INFO:teuthology.orchestra.run.smithi162.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.8 MB/s | 2.4 MB 00:00 2024-05-08T05:39:07.730 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T05:39:07.760 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:39:07.834 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:39:07.835 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:07.835 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T05:39:07.835 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:39:07.835 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:39:07.835 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:39:07.903 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:39:07.990 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:39:07.990 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:07.990 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T05:39:07.990 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:39:08.070 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:39:08.147 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:39:08.147 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:08.147 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T05:39:08.147 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:39:08.147 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:39:08.147 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:39:08.504 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:39:08.589 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:39:08.589 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:08.590 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T05:39:08.590 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:39:08.590 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:39:08.590 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:39:09.732 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:39:09.732 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 1/164 2024-05-08T05:39:09.732 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2/164 2024-05-08T05:39:09.732 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 3/164 2024-05-08T05:39:09.732 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 4/164 2024-05-08T05:39:09.732 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 5/164 2024-05-08T05:39:09.732 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 6/164 2024-05-08T05:39:09.733 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 7/164 2024-05-08T05:39:09.733 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:39:09.733 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:39:09.733 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:39:09.733 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 11/164 2024-05-08T05:39:09.733 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 12/164 2024-05-08T05:39:09.733 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:39:09.733 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 14/164 2024-05-08T05:39:09.733 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 15/164 2024-05-08T05:39:09.733 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 16/164 2024-05-08T05:39:09.733 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 17/164 2024-05-08T05:39:09.733 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 18/164 2024-05-08T05:39:09.734 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 19/164 2024-05-08T05:39:09.734 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 20/164 2024-05-08T05:39:09.734 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 21/164 2024-05-08T05:39:09.734 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 22/164 2024-05-08T05:39:09.734 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 23/164 2024-05-08T05:39:09.734 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 24/164 2024-05-08T05:39:09.734 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 25/164 2024-05-08T05:39:09.734 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 26/164 2024-05-08T05:39:09.734 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 27/164 2024-05-08T05:39:09.734 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 28/164 2024-05-08T05:39:09.734 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 29/164 2024-05-08T05:39:09.734 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 30/164 2024-05-08T05:39:09.734 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 31/164 2024-05-08T05:39:09.734 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 32/164 2024-05-08T05:39:09.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 33/164 2024-05-08T05:39:09.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 34/164 2024-05-08T05:39:09.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 35/164 2024-05-08T05:39:09.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 36/164 2024-05-08T05:39:09.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 37/164 2024-05-08T05:39:09.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 38/164 2024-05-08T05:39:09.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 39/164 2024-05-08T05:39:09.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T05:39:09.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T05:39:09.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T05:39:09.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T05:39:09.735 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T05:39:09.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T05:39:09.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T05:39:09.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T05:39:09.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T05:39:09.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T05:39:09.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T05:39:09.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T05:39:09.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T05:39:09.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T05:39:09.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T05:39:09.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T05:39:09.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T05:39:09.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T05:39:09.736 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T05:39:09.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T05:39:09.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T05:39:09.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T05:39:09.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T05:39:09.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T05:39:09.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T05:39:09.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T05:39:09.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T05:39:09.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T05:39:09.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T05:39:09.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T05:39:09.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T05:39:09.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T05:39:09.737 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T05:39:09.738 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T05:39:09.738 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T05:39:09.738 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T05:39:09.738 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T05:39:09.738 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T05:39:09.738 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T05:39:09.738 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T05:39:09.738 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T05:39:09.738 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T05:39:09.738 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T05:39:09.738 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T05:39:09.738 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T05:39:09.738 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T05:39:09.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T05:39:09.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T05:39:09.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T05:39:09.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T05:39:09.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T05:39:09.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T05:39:09.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T05:39:09.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T05:39:09.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T05:39:09.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T05:39:09.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T05:39:09.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T05:39:09.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T05:39:09.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T05:39:09.740 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T05:39:09.740 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T05:39:09.740 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T05:39:09.740 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T05:39:09.740 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T05:39:09.740 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T05:39:09.740 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T05:39:09.740 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T05:39:09.740 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T05:39:09.740 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T05:39:09.740 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T05:39:09.740 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T05:39:09.740 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T05:39:09.741 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T05:39:09.741 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T05:39:09.741 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T05:39:09.741 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T05:39:09.741 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T05:39:09.741 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T05:39:09.741 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T05:39:09.741 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T05:39:09.741 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T05:39:09.741 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T05:39:09.741 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T05:39:09.741 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T05:39:09.741 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T05:39:09.741 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T05:39:09.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T05:39:09.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T05:39:09.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T05:39:09.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T05:39:09.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T05:39:09.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T05:39:09.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T05:39:09.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T05:39:09.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T05:39:09.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T05:39:09.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T05:39:09.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T05:39:09.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T05:39:09.742 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T05:39:09.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T05:39:09.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T05:39:09.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T05:39:09.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T05:39:09.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T05:39:09.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T05:39:09.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T05:39:09.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T05:39:09.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 149/164 2024-05-08T05:39:09.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T05:39:09.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 151/164 2024-05-08T05:39:09.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T05:39:09.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T05:39:09.743 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T05:39:09.744 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T05:39:09.744 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T05:39:09.744 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T05:39:09.744 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T05:39:09.744 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T05:39:09.744 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T05:39:09.744 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T05:39:09.744 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T05:39:09.744 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T05:39:12.273 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:39:12.274 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-08T05:39:12.274 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-05-08T05:39:12.274 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T05:39:12.274 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T05:39:12.274 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T05:39:12.274 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T05:39:12.274 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.274 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.274 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T05:39:12.274 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T05:39:12.274 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-05-08T05:39:12.275 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T05:39:12.275 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T05:39:12.275 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T05:39:12.275 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T05:39:12.275 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.275 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.275 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.275 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.275 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:12.275 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.275 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.275 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.275 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:12.275 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:12.276 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:12.276 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:12.276 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:12.276 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.276 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.276 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:12.276 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.276 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.276 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.276 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:12.276 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:12.276 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.276 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.277 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:12.277 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T05:39:12.277 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T05:39:12.277 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T05:39:12.277 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T05:39:12.277 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T05:39:12.277 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T05:39:12.277 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T05:39:12.277 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T05:39:12.277 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T05:39:12.277 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T05:39:12.277 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T05:39:12.277 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.278 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.278 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.278 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T05:39:12.278 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T05:39:12.278 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T05:39:12.278 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T05:39:12.278 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T05:39:12.278 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.278 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.278 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T05:39:12.278 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.278 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T05:39:12.278 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T05:39:12.279 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T05:39:12.279 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T05:39:12.279 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T05:39:12.279 INFO:teuthology.orchestra.run.smithi150.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T05:39:12.279 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T05:39:12.279 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T05:39:12.279 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T05:39:12.279 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T05:39:12.279 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T05:39:12.279 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T05:39:12.279 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T05:39:12.279 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T05:39:12.279 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T05:39:12.279 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T05:39:12.280 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T05:39:12.280 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T05:39:12.280 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T05:39:12.280 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T05:39:12.280 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T05:39:12.280 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T05:39:12.280 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T05:39:12.280 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.280 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.280 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.280 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T05:39:12.280 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T05:39:12.280 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T05:39:12.280 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T05:39:12.281 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T05:39:12.281 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T05:39:12.281 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T05:39:12.281 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T05:39:12.281 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T05:39:12.281 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T05:39:12.281 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T05:39:12.281 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T05:39:12.281 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T05:39:12.281 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T05:39:12.281 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T05:39:12.281 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T05:39:12.281 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T05:39:12.281 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T05:39:12.282 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T05:39:12.282 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T05:39:12.282 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T05:39:12.282 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T05:39:12.282 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T05:39:12.282 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T05:39:12.282 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T05:39:12.282 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T05:39:12.282 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T05:39:12.282 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T05:39:12.282 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T05:39:12.282 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T05:39:12.282 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T05:39:12.282 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T05:39:12.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T05:39:12.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T05:39:12.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T05:39:12.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T05:39:12.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T05:39:12.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T05:39:12.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T05:39:12.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T05:39:12.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T05:39:12.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T05:39:12.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T05:39:12.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.284 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T05:39:12.284 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T05:39:12.284 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T05:39:12.284 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T05:39:12.284 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T05:39:12.284 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T05:39:12.284 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T05:39:12.284 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T05:39:12.284 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T05:39:12.284 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T05:39:12.284 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T05:39:12.284 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T05:39:12.284 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T05:39:12.284 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T05:39:12.284 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T05:39:12.285 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.285 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.285 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:12.285 INFO:teuthology.orchestra.run.smithi150.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T05:39:12.285 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T05:39:12.285 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T05:39:12.285 INFO:teuthology.orchestra.run.smithi150.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T05:39:12.285 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T05:39:12.285 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T05:39:12.285 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T05:39:12.286 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-08T05:39:12.286 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-05-08T05:39:12.689 DEBUG:teuthology.parallel:result is None 2024-05-08T05:39:13.632 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 141/164 2024-05-08T05:39:13.690 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 142/164 2024-05-08T05:39:13.769 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 143/164 2024-05-08T05:39:13.805 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T05:39:13.844 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T05:39:14.158 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 146/164 2024-05-08T05:39:14.218 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T05:39:14.246 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 148/164 2024-05-08T05:39:14.282 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T05:39:14.323 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T05:39:15.234 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:39:15.287 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:39:15.353 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:39:15.535 INFO:teuthology.orchestra.run.smithi162.stdout:(155/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 812 B/s | 26 kB 00:32 2024-05-08T05:39:16.699 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 152/164 2024-05-08T05:39:16.810 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T05:39:16.882 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T05:39:17.023 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T05:39:17.087 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T05:39:17.125 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T05:39:17.125 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:39:17.253 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:39:17.299 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T05:39:17.300 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:39:17.437 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:39:17.494 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T05:39:17.527 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:39:17.528 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:39:17.838 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:39:17.838 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:39:18.008 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:39:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:39:18.411 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:39:18.411 INFO:teuthology.orchestra.run.smithi112.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-08T05:39:18.411 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:39:24.341 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 1/164 2024-05-08T05:39:24.341 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2/164 2024-05-08T05:39:24.341 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 3/164 2024-05-08T05:39:24.341 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 4/164 2024-05-08T05:39:24.341 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 5/164 2024-05-08T05:39:24.341 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 6/164 2024-05-08T05:39:24.341 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 7/164 2024-05-08T05:39:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:39:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:39:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:39:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 11/164 2024-05-08T05:39:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 12/164 2024-05-08T05:39:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:39:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 14/164 2024-05-08T05:39:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 15/164 2024-05-08T05:39:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 16/164 2024-05-08T05:39:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 17/164 2024-05-08T05:39:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 18/164 2024-05-08T05:39:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 19/164 2024-05-08T05:39:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 20/164 2024-05-08T05:39:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 21/164 2024-05-08T05:39:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 22/164 2024-05-08T05:39:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 23/164 2024-05-08T05:39:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 24/164 2024-05-08T05:39:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 25/164 2024-05-08T05:39:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 26/164 2024-05-08T05:39:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 27/164 2024-05-08T05:39:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 28/164 2024-05-08T05:39:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 29/164 2024-05-08T05:39:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 30/164 2024-05-08T05:39:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 31/164 2024-05-08T05:39:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 32/164 2024-05-08T05:39:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 33/164 2024-05-08T05:39:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 34/164 2024-05-08T05:39:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 35/164 2024-05-08T05:39:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 36/164 2024-05-08T05:39:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 37/164 2024-05-08T05:39:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 38/164 2024-05-08T05:39:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 39/164 2024-05-08T05:39:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T05:39:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T05:39:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T05:39:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T05:39:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T05:39:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T05:39:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T05:39:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T05:39:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T05:39:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T05:39:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T05:39:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T05:39:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T05:39:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T05:39:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T05:39:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T05:39:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T05:39:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T05:39:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T05:39:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T05:39:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T05:39:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T05:39:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T05:39:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T05:39:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T05:39:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T05:39:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T05:39:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T05:39:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T05:39:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T05:39:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T05:39:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T05:39:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T05:39:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T05:39:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T05:39:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T05:39:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T05:39:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T05:39:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T05:39:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T05:39:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T05:39:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T05:39:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T05:39:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T05:39:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T05:39:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T05:39:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T05:39:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T05:39:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T05:39:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T05:39:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T05:39:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T05:39:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T05:39:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T05:39:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T05:39:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T05:39:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T05:39:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T05:39:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T05:39:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T05:39:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T05:39:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T05:39:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T05:39:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T05:39:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T05:39:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T05:39:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T05:39:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T05:39:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T05:39:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T05:39:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T05:39:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T05:39:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T05:39:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T05:39:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T05:39:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T05:39:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T05:39:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T05:39:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T05:39:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T05:39:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T05:39:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T05:39:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T05:39:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T05:39:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T05:39:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T05:39:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T05:39:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T05:39:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T05:39:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T05:39:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T05:39:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T05:39:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T05:39:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T05:39:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T05:39:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T05:39:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T05:39:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T05:39:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T05:39:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T05:39:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T05:39:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T05:39:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T05:39:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T05:39:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T05:39:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T05:39:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T05:39:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T05:39:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T05:39:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 149/164 2024-05-08T05:39:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T05:39:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 151/164 2024-05-08T05:39:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T05:39:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T05:39:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T05:39:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T05:39:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T05:39:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T05:39:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T05:39:24.355 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T05:39:24.355 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T05:39:24.355 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T05:39:24.355 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T05:39:24.355 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T05:39:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:39:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:39:26.692 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-05-08T05:39:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T05:39:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T05:39:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T05:39:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T05:39:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T05:39:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T05:39:26.693 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-05-08T05:39:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T05:39:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T05:39:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T05:39:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T05:39:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T05:39:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T05:39:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T05:39:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T05:39:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T05:39:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T05:39:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T05:39:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T05:39:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T05:39:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T05:39:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T05:39:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T05:39:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T05:39:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T05:39:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T05:39:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T05:39:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T05:39:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T05:39:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T05:39:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T05:39:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T05:39:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T05:39:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T05:39:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T05:39:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T05:39:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T05:39:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T05:39:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T05:39:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T05:39:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T05:39:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T05:39:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T05:39:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T05:39:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T05:39:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T05:39:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T05:39:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T05:39:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T05:39:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T05:39:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T05:39:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T05:39:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T05:39:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T05:39:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T05:39:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T05:39:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T05:39:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T05:39:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T05:39:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T05:39:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T05:39:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T05:39:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T05:39:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T05:39:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T05:39:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T05:39:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T05:39:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T05:39:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T05:39:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T05:39:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T05:39:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T05:39:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T05:39:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T05:39:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T05:39:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T05:39:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T05:39:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T05:39:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T05:39:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T05:39:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T05:39:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T05:39:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T05:39:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T05:39:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T05:39:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T05:39:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T05:39:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T05:39:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T05:39:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T05:39:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T05:39:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T05:39:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T05:39:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T05:39:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T05:39:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T05:39:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T05:39:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T05:39:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T05:39:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T05:39:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T05:39:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T05:39:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T05:39:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T05:39:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T05:39:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T05:39:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T05:39:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T05:39:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T05:39:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T05:39:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T05:39:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T05:39:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T05:39:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T05:39:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T05:39:26.704 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T05:39:26.704 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:39:26.704 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-05-08T05:39:27.429 DEBUG:teuthology.parallel:result is None 2024-05-08T05:39:27.608 INFO:teuthology.orchestra.run.smithi162.stdout:(156/156): gperftools-libs-2.9.1-2.el9.x86_64.r 4.8 kB/s | 309 kB 01:04 2024-05-08T05:39:27.612 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:39:27.613 INFO:teuthology.orchestra.run.smithi162.stdout:Total 2.6 MB/s | 226 MB 01:28 2024-05-08T05:39:28.749 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-05-08T05:39:28.900 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-05-08T05:39:28.900 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-05-08T05:39:30.714 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-05-08T05:39:30.714 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-05-08T05:39:36.757 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-05-08T05:39:36.836 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T05:39:36.875 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T05:39:36.936 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T05:39:36.979 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T05:39:37.004 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T05:39:37.047 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T05:39:37.367 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T05:39:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:39:37.577 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:39:37.606 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:39:37.697 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:39:37.727 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:39:37.878 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T05:39:37.924 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T05:39:37.952 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:39:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:39:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:39:38.141 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:39:38.163 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:39:38.185 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:39:38.330 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:39:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:39:38.639 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:39:38.695 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:39:38.820 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:39:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:39:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:39:39.223 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T05:39:39.251 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T05:39:39.286 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T05:39:39.360 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T05:39:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T05:39:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T05:39:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T05:39:39.506 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T05:39:39.576 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T05:39:39.662 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T05:39:39.692 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T05:39:39.728 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T05:39:39.803 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T05:39:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 32/164 2024-05-08T05:39:39.881 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 33/164 2024-05-08T05:39:39.910 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 34/164 2024-05-08T05:39:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T05:39:40.042 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T05:39:40.116 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T05:39:40.251 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T05:39:40.302 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T05:39:40.838 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T05:39:40.931 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T05:39:40.968 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T05:39:41.011 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T05:39:41.049 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 44/164 2024-05-08T05:39:41.091 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T05:39:41.258 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T05:39:41.310 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T05:39:41.402 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T05:39:41.475 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T05:39:41.548 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T05:39:41.578 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T05:39:41.606 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T05:39:41.653 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T05:39:41.689 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T05:39:41.732 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T05:39:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T05:39:41.912 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T05:39:41.954 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T05:39:41.984 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T05:39:42.026 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T05:39:42.123 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T05:39:42.159 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T05:39:42.200 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T05:39:42.225 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T05:39:42.250 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T05:39:42.325 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T05:39:42.453 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T05:39:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T05:39:42.526 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T05:39:42.561 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T05:39:42.600 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T05:39:42.631 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T05:39:42.661 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T05:39:42.695 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T05:39:42.729 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T05:39:42.771 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T05:39:42.797 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T05:39:42.885 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T05:39:42.936 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T05:39:42.966 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T05:39:43.013 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T05:39:43.113 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T05:39:43.169 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T05:39:43.908 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T05:39:43.950 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T05:39:43.981 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T05:39:44.083 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T05:39:44.140 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T05:39:44.229 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T05:39:44.275 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T05:39:44.454 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T05:39:44.481 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T05:39:44.562 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T05:39:45.211 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T05:39:45.330 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T05:39:46.569 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T05:39:46.659 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T05:39:46.687 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T05:39:46.720 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T05:39:47.032 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T05:39:47.075 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:39:47.165 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:39:47.196 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 102/164 2024-05-08T05:39:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T05:39:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T05:39:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:39:47.815 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:39:47.837 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 106/164 2024-05-08T05:39:50.143 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:39:50.175 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:39:50.716 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:39:50.770 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:39:50.862 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:39:50.862 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T05:39:50.862 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T05:39:50.862 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:39:50.897 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:40:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:40:02.647 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /sys 2024-05-08T05:40:02.647 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /proc 2024-05-08T05:40:02.647 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /mnt 2024-05-08T05:40:02.647 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /var/tmp 2024-05-08T05:40:02.647 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /home 2024-05-08T05:40:02.647 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /root 2024-05-08T05:40:02.647 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /tmp 2024-05-08T05:40:02.647 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:40:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:40:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:40:02.844 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:02.844 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T05:40:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:40:04.607 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:40:04.629 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:40:04.630 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:04.630 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T05:40:04.630 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:40:04.630 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:40:04.630 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:40:05.075 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:40:05.097 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:40:05.097 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:05.097 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T05:40:05.097 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:40:05.097 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:40:05.097 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:40:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T05:40:05.253 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T05:40:05.309 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T05:40:05.494 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T05:40:05.542 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T05:40:05.637 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T05:40:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T05:40:05.873 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T05:40:05.933 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T05:40:05.965 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T05:40:05.993 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T05:40:06.404 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:40:06.433 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:40:06.466 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 126/164 2024-05-08T05:40:06.527 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 127/164 2024-05-08T05:40:06.556 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:40:07.601 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:40:07.642 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:40:08.610 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:40:08.650 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:40:08.762 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:40:08.929 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 131/164 2024-05-08T05:40:08.961 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:40:08.984 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:40:08.984 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:08.984 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T05:40:08.984 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:40:08.984 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:40:08.984 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:40:09.028 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:40:09.039 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:40:09.137 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 134/164 2024-05-08T05:40:09.185 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 135/164 2024-05-08T05:40:10.113 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T05:40:10.148 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:40:10.171 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:40:10.171 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:10.171 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T05:40:10.171 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:40:10.171 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:40:10.171 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:40:10.214 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:40:10.236 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:40:10.236 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:10.237 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T05:40:10.237 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:40:10.283 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:40:10.304 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:40:10.304 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:10.304 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T05:40:10.304 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:40:10.304 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:40:10.305 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:40:10.608 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:40:10.629 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:40:10.630 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:10.630 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T05:40:10.630 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:40:10.630 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:40:10.630 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:40:15.253 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 141/164 2024-05-08T05:40:15.294 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 142/164 2024-05-08T05:40:15.343 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 143/164 2024-05-08T05:40:15.375 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T05:40:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T05:40:15.593 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 146/164 2024-05-08T05:40:15.640 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T05:40:15.671 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 148/164 2024-05-08T05:40:15.700 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T05:40:15.738 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T05:40:16.397 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:40:16.436 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:40:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:40:17.296 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 152/164 2024-05-08T05:40:17.399 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T05:40:17.449 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T05:40:17.530 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T05:40:17.561 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T05:40:17.587 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T05:40:17.587 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:40:17.680 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:40:17.712 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T05:40:17.713 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:40:17.797 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:40:17.822 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T05:40:17.827 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:40:17.827 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:40:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:40:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:40:18.100 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:40:18.164 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:40:23.675 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:40:23.676 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 1/164 2024-05-08T05:40:23.676 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2/164 2024-05-08T05:40:23.676 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 3/164 2024-05-08T05:40:23.676 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 4/164 2024-05-08T05:40:23.676 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 5/164 2024-05-08T05:40:23.676 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 6/164 2024-05-08T05:40:23.676 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 7/164 2024-05-08T05:40:23.676 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:40:23.676 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:40:23.676 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:40:23.676 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 11/164 2024-05-08T05:40:23.677 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 12/164 2024-05-08T05:40:23.677 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:40:23.677 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 14/164 2024-05-08T05:40:23.677 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 15/164 2024-05-08T05:40:23.677 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 16/164 2024-05-08T05:40:23.677 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 17/164 2024-05-08T05:40:23.677 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 18/164 2024-05-08T05:40:23.677 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 19/164 2024-05-08T05:40:23.677 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 20/164 2024-05-08T05:40:23.677 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 21/164 2024-05-08T05:40:23.677 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 22/164 2024-05-08T05:40:23.677 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 23/164 2024-05-08T05:40:23.677 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 24/164 2024-05-08T05:40:23.678 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 25/164 2024-05-08T05:40:23.678 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 26/164 2024-05-08T05:40:23.678 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 27/164 2024-05-08T05:40:23.678 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 28/164 2024-05-08T05:40:23.678 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 29/164 2024-05-08T05:40:23.678 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 30/164 2024-05-08T05:40:23.690 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 31/164 2024-05-08T05:40:23.691 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 32/164 2024-05-08T05:40:23.691 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 33/164 2024-05-08T05:40:23.691 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 34/164 2024-05-08T05:40:23.691 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 35/164 2024-05-08T05:40:23.691 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 36/164 2024-05-08T05:40:23.691 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 37/164 2024-05-08T05:40:23.691 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 38/164 2024-05-08T05:40:23.691 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 39/164 2024-05-08T05:40:23.691 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T05:40:23.691 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T05:40:23.691 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T05:40:23.691 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T05:40:23.692 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T05:40:23.692 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T05:40:23.692 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T05:40:23.692 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T05:40:23.692 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T05:40:23.692 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T05:40:23.692 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T05:40:23.692 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T05:40:23.692 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T05:40:23.692 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T05:40:23.692 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T05:40:23.692 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T05:40:23.692 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T05:40:23.693 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T05:40:23.693 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T05:40:23.693 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T05:40:23.693 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T05:40:23.693 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T05:40:23.693 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T05:40:23.693 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T05:40:23.693 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T05:40:23.693 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T05:40:23.693 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T05:40:23.693 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T05:40:23.693 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T05:40:23.693 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T05:40:23.694 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T05:40:23.694 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T05:40:23.694 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T05:40:23.694 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T05:40:23.694 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T05:40:23.694 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T05:40:23.694 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T05:40:23.694 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T05:40:23.694 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T05:40:23.694 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T05:40:23.694 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T05:40:23.694 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T05:40:23.694 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T05:40:23.695 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T05:40:23.695 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T05:40:23.695 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T05:40:23.695 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T05:40:23.695 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T05:40:23.695 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T05:40:23.695 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T05:40:23.695 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T05:40:23.695 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T05:40:23.695 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T05:40:23.695 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T05:40:23.695 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T05:40:23.695 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T05:40:23.695 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T05:40:23.696 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T05:40:23.696 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T05:40:23.696 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T05:40:23.696 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T05:40:23.696 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T05:40:23.696 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T05:40:23.696 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T05:40:23.696 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T05:40:23.696 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T05:40:23.696 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T05:40:23.696 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T05:40:23.696 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T05:40:23.696 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T05:40:23.696 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T05:40:23.697 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T05:40:23.697 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T05:40:23.697 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T05:40:23.697 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T05:40:23.697 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T05:40:23.697 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T05:40:23.697 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T05:40:23.697 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T05:40:23.697 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T05:40:23.697 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T05:40:23.697 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T05:40:23.697 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T05:40:23.697 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T05:40:23.697 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T05:40:23.698 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T05:40:23.698 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T05:40:23.698 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T05:40:23.698 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T05:40:23.698 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T05:40:23.698 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T05:40:23.698 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T05:40:23.698 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T05:40:23.698 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T05:40:23.698 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T05:40:23.698 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T05:40:23.698 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T05:40:23.698 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T05:40:23.699 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T05:40:23.699 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T05:40:23.699 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T05:40:23.699 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T05:40:23.699 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T05:40:23.699 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T05:40:23.699 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T05:40:23.699 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T05:40:23.699 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T05:40:23.699 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T05:40:23.699 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T05:40:23.699 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 149/164 2024-05-08T05:40:23.699 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T05:40:23.699 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 151/164 2024-05-08T05:40:23.700 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T05:40:23.700 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T05:40:23.700 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T05:40:23.700 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T05:40:23.700 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T05:40:23.700 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T05:40:23.700 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T05:40:23.700 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T05:40:23.700 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T05:40:23.700 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T05:40:23.700 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T05:40:23.700 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T05:40:25.972 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:40:25.972 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:40:25.972 INFO:teuthology.orchestra.run.smithi162.stdout:Upgraded: 2024-05-08T05:40:25.972 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T05:40:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T05:40:25.973 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-05-08T05:40:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T05:40:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T05:40:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T05:40:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T05:40:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.973 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:25.974 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T05:40:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T05:40:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T05:40:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T05:40:25.975 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T05:40:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T05:40:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T05:40:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T05:40:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T05:40:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T05:40:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T05:40:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T05:40:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T05:40:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T05:40:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T05:40:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T05:40:25.976 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T05:40:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T05:40:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T05:40:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T05:40:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T05:40:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T05:40:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T05:40:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T05:40:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T05:40:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T05:40:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T05:40:25.977 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T05:40:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T05:40:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T05:40:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T05:40:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T05:40:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T05:40:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T05:40:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T05:40:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T05:40:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T05:40:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T05:40:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T05:40:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T05:40:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T05:40:25.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T05:40:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T05:40:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T05:40:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T05:40:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T05:40:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T05:40:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T05:40:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T05:40:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T05:40:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T05:40:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T05:40:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T05:40:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T05:40:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T05:40:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T05:40:25.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T05:40:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T05:40:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T05:40:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T05:40:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T05:40:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T05:40:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T05:40:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T05:40:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T05:40:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T05:40:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T05:40:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T05:40:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T05:40:25.981 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T05:40:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T05:40:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T05:40:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T05:40:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T05:40:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T05:40:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T05:40:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T05:40:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T05:40:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T05:40:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T05:40:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T05:40:25.982 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T05:40:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T05:40:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T05:40:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T05:40:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T05:40:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T05:40:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T05:40:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T05:40:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T05:40:25.983 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T05:40:25.984 INFO:teuthology.orchestra.run.smithi162.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T05:40:25.984 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:40:25.984 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-05-08T05:40:26.413 DEBUG:teuthology.parallel:result is None 2024-05-08T05:40:26.414 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:40:26.633 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T05:40:26.705 INFO:teuthology.orchestra.run.smithi112.stdout:19.0.0-3538.gbee299f4.el9 2024-05-08T05:40:26.706 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gbee299f4.el9 2024-05-08T05:40:26.706 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gbee299f4 is installed. 2024-05-08T05:40:26.708 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:40:26.848 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T05:40:26.875 INFO:teuthology.orchestra.run.smithi150.stdout:19.0.0-3538.gbee299f4.el9 2024-05-08T05:40:26.875 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gbee299f4.el9 2024-05-08T05:40:26.875 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gbee299f4 is installed. 2024-05-08T05:40:26.877 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:40:27.119 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T05:40:27.142 INFO:teuthology.orchestra.run.smithi162.stdout:19.0.0-3538.gbee299f4.el9 2024-05-08T05:40:27.143 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gbee299f4.el9 2024-05-08T05:40:27.143 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gbee299f4 is installed. 2024-05-08T05:40:27.145 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T05:40:27.145 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:40:27.145 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T05:40:27.258 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-08T05:40:27.258 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T05:40:27.299 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-08T05:40:27.300 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T05:40:27.327 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T05:40:27.328 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:40:27.328 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T05:40:27.417 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T05:40:27.500 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-08T05:40:27.500 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T05:40:27.529 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T05:40:27.597 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-08T05:40:27.597 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T05:40:27.624 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T05:40:27.691 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T05:40:27.691 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:40:27.691 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T05:40:27.779 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T05:40:27.904 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-08T05:40:27.904 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T05:40:27.929 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T05:40:27.992 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-08T05:40:27.992 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T05:40:28.020 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T05:40:28.087 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T05:40:28.087 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:40:28.087 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T05:40:28.175 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T05:40:28.257 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-08T05:40:28.258 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T05:40:28.282 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T05:40:28.345 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-08T05:40:28.345 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T05:40:28.373 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T05:40:28.441 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-08T05:40:28.537 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-08T05:40:28.537 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:40:28.537 INFO:tasks.cephadm:Cluster fsid is 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:40:28.538 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-08T05:40:28.538 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.112', 'mon.b': '172.21.15.150', 'mon.c': '172.21.15.162'} 2024-05-08T05:40:28.538 INFO:tasks.cephadm:First mon is mon.a on smithi112 2024-05-08T05:40:28.538 INFO:tasks.cephadm:First mgr is x 2024-05-08T05:40:28.538 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-08T05:40:28.538 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname $(hostname -s) 2024-05-08T05:40:28.627 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname $(hostname -s) 2024-05-08T05:40:28.651 DEBUG:teuthology.orchestra.run.smithi162:> sudo hostname $(hostname -s) 2024-05-08T05:40:28.679 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-08T05:40:28.679 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:40:28.896 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79166/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3538-gbee299f4', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3538.gbee299f4'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-07 17:50:48.409840', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-57676-3', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-08T05:40:29.036 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-57676-3, sha1 bee299f4ac344e4c45e550d7a80abcadd7bc3d5a from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:40:29.037 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm 2024-05-08T05:40:29.037 DEBUG:teuthology.orchestra.run.smithi112:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:29.937 INFO:teuthology.orchestra.run.smithi112.stdout:-rw-r--r--. 1 ubuntu ubuntu 782795 May 8 05:40 /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:29.938 DEBUG:teuthology.orchestra.run.smithi150:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:30.673 INFO:teuthology.orchestra.run.smithi150.stdout:-rw-r--r--. 1 ubuntu ubuntu 782795 May 8 05:40 /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:30.673 DEBUG:teuthology.orchestra.run.smithi162:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:31.538 INFO:teuthology.orchestra.run.smithi162.stdout:-rw-r--r--. 1 ubuntu ubuntu 782795 May 8 05:40 /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:31.538 DEBUG:teuthology.orchestra.run.smithi112:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:31.598 DEBUG:teuthology.orchestra.run.smithi150:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:31.617 DEBUG:teuthology.orchestra.run.smithi162:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:31.651 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a on all hosts... 2024-05-08T05:40:31.652 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a pull 2024-05-08T05:40:31.657 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a pull 2024-05-08T05:40:31.659 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a pull 2024-05-08T05:40:33.246 INFO:teuthology.orchestra.run.smithi150.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:40:33.271 INFO:teuthology.orchestra.run.smithi162.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:40:33.697 INFO:teuthology.orchestra.run.smithi112.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:41:18.708 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-08T05:41:18.709 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev)", 2024-05-08T05:41:18.709 INFO:teuthology.orchestra.run.smithi150.stdout: "image_id": "352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce", 2024-05-08T05:41:18.709 INFO:teuthology.orchestra.run.smithi150.stdout: "repo_digests": [ 2024-05-08T05:41:18.709 INFO:teuthology.orchestra.run.smithi150.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c" 2024-05-08T05:41:18.709 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-08T05:41:18.709 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-08T05:41:32.118 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-05-08T05:41:32.118 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_version": "ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev)", 2024-05-08T05:41:32.118 INFO:teuthology.orchestra.run.smithi162.stdout: "image_id": "352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce", 2024-05-08T05:41:32.118 INFO:teuthology.orchestra.run.smithi162.stdout: "repo_digests": [ 2024-05-08T05:41:32.118 INFO:teuthology.orchestra.run.smithi162.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c" 2024-05-08T05:41:32.118 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-05-08T05:41:32.118 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-05-08T05:41:35.023 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:41:35.024 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev)", 2024-05-08T05:41:35.024 INFO:teuthology.orchestra.run.smithi112.stdout: "image_id": "352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce", 2024-05-08T05:41:35.024 INFO:teuthology.orchestra.run.smithi112.stdout: "repo_digests": [ 2024-05-08T05:41:35.024 INFO:teuthology.orchestra.run.smithi112.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c" 2024-05-08T05:41:35.024 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-08T05:41:35.024 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:41:35.090 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph 2024-05-08T05:41:35.179 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph 2024-05-08T05:41:35.211 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph 2024-05-08T05:41:35.243 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /etc/ceph 2024-05-08T05:41:35.333 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /etc/ceph 2024-05-08T05:41:35.361 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 777 /etc/ceph 2024-05-08T05:41:35.389 INFO:tasks.cephadm:Writing seed config... 2024-05-08T05:41:35.391 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-08T05:41:35.391 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-08T05:41:35.391 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-08T05:41:35.391 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-08T05:41:35.391 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-08T05:41:35.391 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-08T05:41:35.391 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-08T05:41:35.391 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-08T05:41:35.391 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-08T05:41:35.391 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-08T05:41:35.391 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-08T05:41:35.392 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-08T05:41:35.392 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-08T05:41:35.392 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-08T05:41:35.392 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-08T05:41:35.392 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-08T05:41:35.392 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-08T05:41:35.392 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-08T05:41:35.392 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-08T05:41:35.392 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-08T05:41:35.392 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-08T05:41:35.392 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-08T05:41:35.392 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-08T05:41:35.392 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-08T05:41:35.392 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-08T05:41:35.392 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-08T05:41:35.393 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-08T05:41:35.394 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-08T05:41:35.394 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-08T05:41:35.394 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-08T05:41:35.394 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-08T05:41:35.395 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:41:35.395 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-08T05:41:35.452 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 7a07ff60-0cfd-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-08T05:41:35.452 DEBUG:teuthology.orchestra.run.smithi112:mon.a> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mon.a.service 2024-05-08T05:41:35.499 DEBUG:teuthology.orchestra.run.smithi112:mgr.x> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.x.service 2024-05-08T05:41:35.546 INFO:tasks.cephadm:Bootstrapping... 2024-05-08T05:41:35.546 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a -v bootstrap --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.112 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:41:35.811 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:41:35.811 INFO:teuthology.orchestra.run.smithi112.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', '-v', 'bootstrap', '--fsid', '7a07ff60-0cfd-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.112', '--skip-admin-label'] 2024-05-08T05:41:35.811 INFO:teuthology.orchestra.run.smithi112.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-08T05:41:35.812 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying podman|docker is present... 2024-05-08T05:41:35.866 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-08T05:41:35.867 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying lvm2 is present... 2024-05-08T05:41:35.867 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying time synchronization is in place... 2024-05-08T05:41:35.882 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T05:41:35.882 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T05:41:35.895 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T05:41:35.896 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-05-08T05:41:35.913 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-05-08T05:41:35.933 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-05-08T05:41:35.933 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-05-08T05:41:35.933 INFO:teuthology.orchestra.run.smithi112.stdout:Repeating the final host check... 2024-05-08T05:41:35.993 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-08T05:41:35.993 INFO:teuthology.orchestra.run.smithi112.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-08T05:41:35.993 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl is present 2024-05-08T05:41:35.993 INFO:teuthology.orchestra.run.smithi112.stdout:lvcreate is present 2024-05-08T05:41:36.008 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T05:41:36.008 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T05:41:36.025 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T05:41:36.025 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-05-08T05:41:36.045 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-05-08T05:41:36.062 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-05-08T05:41:36.063 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-05-08T05:41:36.063 INFO:teuthology.orchestra.run.smithi112.stdout:Host looks OK 2024-05-08T05:41:36.063 INFO:teuthology.orchestra.run.smithi112.stdout:Cluster fsid: 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:36.063 INFO:teuthology.orchestra.run.smithi112.stdout:Acquiring lock 140686746119616 on /run/cephadm/7a07ff60-0cfd-11ef-bc97-c7b262605968.lock 2024-05-08T05:41:36.063 INFO:teuthology.orchestra.run.smithi112.stdout:Lock 140686746119616 acquired on /run/cephadm/7a07ff60-0cfd-11ef-bc97-c7b262605968.lock 2024-05-08T05:41:36.064 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 3300 ... 2024-05-08T05:41:36.064 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 6789 ... 2024-05-08T05:41:36.064 INFO:teuthology.orchestra.run.smithi112.stdout:Base mon IP(s) is [172.21.15.112:3300, 172.21.15.112:6789], mon addrv is [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-05-08T05:41:36.074 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.112 metric 100 2024-05-08T05:41:36.074 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.112 metric 100 2024-05-08T05:41:36.082 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-08T05:41:36.083 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-08T05:41:36.083 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1676sec hoplimit 64 pref medium 2024-05-08T05:41:36.091 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-08T05:41:36.091 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-08T05:41:36.091 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T05:41:36.091 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-08T05:41:36.091 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2024-05-08T05:41:36.091 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T05:41:36.092 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-05-08T05:41:36.092 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-05-08T05:41:36.092 INFO:teuthology.orchestra.run.smithi112.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-08T05:41:36.093 INFO:teuthology.orchestra.run.smithi112.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-08T05:41:36.095 INFO:teuthology.orchestra.run.smithi112.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:41:36.789 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce 2024-05-08T05:41:36.790 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:41:36.790 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Getting image source signatures 2024-05-08T05:41:36.790 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-08T05:41:36.790 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:f08a3efca46759c0efc03dda6bb4c9adf7daf77004822005f2f0da3e8d1877bb 2024-05-08T05:41:36.790 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying config sha256:352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce 2024-05-08T05:41:36.790 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-08T05:41:38.071 INFO:teuthology.orchestra.run.smithi112.stdout:ceph: stdout ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev) 2024-05-08T05:41:38.071 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph version: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev) 2024-05-08T05:41:38.071 INFO:teuthology.orchestra.run.smithi112.stdout:Extracting ceph user uid/gid from container image... 2024-05-08T05:41:39.445 INFO:teuthology.orchestra.run.smithi112.stdout:stat: stdout 167 167 2024-05-08T05:41:39.445 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial keys... 2024-05-08T05:41:40.801 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQCTEDtmaT4YOxAAqvXI/Fsk7iFUz3NZ1T8R5g== 2024-05-08T05:41:42.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQCVEDtmn5xrFBAA7BwBPLSl3pT5U7h5bsmBGA== 2024-05-08T05:41:43.234 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQCWEDtmIiBnIBAAUuHgUysULB6fa24n9m2DYA== 2024-05-08T05:41:43.236 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial monmap... 2024-05-08T05:41:44.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T05:41:44.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-08T05:41:44.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:44.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T05:41:44.844 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool for a [v2:172.21.15.112:3300,v1:172.21.15.112:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T05:41:44.844 INFO:teuthology.orchestra.run.smithi112.stdout:setting min_mon_release = quincy 2024-05-08T05:41:44.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: set fsid to 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:44.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T05:41:44.844 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:41:44.844 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mon... 2024-05-08T05:41:46.136 INFO:teuthology.orchestra.run.smithi112.stdout:create mon.a on 2024-05-08T05:41:46.537 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-08T05:41:46.780 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-08T05:41:47.085 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968.target. 2024-05-08T05:41:47.085 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968.target. 2024-05-08T05:41:47.498 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mon.a 2024-05-08T05:41:47.499 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-08T05:41:47.783 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968.target.wants/ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@.service. 2024-05-08T05:41:48.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:48 smithi112 podman[29615]: 2024-05-08 05:41:48.422850358 +0000 UTC m=+0.368390185 container start 806961f125f510825887dc525aa4e927522f92c55dd26df4e83e9d06160d3664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:41:48.579 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:41:48.579 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-05-08T05:41:48.596 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:41:48.596 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-05-08T05:41:48.596 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-05-08T05:41:48.597 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T05:41:48.598 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon to start... 2024-05-08T05:41:48.598 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon... 2024-05-08T05:41:48.762 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:48 smithi112 ceph-mon[29634]: mkfs 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:48.762 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:48 smithi112 ceph-mon[29634]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T05:41:48.762 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:48 smithi112 bash[29615]: 806961f125f510825887dc525aa4e927522f92c55dd26df4e83e9d06160d3664 2024-05-08T05:41:48.762 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:48 smithi112 systemd[1]: Started Ceph mon.a for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:41:49.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:49 smithi112 ceph-mon[29634]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T05:41:49.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:49 smithi112 ceph-mon[29634]: monmap epoch 1 2024-05-08T05:41:49.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:49 smithi112 ceph-mon[29634]: fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:49.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:49 smithi112 ceph-mon[29634]: last_changed 2024-05-08T05:41:43.845828+0000 2024-05-08T05:41:49.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:49 smithi112 ceph-mon[29634]: created 2024-05-08T05:41:43.845828+0000 2024-05-08T05:41:49.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:49 smithi112 ceph-mon[29634]: min_mon_release 19 (squid) 2024-05-08T05:41:49.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:49 smithi112 ceph-mon[29634]: election_strategy: 1 2024-05-08T05:41:49.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:49 smithi112 ceph-mon[29634]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-08T05:41:49.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:49 smithi112 ceph-mon[29634]: fsmap 2024-05-08T05:41:49.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:49 smithi112 ceph-mon[29634]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T05:41:49.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:49 smithi112 ceph-mon[29634]: mgrmap e1: no daemons active 2024-05-08T05:41:49.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:49 smithi112 ceph-mon[29634]: from='client.? 172.21.15.112:0/1107939585' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-08T05:41:49.984 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout cluster: 2024-05-08T05:41:49.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout id: 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:49.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-08T05:41:49.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:49.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout services: 2024-05-08T05:41:49.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.790156s) 2024-05-08T05:41:49.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-08T05:41:49.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-08T05:41:49.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:49.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout data: 2024-05-08T05:41:49.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-08T05:41:49.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-08T05:41:49.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-08T05:41:49.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pgs: 2024-05-08T05:41:49.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:49.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.197+0000 7fc5f7200700 1 Processor -- start 2024-05-08T05:41:49.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.197+0000 7fc5f7200700 1 -- start start 2024-05-08T05:41:49.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.197+0000 7fc5f7200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5f010b070 0x7fc5f010b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.198+0000 7fc5f7200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f010ba60 con 0x7fc5f010b070 2024-05-08T05:41:49.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.198+0000 7fc5efe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5f010b070 0x7fc5f010b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.199+0000 7fc5efe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5f010b070 0x7fc5f010b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44694/0 (socket says 172.21.15.112:44694) 2024-05-08T05:41:49.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.199+0000 7fc5efe00700 1 -- 172.21.15.112:0/3514879461 learned_addr learned my addr 172.21.15.112:0/3514879461 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:41:49.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.199+0000 7fc5efe00700 1 -- 172.21.15.112:0/3514879461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f010c270 con 0x7fc5f010b070 2024-05-08T05:41:49.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.199+0000 7fc5efe00700 1 --2- 172.21.15.112:0/3514879461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5f010b070 0x7fc5f010b490 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00098a0 tx=0x7fc5e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7a16547ebb2598 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.200+0000 7fc5eea00700 1 -- 172.21.15.112:0/3514879461 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5e0004160 con 0x7fc5f010b070 2024-05-08T05:41:49.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.200+0000 7fc5eea00700 1 -- 172.21.15.112:0/3514879461 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5e0004300 con 0x7fc5f010b070 2024-05-08T05:41:49.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.200+0000 7fc5eea00700 1 -- 172.21.15.112:0/3514879461 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5e00036a0 con 0x7fc5f010b070 2024-05-08T05:41:49.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.200+0000 7fc5f7200700 1 -- 172.21.15.112:0/3514879461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5f010b070 msgr2=0x7fc5f010b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.200+0000 7fc5f7200700 1 --2- 172.21.15.112:0/3514879461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5f010b070 0x7fc5f010b490 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00098a0 tx=0x7fc5e0009bb0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.200+0000 7fc5f7200700 1 -- 172.21.15.112:0/3514879461 shutdown_connections 2024-05-08T05:41:49.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.200+0000 7fc5f7200700 1 --2- 172.21.15.112:0/3514879461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5f010b070 0x7fc5f010b490 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.200+0000 7fc5f7200700 1 -- 172.21.15.112:0/3514879461 >> 172.21.15.112:0/3514879461 conn(0x7fc5f007e440 msgr2=0x7fc5f007e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:49.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.201+0000 7fc5f7200700 1 -- 172.21.15.112:0/3514879461 shutdown_connections 2024-05-08T05:41:49.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.201+0000 7fc5f7200700 1 -- 172.21.15.112:0/3514879461 wait complete. 2024-05-08T05:41:49.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.202+0000 7fc5f7200700 1 Processor -- start 2024-05-08T05:41:49.994 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.202+0000 7fc5f7200700 1 -- start start 2024-05-08T05:41:49.994 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.202+0000 7fc5f7200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5f010b070 0x7fc5f019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.994 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.202+0000 7fc5f7200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f010c7a0 con 0x7fc5f010b070 2024-05-08T05:41:49.994 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.203+0000 7fc5efe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5f010b070 0x7fc5f019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.994 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.203+0000 7fc5efe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5f010b070 0x7fc5f019e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44702/0 (socket says 172.21.15.112:44702) 2024-05-08T05:41:49.994 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.203+0000 7fc5efe00700 1 -- 172.21.15.112:0/1107939585 learned_addr learned my addr 172.21.15.112:0/1107939585 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:41:49.994 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.203+0000 7fc5efe00700 1 -- 172.21.15.112:0/1107939585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f019ed50 con 0x7fc5f010b070 2024-05-08T05:41:49.994 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.203+0000 7fc5efe00700 1 --2- 172.21.15.112:0/1107939585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5f010b070 0x7fc5f019e810 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0000c00 tx=0x7fc5e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.994 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.204+0000 7fc5ecc00700 1 -- 172.21.15.112:0/1107939585 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5e0003db0 con 0x7fc5f010b070 2024-05-08T05:41:49.994 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.204+0000 7fc5ecc00700 1 -- 172.21.15.112:0/1107939585 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5e0011630 con 0x7fc5f010b070 2024-05-08T05:41:49.994 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.204+0000 7fc5ecc00700 1 -- 172.21.15.112:0/1107939585 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5e0011950 con 0x7fc5f010b070 2024-05-08T05:41:49.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.204+0000 7fc5f7200700 1 -- 172.21.15.112:0/1107939585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f019efe0 con 0x7fc5f010b070 2024-05-08T05:41:49.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.204+0000 7fc5f7200700 1 -- 172.21.15.112:0/1107939585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f01a1cd0 con 0x7fc5f010b070 2024-05-08T05:41:49.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.204+0000 7fc5ecc00700 1 -- 172.21.15.112:0/1107939585 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc5e001a460 con 0x7fc5f010b070 2024-05-08T05:41:49.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.204+0000 7fc5ecc00700 1 -- 172.21.15.112:0/1107939585 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc5e001b780 con 0x7fc5f010b070 2024-05-08T05:41:49.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.205+0000 7fc5f7200700 1 -- 172.21.15.112:0/1107939585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f0064b00 con 0x7fc5f010b070 2024-05-08T05:41:49.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.208+0000 7fc5ecc00700 1 -- 172.21.15.112:0/1107939585 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc5e0031020 con 0x7fc5f010b070 2024-05-08T05:41:49.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.287+0000 7fc5f7200700 1 -- 172.21.15.112:0/1107939585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc5f002e810 con 0x7fc5f010b070 2024-05-08T05:41:49.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.287+0000 7fc5ecc00700 1 -- 172.21.15.112:0/1107939585 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fc5e001ba20 con 0x7fc5f010b070 2024-05-08T05:41:49.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.289+0000 7fc5f7200700 1 -- 172.21.15.112:0/1107939585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5f010b070 msgr2=0x7fc5f019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.289+0000 7fc5f7200700 1 --2- 172.21.15.112:0/1107939585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5f010b070 0x7fc5f019e810 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0000c00 tx=0x7fc5e0004750 comp rx=0 tx=0).stop 2024-05-08T05:41:49.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.289+0000 7fc5f7200700 1 -- 172.21.15.112:0/1107939585 shutdown_connections 2024-05-08T05:41:49.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.289+0000 7fc5f7200700 1 --2- 172.21.15.112:0/1107939585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5f010b070 0x7fc5f019e810 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.289+0000 7fc5f7200700 1 -- 172.21.15.112:0/1107939585 >> 172.21.15.112:0/1107939585 conn(0x7fc5f007e440 msgr2=0x7fc5f0195470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:49.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.289+0000 7fc5f7200700 1 -- 172.21.15.112:0/1107939585 shutdown_connections 2024-05-08T05:41:49.996 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:49.289+0000 7fc5f7200700 1 -- 172.21.15.112:0/1107939585 wait complete. 2024-05-08T05:41:49.996 INFO:teuthology.orchestra.run.smithi112.stdout:mon is available 2024-05-08T05:41:49.996 INFO:teuthology.orchestra.run.smithi112.stdout:Assimilating anything we can from ceph.conf... 2024-05-08T05:41:51.059 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:50 smithi112 ceph-mon[29634]: from='client.? 172.21.15.112:0/2848956159' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T05:41:51.059 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:50 smithi112 ceph-mon[29634]: from='client.? 172.21.15.112:0/2848956159' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-08T05:41:51.368 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:51.368 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T05:41:51.368 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:51.369 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-08T05:41:51.369 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-05-08T05:41:51.369 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T05:41:51.369 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T05:41:51.369 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T05:41:51.369 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T05:41:51.369 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:51.369 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T05:41:51.369 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T05:41:51.369 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:51.369 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T05:41:51.369 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-08T05:41:51.369 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T05:41:51.370 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T05:41:51.370 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T05:41:51.370 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T05:41:51.370 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T05:41:51.370 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.574+0000 7f749c200700 1 Processor -- start 2024-05-08T05:41:51.370 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.574+0000 7f749c200700 1 -- start start 2024-05-08T05:41:51.370 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.575+0000 7f749c200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749410b050 0x7f749410b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.370 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.575+0000 7f749c200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f749410ba40 con 0x7f749410b050 2024-05-08T05:41:51.370 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.575+0000 7f7499800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749410b050 0x7f749410b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.370 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.575+0000 7f7499800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749410b050 0x7f749410b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44704/0 (socket says 172.21.15.112:44704) 2024-05-08T05:41:51.370 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.575+0000 7f7499800700 1 -- 172.21.15.112:0/3251470186 learned_addr learned my addr 172.21.15.112:0/3251470186 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:41:51.370 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.576+0000 7f7499800700 1 -- 172.21.15.112:0/3251470186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749410c250 con 0x7f749410b050 2024-05-08T05:41:51.370 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.576+0000 7f7499800700 1 --2- 172.21.15.112:0/3251470186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749410b050 0x7f749410b470 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f74880098a0 tx=0x7f7488009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b61bc9a0de7a78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:51.371 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.576+0000 7f7493e00700 1 -- 172.21.15.112:0/3251470186 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7488004160 con 0x7f749410b050 2024-05-08T05:41:51.371 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.576+0000 7f7493e00700 1 -- 172.21.15.112:0/3251470186 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7488004300 con 0x7f749410b050 2024-05-08T05:41:51.371 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.576+0000 7f7493e00700 1 -- 172.21.15.112:0/3251470186 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74880036a0 con 0x7f749410b050 2024-05-08T05:41:51.371 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.577+0000 7f749c200700 1 -- 172.21.15.112:0/3251470186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749410b050 msgr2=0x7f749410b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.371 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.577+0000 7f749c200700 1 --2- 172.21.15.112:0/3251470186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749410b050 0x7f749410b470 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f74880098a0 tx=0x7f7488009bb0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.371 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.577+0000 7f749c200700 1 -- 172.21.15.112:0/3251470186 shutdown_connections 2024-05-08T05:41:51.371 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.577+0000 7f749c200700 1 --2- 172.21.15.112:0/3251470186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749410b050 0x7f749410b470 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.371 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.577+0000 7f749c200700 1 -- 172.21.15.112:0/3251470186 >> 172.21.15.112:0/3251470186 conn(0x7f749407e460 msgr2=0x7f749407e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:51.371 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.577+0000 7f749c200700 1 -- 172.21.15.112:0/3251470186 shutdown_connections 2024-05-08T05:41:51.371 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.577+0000 7f749c200700 1 -- 172.21.15.112:0/3251470186 wait complete. 2024-05-08T05:41:51.371 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.578+0000 7f749c200700 1 Processor -- start 2024-05-08T05:41:51.371 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.578+0000 7f749c200700 1 -- start start 2024-05-08T05:41:51.371 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.579+0000 7f749c200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749410b050 0x7f749419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.372 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.579+0000 7f749c200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f749410c780 con 0x7f749410b050 2024-05-08T05:41:51.372 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.579+0000 7f7499800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749410b050 0x7f749419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.372 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.579+0000 7f7499800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749410b050 0x7f749419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44708/0 (socket says 172.21.15.112:44708) 2024-05-08T05:41:51.372 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.579+0000 7f7499800700 1 -- 172.21.15.112:0/2848956159 learned_addr learned my addr 172.21.15.112:0/2848956159 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:41:51.372 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.579+0000 7f7499800700 1 -- 172.21.15.112:0/2848956159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749419eca0 con 0x7f749410b050 2024-05-08T05:41:51.372 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.580+0000 7f7499800700 1 --2- 172.21.15.112:0/2848956159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749410b050 0x7f749419e760 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f749410bce0 tx=0x7f7488004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:51.372 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.580+0000 7f7492000700 1 -- 172.21.15.112:0/2848956159 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7488003db0 con 0x7f749410b050 2024-05-08T05:41:51.372 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.580+0000 7f7492000700 1 -- 172.21.15.112:0/2848956159 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7488011630 con 0x7f749410b050 2024-05-08T05:41:51.372 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.580+0000 7f7492000700 1 -- 172.21.15.112:0/2848956159 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7488011930 con 0x7f749410b050 2024-05-08T05:41:51.372 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.580+0000 7f749c200700 1 -- 172.21.15.112:0/2848956159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749419ef30 con 0x7f749410b050 2024-05-08T05:41:51.372 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.580+0000 7f749c200700 1 -- 172.21.15.112:0/2848956159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74941a1c20 con 0x7f749410b050 2024-05-08T05:41:51.372 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.580+0000 7f7492000700 1 -- 172.21.15.112:0/2848956159 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7488011ad0 con 0x7f749410b050 2024-05-08T05:41:51.372 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.580+0000 7f7492000700 1 -- 172.21.15.112:0/2848956159 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f748801b450 con 0x7f749410b050 2024-05-08T05:41:51.372 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.581+0000 7f749c200700 1 -- 172.21.15.112:0/2848956159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7494064b00 con 0x7f749410b050 2024-05-08T05:41:51.372 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.584+0000 7f7492000700 1 -- 172.21.15.112:0/2848956159 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7488004260 con 0x7f749410b050 2024-05-08T05:41:51.373 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.660+0000 7f749c200700 1 -- 172.21.15.112:0/2848956159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f749402e660 con 0x7f749410b050 2024-05-08T05:41:51.373 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.668+0000 7f7492000700 1 -- 172.21.15.112:0/2848956159 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+494 (secure 0 0 0) 0x7f7488025480 con 0x7f749410b050 2024-05-08T05:41:51.373 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.669+0000 7f7492000700 1 -- 172.21.15.112:0/2848956159 <== mon.0 v2:172.21.15.112:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7488025660 con 0x7f749410b050 2024-05-08T05:41:51.373 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.669+0000 7f749c200700 1 -- 172.21.15.112:0/2848956159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749410b050 msgr2=0x7f749419e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.373 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.669+0000 7f749c200700 1 --2- 172.21.15.112:0/2848956159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749410b050 0x7f749419e760 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f749410bce0 tx=0x7f7488004750 comp rx=0 tx=0).stop 2024-05-08T05:41:51.373 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.669+0000 7f749c200700 1 -- 172.21.15.112:0/2848956159 shutdown_connections 2024-05-08T05:41:51.373 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.669+0000 7f749c200700 1 --2- 172.21.15.112:0/2848956159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f749410b050 0x7f749419e760 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.373 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.669+0000 7f749c200700 1 -- 172.21.15.112:0/2848956159 >> 172.21.15.112:0/2848956159 conn(0x7f749407e460 msgr2=0x7f7494195400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:51.373 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.670+0000 7f749c200700 1 -- 172.21.15.112:0/2848956159 shutdown_connections 2024-05-08T05:41:51.373 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:50.670+0000 7f749c200700 1 -- 172.21.15.112:0/2848956159 wait complete. 2024-05-08T05:41:51.373 INFO:teuthology.orchestra.run.smithi112.stdout:Generating new minimal ceph.conf... 2024-05-08T05:41:52.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:52 smithi112 ceph-mon[29634]: from='client.? 172.21.15.112:0/3358175640' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:41:53.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.012+0000 7faa53200700 1 Processor -- start 2024-05-08T05:41:53.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.012+0000 7faa53200700 1 -- start start 2024-05-08T05:41:53.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.013+0000 7faa53200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa4c07ede0 0x7faa4c07d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.013+0000 7faa53200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c07d810 con 0x7faa4c07ede0 2024-05-08T05:41:53.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.013+0000 7faa51e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa4c07ede0 0x7faa4c07d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.013+0000 7faa51e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa4c07ede0 0x7faa4c07d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44716/0 (socket says 172.21.15.112:44716) 2024-05-08T05:41:53.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.013+0000 7faa51e00700 1 -- 172.21.15.112:0/2596042679 learned_addr learned my addr 172.21.15.112:0/2596042679 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:41:53.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.014+0000 7faa51e00700 1 -- 172.21.15.112:0/2596042679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4c07d990 con 0x7faa4c07ede0 2024-05-08T05:41:53.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.014+0000 7faa51e00700 1 --2- 172.21.15.112:0/2596042679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa4c07ede0 0x7faa4c07d240 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faa400098a0 tx=0x7faa40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8965270e7ad2d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:53.048 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.014+0000 7faa50a00700 1 -- 172.21.15.112:0/2596042679 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa40004160 con 0x7faa4c07ede0 2024-05-08T05:41:53.048 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.015+0000 7faa50a00700 1 -- 172.21.15.112:0/2596042679 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faa40004300 con 0x7faa4c07ede0 2024-05-08T05:41:53.048 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.015+0000 7faa50a00700 1 -- 172.21.15.112:0/2596042679 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa40011af0 con 0x7faa4c07ede0 2024-05-08T05:41:53.048 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.015+0000 7faa53200700 1 -- 172.21.15.112:0/2596042679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa4c07ede0 msgr2=0x7faa4c07d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:53.048 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.015+0000 7faa53200700 1 --2- 172.21.15.112:0/2596042679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa4c07ede0 0x7faa4c07d240 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faa400098a0 tx=0x7faa40009bb0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.048 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.015+0000 7faa53200700 1 -- 172.21.15.112:0/2596042679 shutdown_connections 2024-05-08T05:41:53.048 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.015+0000 7faa53200700 1 --2- 172.21.15.112:0/2596042679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa4c07ede0 0x7faa4c07d240 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.048 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.016+0000 7faa53200700 1 -- 172.21.15.112:0/2596042679 >> 172.21.15.112:0/2596042679 conn(0x7faa4c104250 msgr2=0x7faa4c106670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:53.048 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.016+0000 7faa53200700 1 -- 172.21.15.112:0/2596042679 shutdown_connections 2024-05-08T05:41:53.048 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.016+0000 7faa53200700 1 -- 172.21.15.112:0/2596042679 wait complete. 2024-05-08T05:41:53.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.017+0000 7faa53200700 1 Processor -- start 2024-05-08T05:41:53.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.017+0000 7faa53200700 1 -- start start 2024-05-08T05:41:53.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.017+0000 7faa53200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa4c07ede0 0x7faa4c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.017+0000 7faa53200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c10f220 con 0x7faa4c07ede0 2024-05-08T05:41:53.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.018+0000 7faa51e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa4c07ede0 0x7faa4c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.018+0000 7faa51e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa4c07ede0 0x7faa4c19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44724/0 (socket says 172.21.15.112:44724) 2024-05-08T05:41:53.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.018+0000 7faa51e00700 1 -- 172.21.15.112:0/3358175640 learned_addr learned my addr 172.21.15.112:0/3358175640 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:41:53.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.018+0000 7faa51e00700 1 -- 172.21.15.112:0/3358175640 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4c19f330 con 0x7faa4c07ede0 2024-05-08T05:41:53.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.018+0000 7faa51e00700 1 --2- 172.21.15.112:0/3358175640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa4c07ede0 0x7faa4c19edf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faa40009fb0 tx=0x7faa40011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:53.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.019+0000 7faa46a00700 1 -- 172.21.15.112:0/3358175640 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa400036a0 con 0x7faa4c07ede0 2024-05-08T05:41:53.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.019+0000 7faa46a00700 1 -- 172.21.15.112:0/3358175640 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faa4001a430 con 0x7faa4c07ede0 2024-05-08T05:41:53.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.019+0000 7faa46a00700 1 -- 172.21.15.112:0/3358175640 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa40019550 con 0x7faa4c07ede0 2024-05-08T05:41:53.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.019+0000 7faa53200700 1 -- 172.21.15.112:0/3358175640 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4c19f5c0 con 0x7faa4c07ede0 2024-05-08T05:41:53.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.019+0000 7faa53200700 1 -- 172.21.15.112:0/3358175640 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa4c1a22b0 con 0x7faa4c07ede0 2024-05-08T05:41:53.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.019+0000 7faa46a00700 1 -- 172.21.15.112:0/3358175640 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faa40019b90 con 0x7faa4c07ede0 2024-05-08T05:41:53.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.019+0000 7faa46a00700 1 -- 172.21.15.112:0/3358175640 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faa400248a0 con 0x7faa4c07ede0 2024-05-08T05:41:53.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.020+0000 7faa53200700 1 -- 172.21.15.112:0/3358175640 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa4c064b00 con 0x7faa4c07ede0 2024-05-08T05:41:53.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.023+0000 7faa46a00700 1 -- 172.21.15.112:0/3358175640 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7faa40043ec0 con 0x7faa4c07ede0 2024-05-08T05:41:53.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.100+0000 7faa53200700 1 -- 172.21.15.112:0/3358175640 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7faa4c1a27f0 con 0x7faa4c07ede0 2024-05-08T05:41:53.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.100+0000 7faa46a00700 1 -- 172.21.15.112:0/3358175640 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7faa40032030 con 0x7faa4c07ede0 2024-05-08T05:41:53.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.102+0000 7faa53200700 1 -- 172.21.15.112:0/3358175640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa4c07ede0 msgr2=0x7faa4c19edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:53.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.102+0000 7faa53200700 1 --2- 172.21.15.112:0/3358175640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa4c07ede0 0x7faa4c19edf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faa40009fb0 tx=0x7faa40011940 comp rx=0 tx=0).stop 2024-05-08T05:41:53.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.102+0000 7faa53200700 1 -- 172.21.15.112:0/3358175640 shutdown_connections 2024-05-08T05:41:53.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.102+0000 7faa53200700 1 --2- 172.21.15.112:0/3358175640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa4c07ede0 0x7faa4c19edf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.051 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.102+0000 7faa53200700 1 -- 172.21.15.112:0/3358175640 >> 172.21.15.112:0/3358175640 conn(0x7faa4c104250 msgr2=0x7faa4c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:53.051 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.102+0000 7faa53200700 1 -- 172.21.15.112:0/3358175640 shutdown_connections 2024-05-08T05:41:53.051 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:52.102+0000 7faa53200700 1 -- 172.21.15.112:0/3358175640 wait complete. 2024-05-08T05:41:53.051 INFO:teuthology.orchestra.run.smithi112.stdout:Restarting the monitor... 2024-05-08T05:41:53.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:53 smithi112 systemd[1]: Stopping Ceph mon.a for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:41:59.230 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:58 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a[29630]: 2024-05-08T05:41:58.971+0000 7f5159000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:41:59.230 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:58 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a[29630]: 2024-05-08T05:41:58.971+0000 7f5159000700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-08T05:41:59.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:41:59 smithi112 podman[29878]: 2024-05-08 05:41:59.229875113 +0000 UTC m=+5.280940393 container died 806961f125f510825887dc525aa4e927522f92c55dd26df4e83e9d06160d3664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-08T05:42:01.563 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:01 smithi112 podman[29878]: 2024-05-08 05:42:01.325275402 +0000 UTC m=+7.376340692 container cleanup 806961f125f510825887dc525aa4e927522f92c55dd26df4e83e9d06160d3664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:42:01.563 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:01 smithi112 bash[29878]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a 2024-05-08T05:42:01.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:01 smithi112 podman[29891]: 2024-05-08 05:42:01.559571946 +0000 UTC m=+2.318619623 container remove 806961f125f510825887dc525aa4e927522f92c55dd26df4e83e9d06160d3664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:42:02.471 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 systemd[1]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-08T05:42:02.472 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 systemd[1]: Stopped Ceph mon.a for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:42:02.472 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 systemd[1]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mon.a.service: Consumed 1.976s CPU time. 2024-05-08T05:42:02.472 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 systemd[1]: Starting Ceph mon.a for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:42:02.732 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 podman[29984]: 2024-05-08 05:42:02.37703086 +0000 UTC m=+0.032753721 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:42:02.732 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 podman[29984]: 2024-05-08 05:42:02.485396891 +0000 UTC m=+0.141119795 container create 11e2e06de9b6c2d3298040188484e2e2f6ab0b727158e999cc81a7947a159be6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:42:02.732 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 podman[29984]: 2024-05-08 05:42:02.721883254 +0000 UTC m=+0.377606159 container init 11e2e06de9b6c2d3298040188484e2e2f6ab0b727158e999cc81a7947a159be6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:42:02.839 INFO:teuthology.orchestra.run.smithi112.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-08T05:42:02.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 podman[29984]: 2024-05-08 05:42:02.732056242 +0000 UTC m=+0.387779147 container start 11e2e06de9b6c2d3298040188484e2e2f6ab0b727158e999cc81a7947a159be6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:42:02.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T05:42:02.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev), process ceph-mon, pid 2 2024-05-08T05:42:02.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: pidfile_write: ignore empty --pid-file 2024-05-08T05:42:02.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: load: jerasure load: lrc 2024-05-08T05:42:02.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: RocksDB version: 7.9.2 2024-05-08T05:42:02.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Git sha 0 2024-05-08T05:42:02.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Compile date 2024-05-07 11:18:56 2024-05-08T05:42:02.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: DB SUMMARY 2024-05-08T05:42:02.992 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: DB Session ID: S32M91S8MR1R6AUB7O4Z 2024-05-08T05:42:02.992 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: CURRENT file: CURRENT 2024-05-08T05:42:02.992 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T05:42:02.992 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-08T05:42:02.992 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-08T05:42:02.992 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104472 ; 2024-05-08T05:42:02.992 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.error_if_exists: 0 2024-05-08T05:42:02.992 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.create_if_missing: 0 2024-05-08T05:42:02.992 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.paranoid_checks: 1 2024-05-08T05:42:02.992 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T05:42:02.992 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T05:42:02.992 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T05:42:02.992 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.env: 0x55fb027a7cc0 2024-05-08T05:42:02.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T05:42:02.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.info_log: 0x55fb09c9fde0 2024-05-08T05:42:02.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T05:42:02.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.statistics: (nil) 2024-05-08T05:42:02.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.use_fsync: 0 2024-05-08T05:42:02.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_log_file_size: 0 2024-05-08T05:42:02.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T05:42:02.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T05:42:02.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T05:42:02.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T05:42:02.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.allow_fallocate: 1 2024-05-08T05:42:02.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T05:42:02.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T05:42:02.994 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.use_direct_reads: 0 2024-05-08T05:42:02.994 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T05:42:02.994 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T05:42:02.994 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.db_log_dir: 2024-05-08T05:42:02.994 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.wal_dir: 2024-05-08T05:42:02.994 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T05:42:02.994 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T05:42:02.994 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T05:42:02.994 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T05:42:02.994 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T05:42:02.994 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T05:42:02.994 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T05:42:02.994 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T05:42:02.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.write_buffer_manager: 0x55fb09c710e0 2024-05-08T05:42:02.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T05:42:02.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T05:42:02.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T05:42:02.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T05:42:02.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T05:42:02.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T05:42:02.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T05:42:02.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T05:42:02.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.unordered_write: 0 2024-05-08T05:42:02.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T05:42:02.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T05:42:02.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T05:42:02.995 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T05:42:02.996 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.row_cache: None 2024-05-08T05:42:02.996 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.wal_filter: None 2024-05-08T05:42:02.996 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T05:42:02.996 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T05:42:02.996 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.two_write_queues: 0 2024-05-08T05:42:02.996 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T05:42:02.996 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.wal_compression: 0 2024-05-08T05:42:02.996 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.atomic_flush: 0 2024-05-08T05:42:02.996 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T05:42:02.996 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T05:42:02.996 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T05:42:02.997 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.log_readahead_size: 0 2024-05-08T05:42:02.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T05:42:02.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T05:42:02.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T05:42:02.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T05:42:02.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T05:42:02.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T05:42:02.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T05:42:02.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_background_jobs: 2 2024-05-08T05:42:02.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_background_compactions: -1 2024-05-08T05:42:02.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_subcompactions: 1 2024-05-08T05:42:02.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T05:42:02.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T05:42:02.998 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T05:42:02.999 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T05:42:02.999 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T05:42:02.999 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T05:42:02.999 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T05:42:02.999 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T05:42:02.999 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_open_files: -1 2024-05-08T05:42:02.999 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T05:42:02.999 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T05:42:02.999 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T05:42:02.999 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T05:42:02.999 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_background_flushes: -1 2024-05-08T05:42:02.999 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Compression algorithms supported: 2024-05-08T05:42:02.999 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T05:42:03.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: kZSTD supported: 0 2024-05-08T05:42:03.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: kXpressCompression supported: 0 2024-05-08T05:42:03.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T05:42:03.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: kZlibCompression supported: 1 2024-05-08T05:42:03.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: kSnappyCompression supported: 1 2024-05-08T05:42:03.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: kLZ4Compression supported: 1 2024-05-08T05:42:03.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: kBZip2Compression supported: 0 2024-05-08T05:42:03.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T05:42:03.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T05:42:03.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-08T05:42:03.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T05:42:03.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T05:42:03.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.merge_operator: 2024-05-08T05:42:03.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compaction_filter: None 2024-05-08T05:42:03.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T05:42:03.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T05:42:03.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T05:42:03.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T05:42:03.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fb09c9ffa0) 2024-05-08T05:42:03.001 INFO:journalctl@ceph.mon.a.smithi112.stdout: cache_index_and_filter_blocks: 1 2024-05-08T05:42:03.001 INFO:journalctl@ceph.mon.a.smithi112.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T05:42:03.001 INFO:journalctl@ceph.mon.a.smithi112.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T05:42:03.001 INFO:journalctl@ceph.mon.a.smithi112.stdout: pin_top_level_index_and_filter: 1 2024-05-08T05:42:03.001 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_type: 0 2024-05-08T05:42:03.001 INFO:journalctl@ceph.mon.a.smithi112.stdout: data_block_index_type: 0 2024-05-08T05:42:03.001 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_shortening: 1 2024-05-08T05:42:03.001 INFO:journalctl@ceph.mon.a.smithi112.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T05:42:03.001 INFO:journalctl@ceph.mon.a.smithi112.stdout: checksum: 4 2024-05-08T05:42:03.001 INFO:journalctl@ceph.mon.a.smithi112.stdout: no_block_cache: 0 2024-05-08T05:42:03.002 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache: 0x55fb09ca31f0 2024-05-08T05:42:03.002 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_name: BinnedLRUCache 2024-05-08T05:42:03.002 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_options: 2024-05-08T05:42:03.002 INFO:journalctl@ceph.mon.a.smithi112.stdout: capacity : 536870912 2024-05-08T05:42:03.002 INFO:journalctl@ceph.mon.a.smithi112.stdout: num_shard_bits : 4 2024-05-08T05:42:03.002 INFO:journalctl@ceph.mon.a.smithi112.stdout: strict_capacity_limit : 0 2024-05-08T05:42:03.002 INFO:journalctl@ceph.mon.a.smithi112.stdout: high_pri_pool_ratio: 0.000 2024-05-08T05:42:03.002 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_compressed: (nil) 2024-05-08T05:42:03.002 INFO:journalctl@ceph.mon.a.smithi112.stdout: persistent_cache: (nil) 2024-05-08T05:42:03.002 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_size: 4096 2024-05-08T05:42:03.002 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_size_deviation: 10 2024-05-08T05:42:03.002 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_restart_interval: 16 2024-05-08T05:42:03.002 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_block_restart_interval: 1 2024-05-08T05:42:03.003 INFO:journalctl@ceph.mon.a.smithi112.stdout: metadata_block_size: 4096 2024-05-08T05:42:03.003 INFO:journalctl@ceph.mon.a.smithi112.stdout: partition_filters: 0 2024-05-08T05:42:03.003 INFO:journalctl@ceph.mon.a.smithi112.stdout: use_delta_encoding: 1 2024-05-08T05:42:03.003 INFO:journalctl@ceph.mon.a.smithi112.stdout: filter_policy: bloomfilter 2024-05-08T05:42:03.003 INFO:journalctl@ceph.mon.a.smithi112.stdout: whole_key_filtering: 1 2024-05-08T05:42:03.003 INFO:journalctl@ceph.mon.a.smithi112.stdout: verify_compression: 0 2024-05-08T05:42:03.003 INFO:journalctl@ceph.mon.a.smithi112.stdout: read_amp_bytes_per_bit: 0 2024-05-08T05:42:03.003 INFO:journalctl@ceph.mon.a.smithi112.stdout: format_version: 5 2024-05-08T05:42:03.003 INFO:journalctl@ceph.mon.a.smithi112.stdout: enable_index_compression: 1 2024-05-08T05:42:03.003 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_align: 0 2024-05-08T05:42:03.003 INFO:journalctl@ceph.mon.a.smithi112.stdout: max_auto_readahead_size: 262144 2024-05-08T05:42:03.003 INFO:journalctl@ceph.mon.a.smithi112.stdout: prepopulate_block_cache: 0 2024-05-08T05:42:03.003 INFO:journalctl@ceph.mon.a.smithi112.stdout: initial_auto_readahead_size: 8192 2024-05-08T05:42:03.003 INFO:journalctl@ceph.mon.a.smithi112.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T05:42:03.003 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T05:42:03.004 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T05:42:03.004 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compression: NoCompression 2024-05-08T05:42:03.004 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T05:42:03.004 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T05:42:03.004 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T05:42:03.004 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.num_levels: 7 2024-05-08T05:42:03.004 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T05:42:03.004 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T05:42:03.004 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T05:42:03.004 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T05:42:03.004 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T05:42:03.004 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T05:42:03.004 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T05:42:03.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:42:03.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T05:42:03.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T05:42:03.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:42:03.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:42:03.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T05:42:03.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T05:42:03.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T05:42:03.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T05:42:03.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:42:03.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:42:03.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T05:42:03.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T05:42:03.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:42:03.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T05:42:03.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T05:42:03.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T05:42:03.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T05:42:03.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T05:42:03.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T05:42:03.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T05:42:03.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T05:42:03.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T05:42:03.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T05:42:03.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T05:42:03.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T05:42:03.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T05:42:03.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T05:42:03.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T05:42:03.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T05:42:03.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T05:42:03.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T05:42:03.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T05:42:03.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T05:42:03.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T05:42:03.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T05:42:03.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T05:42:03.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T05:42:03.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T05:42:03.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T05:42:03.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T05:42:03.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T05:42:03.008 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T05:42:03.008 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T05:42:03.008 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T05:42:03.008 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T05:42:03.008 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-08T05:42:03.008 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.inplace_update_support: 0 2024-05-08T05:42:03.008 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T05:42:03.008 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T05:42:03.008 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T05:42:03.008 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T05:42:03.008 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.bloom_locality: 0 2024-05-08T05:42:03.008 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.max_successive_merges: 0 2024-05-08T05:42:03.008 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T05:42:03.008 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T05:42:03.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T05:42:03.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T05:42:03.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.ttl: 2592000 2024-05-08T05:42:03.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T05:42:03.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T05:42:03.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T05:42:03.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.enable_blob_files: false 2024-05-08T05:42:03.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.min_blob_size: 0 2024-05-08T05:42:03.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T05:42:03.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T05:42:03.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T05:42:03.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T05:42:03.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T05:42:03.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T05:42:03.010 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T05:42:03.010 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T05:42:03.010 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-08T05:42:03.010 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-08T05:42:03.010 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a3db533c-8ddf-44ec-b58a-23062622246f 2024-05-08T05:42:03.010 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715146922788978, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-08T05:42:03.010 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-08T05:42:03.010 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715146922790977, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99753, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97642, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14020, "raw_average_key_size": 49, "raw_value_size": 89947, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715146922, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a3db533c-8ddf-44ec-b58a-23062622246f", "db_session_id": "S32M91S8MR1R6AUB7O4Z", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-08T05:42:03.010 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715146922791137, "job": 1, "event": "recovery_finished"} 2024-05-08T05:42:03.010 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-08T05:42:03.010 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T05:42:03.010 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fb09dc0000 2024-05-08T05:42:03.010 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: DB pointer 0x55fb09dac000 2024-05-08T05:42:03.011 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T05:42:03.011 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T05:42:03.011 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** DB Stats ** 2024-05-08T05:42:03.011 INFO:journalctl@ceph.mon.a.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:42:03.011 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T05:42:03.011 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:42:03.011 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:42:03.011 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T05:42:03.011 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:42:03.011 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:42:03.011 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-05-08T05:42:03.012 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** Compaction Stats [default] ** 2024-05-08T05:42:03.012 INFO:journalctl@ceph.mon.a.smithi112.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:42:03.012 INFO:journalctl@ceph.mon.a.smithi112.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T05:42:03.012 INFO:journalctl@ceph.mon.a.smithi112.stdout: L0 2/0 99.28 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:42:03.012 INFO:journalctl@ceph.mon.a.smithi112.stdout: Sum 2/0 99.28 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:42:03.012 INFO:journalctl@ceph.mon.a.smithi112.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:42:03.012 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-05-08T05:42:03.012 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** Compaction Stats [default] ** 2024-05-08T05:42:03.012 INFO:journalctl@ceph.mon.a.smithi112.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:42:03.012 INFO:journalctl@ceph.mon.a.smithi112.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T05:42:03.012 INFO:journalctl@ceph.mon.a.smithi112.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 77.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:42:03.012 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-05-08T05:42:03.012 INFO:journalctl@ceph.mon.a.smithi112.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T05:42:03.013 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-05-08T05:42:03.013 INFO:journalctl@ceph.mon.a.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:42:03.013 INFO:journalctl@ceph.mon.a.smithi112.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T05:42:03.013 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T05:42:03.013 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T05:42:03.013 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T05:42:03.013 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T05:42:03.013 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative compaction: 0.00 GB write, 13.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:42:03.013 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval compaction: 0.00 GB write, 13.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:42:03.013 INFO:journalctl@ceph.mon.a.smithi112.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T05:42:03.013 INFO:journalctl@ceph.mon.a.smithi112.stdout: Block cache BinnedLRUCache@0x55fb09ca31f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-05-08T05:42:03.013 INFO:journalctl@ceph.mon.a.smithi112.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-08T05:42:03.013 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-05-08T05:42:03.014 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T05:42:03.014 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: starting mon.a rank 0 at public addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] at bind addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:42:03.014 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: mon.a@-1(???) e1 preinit fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:42:03.014 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: mon.a@-1(???).mds e1 new map 2024-05-08T05:42:03.014 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: mon.a@-1(???).mds e1 print_map 2024-05-08T05:42:03.014 INFO:journalctl@ceph.mon.a.smithi112.stdout: e1 2024-05-08T05:42:03.014 INFO:journalctl@ceph.mon.a.smithi112.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T05:42:03.014 INFO:journalctl@ceph.mon.a.smithi112.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T05:42:03.014 INFO:journalctl@ceph.mon.a.smithi112.stdout: legacy client fscid: -1 2024-05-08T05:42:03.014 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-05-08T05:42:03.014 INFO:journalctl@ceph.mon.a.smithi112.stdout: No filesystems configured 2024-05-08T05:42:03.014 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T05:42:03.014 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:42:03.015 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:42:03.015 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:42:03.015 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-08T05:42:03.015 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 bash[29984]: 11e2e06de9b6c2d3298040188484e2e2f6ab0b727158e999cc81a7947a159be6 2024-05-08T05:42:03.015 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 systemd[1]: Started Ceph mon.a for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:42:03.015 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T05:42:03.015 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: monmap epoch 1 2024-05-08T05:42:03.015 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:42:03.015 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: last_changed 2024-05-08T05:41:43.845828+0000 2024-05-08T05:42:03.015 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: created 2024-05-08T05:41:43.845828+0000 2024-05-08T05:42:03.015 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: min_mon_release 19 (squid) 2024-05-08T05:42:03.015 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: election_strategy: 1 2024-05-08T05:42:03.015 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-08T05:42:03.016 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: fsmap 2024-05-08T05:42:03.016 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T05:42:03.016 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:02 smithi112 ceph-mon[30003]: mgrmap e1: no daemons active 2024-05-08T05:42:04.277 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.537+0000 7fec6fe00700 1 Processor -- start 2024-05-08T05:42:04.278 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.538+0000 7fec6fe00700 1 -- start start 2024-05-08T05:42:04.278 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.538+0000 7fec6fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec70108c70 0x7fec70109090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:04.278 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.538+0000 7fec6fe00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec70109660 con 0x7fec70108c70 2024-05-08T05:42:04.278 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.539+0000 7fec6ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec70108c70 0x7fec70109090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:04.278 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.539+0000 7fec6ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec70108c70 0x7fec70109090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56478/0 (socket says 172.21.15.112:56478) 2024-05-08T05:42:04.278 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.539+0000 7fec6ea00700 1 -- 172.21.15.112:0/2191678088 learned_addr learned my addr 172.21.15.112:0/2191678088 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:04.278 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.539+0000 7fec6ea00700 1 -- 172.21.15.112:0/2191678088 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec70109e70 con 0x7fec70108c70 2024-05-08T05:42:04.278 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.540+0000 7fec6ea00700 1 --2- 172.21.15.112:0/2191678088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec70108c70 0x7fec70109090 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fec600098a0 tx=0x7fec60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31153b228d66751e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:04.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.540+0000 7fec6d600700 1 -- 172.21.15.112:0/2191678088 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec60004160 con 0x7fec70108c70 2024-05-08T05:42:04.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.540+0000 7fec6d600700 1 -- 172.21.15.112:0/2191678088 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fec60004300 con 0x7fec70108c70 2024-05-08T05:42:04.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.540+0000 7fec6d600700 1 -- 172.21.15.112:0/2191678088 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec60011af0 con 0x7fec70108c70 2024-05-08T05:42:04.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.541+0000 7fec6fe00700 1 -- 172.21.15.112:0/2191678088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec70108c70 msgr2=0x7fec70109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:04.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.541+0000 7fec6fe00700 1 --2- 172.21.15.112:0/2191678088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec70108c70 0x7fec70109090 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fec600098a0 tx=0x7fec60009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.541+0000 7fec6fe00700 1 -- 172.21.15.112:0/2191678088 shutdown_connections 2024-05-08T05:42:04.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.541+0000 7fec6fe00700 1 --2- 172.21.15.112:0/2191678088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec70108c70 0x7fec70109090 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.541+0000 7fec6fe00700 1 -- 172.21.15.112:0/2191678088 >> 172.21.15.112:0/2191678088 conn(0x7fec70104210 msgr2=0x7fec70106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:04.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.541+0000 7fec6fe00700 1 -- 172.21.15.112:0/2191678088 shutdown_connections 2024-05-08T05:42:04.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.541+0000 7fec6fe00700 1 -- 172.21.15.112:0/2191678088 wait complete. 2024-05-08T05:42:04.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.542+0000 7fec6fe00700 1 Processor -- start 2024-05-08T05:42:04.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.542+0000 7fec6fe00700 1 -- start start 2024-05-08T05:42:04.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.543+0000 7fec6fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec70108c70 0x7fec7019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:04.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.543+0000 7fec6fe00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec7010d220 con 0x7fec70108c70 2024-05-08T05:42:04.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.543+0000 7fec6ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec70108c70 0x7fec7019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:04.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.543+0000 7fec6ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec70108c70 0x7fec7019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56480/0 (socket says 172.21.15.112:56480) 2024-05-08T05:42:04.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.543+0000 7fec6ea00700 1 -- 172.21.15.112:0/1067201791 learned_addr learned my addr 172.21.15.112:0/1067201791 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:04.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.544+0000 7fec6ea00700 1 -- 172.21.15.112:0/1067201791 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7019f440 con 0x7fec70108c70 2024-05-08T05:42:04.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.544+0000 7fec6ea00700 1 --2- 172.21.15.112:0/1067201791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec70108c70 0x7fec7019ef00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fec600117b0 tx=0x7fec60011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:04.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.544+0000 7fec67400700 1 -- 172.21.15.112:0/1067201791 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec60003af0 con 0x7fec70108c70 2024-05-08T05:42:04.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.544+0000 7fec67400700 1 -- 172.21.15.112:0/1067201791 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fec60020070 con 0x7fec70108c70 2024-05-08T05:42:04.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.544+0000 7fec6fe00700 1 -- 172.21.15.112:0/1067201791 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7019f6d0 con 0x7fec70108c70 2024-05-08T05:42:04.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.544+0000 7fec67400700 1 -- 172.21.15.112:0/1067201791 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec60019630 con 0x7fec70108c70 2024-05-08T05:42:04.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.544+0000 7fec6fe00700 1 -- 172.21.15.112:0/1067201791 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec701a23c0 con 0x7fec70108c70 2024-05-08T05:42:04.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.545+0000 7fec67400700 1 -- 172.21.15.112:0/1067201791 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fec60019c30 con 0x7fec70108c70 2024-05-08T05:42:04.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.545+0000 7fec67400700 1 -- 172.21.15.112:0/1067201791 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fec6000b720 con 0x7fec70108c70 2024-05-08T05:42:04.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.546+0000 7fec6fe00700 1 -- 172.21.15.112:0/1067201791 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec70064b00 con 0x7fec70108c70 2024-05-08T05:42:04.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.549+0000 7fec67400700 1 -- 172.21.15.112:0/1067201791 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fec600042a0 con 0x7fec70108c70 2024-05-08T05:42:04.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.626+0000 7fec6fe00700 1 -- 172.21.15.112:0/1067201791 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fec701a2590 con 0x7fec70108c70 2024-05-08T05:42:04.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.629+0000 7fec67400700 1 -- 172.21.15.112:0/1067201791 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fec6001a9c0 con 0x7fec70108c70 2024-05-08T05:42:04.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.630+0000 7fec6fe00700 1 -- 172.21.15.112:0/1067201791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec70108c70 msgr2=0x7fec7019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:04.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.630+0000 7fec6fe00700 1 --2- 172.21.15.112:0/1067201791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec70108c70 0x7fec7019ef00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fec600117b0 tx=0x7fec60011b70 comp rx=0 tx=0).stop 2024-05-08T05:42:04.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.631+0000 7fec6fe00700 1 -- 172.21.15.112:0/1067201791 shutdown_connections 2024-05-08T05:42:04.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.631+0000 7fec6fe00700 1 --2- 172.21.15.112:0/1067201791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec70108c70 0x7fec7019ef00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.631+0000 7fec6fe00700 1 -- 172.21.15.112:0/1067201791 >> 172.21.15.112:0/1067201791 conn(0x7fec70104210 msgr2=0x7fec70104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:04.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.631+0000 7fec6fe00700 1 -- 172.21.15.112:0/1067201791 shutdown_connections 2024-05-08T05:42:04.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.631+0000 7fec6fe00700 1 -- 172.21.15.112:0/1067201791 wait complete. 2024-05-08T05:42:04.283 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-08T05:42:04.293 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:42:04.293 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mgr... 2024-05-08T05:42:04.294 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-08T05:42:04.295 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-08T05:42:04.731 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.x 2024-05-08T05:42:04.732 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-08T05:42:05.001 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968.target.wants/ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@.service. 2024-05-08T05:42:05.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:04 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1067201791' entity='client.admin' 2024-05-08T05:42:05.316 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:05 smithi112 systemd[1]: Starting Ceph mgr.x for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:42:05.676 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:05 smithi112 podman[30228]: 2024-05-08 05:42:05.292612511 +0000 UTC m=+0.031862398 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:42:05.676 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:05 smithi112 podman[30228]: 2024-05-08 05:42:05.4282655 +0000 UTC m=+0.167515422 container create 935065eb06dfb6b3ad266456e9a5905237da1f4776b221602b5f5a63360d3502 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:42:05.676 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:05 smithi112 podman[30228]: 2024-05-08 05:42:05.67514465 +0000 UTC m=+0.414394579 container init 935065eb06dfb6b3ad266456e9a5905237da1f4776b221602b5f5a63360d3502 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-08T05:42:05.790 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:42:05.790 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-05-08T05:42:05.807 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:42:05.807 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-05-08T05:42:05.808 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-05-08T05:42:05.808 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T05:42:05.827 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:42:05.827 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-05-08T05:42:05.844 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:42:05.844 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-05-08T05:42:05.844 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-05-08T05:42:05.844 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-08T05:42:05.845 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr to start... 2024-05-08T05:42:05.845 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr... 2024-05-08T05:42:05.941 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:05 smithi112 podman[30228]: 2024-05-08 05:42:05.685290944 +0000 UTC m=+0.424540873 container start 935065eb06dfb6b3ad266456e9a5905237da1f4776b221602b5f5a63360d3502 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:42:05.941 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:05 smithi112 bash[30228]: 935065eb06dfb6b3ad266456e9a5905237da1f4776b221602b5f5a63360d3502 2024-05-08T05:42:05.941 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:05 smithi112 systemd[1]: Started Ceph mgr.x for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:42:06.195 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:05 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:05.940+0000 7f42150e4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:42:06.196 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:06 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:06.193+0000 7f42150e4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:42:06.507 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:06 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:06.328+0000 7f42150e4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:42:06.777 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:06 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:06.739+0000 7f42150e4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:42:07.027 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:06 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:06.857+0000 7f42150e4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:42:07.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:07 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/544311868' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T05:42:07.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-08T05:42:07.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:42:07.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "7a07ff60-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:42:07.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T05:42:07.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T05:42:07.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T05:42:07.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T05:42:07.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:07.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T05:42:07.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T05:42:07.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T05:42:07.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T05:42:07.737 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T05:42:07.737 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T05:42:07.737 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:07.737 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T05:42:07.737 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T05:42:07.738 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T05:42:07.738 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T05:42:07.738 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T05:42:07.738 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T05:42:07.738 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T05:42:07.738 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T05:42:07.738 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T05:42:07.738 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:07.738 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T05:42:07.738 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:07.738 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T05:42:07.738 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T05:42:07.738 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:07.738 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T05:42:07.739 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T05:42:07.739 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T05:42:07.739 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T05:42:07.739 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T05:42:07.739 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T05:42:07.739 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T05:42:07.739 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:42:07.739 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:42:07.739 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:07.739 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T05:42:07.739 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:07.740 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T05:41:48.499480+0000", 2024-05-08T05:42:07.740 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:42:07.740 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:07.740 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T05:42:07.740 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:42:07.740 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.931+0000 7f1417000700 1 Processor -- start 2024-05-08T05:42:07.740 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.931+0000 7f1417000700 1 -- start start 2024-05-08T05:42:07.740 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.931+0000 7f1417000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141007ee00 0x7f141007d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.740 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.932+0000 7f1417000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f141007d830 con 0x7f141007ee00 2024-05-08T05:42:07.740 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.932+0000 7f1415c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141007ee00 0x7f141007d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.740 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.932+0000 7f1415c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141007ee00 0x7f141007d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48080/0 (socket says 172.21.15.112:48080) 2024-05-08T05:42:07.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.932+0000 7f1415c00700 1 -- 172.21.15.112:0/2471385780 learned_addr learned my addr 172.21.15.112:0/2471385780 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:07.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.932+0000 7f1415c00700 1 -- 172.21.15.112:0/2471385780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141007d9b0 con 0x7f141007ee00 2024-05-08T05:42:07.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.933+0000 7f1415c00700 1 --2- 172.21.15.112:0/2471385780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141007ee00 0x7f141007d260 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f14040098a0 tx=0x7f1404009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90830612d1fdda45 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:07.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.933+0000 7f140fe00700 1 -- 172.21.15.112:0/2471385780 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1404004160 con 0x7f141007ee00 2024-05-08T05:42:07.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.933+0000 7f140fe00700 1 -- 172.21.15.112:0/2471385780 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1404004300 con 0x7f141007ee00 2024-05-08T05:42:07.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.933+0000 7f140fe00700 1 -- 172.21.15.112:0/2471385780 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1404011af0 con 0x7f141007ee00 2024-05-08T05:42:07.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.934+0000 7f1417000700 1 -- 172.21.15.112:0/2471385780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141007ee00 msgr2=0x7f141007d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.934+0000 7f1417000700 1 --2- 172.21.15.112:0/2471385780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141007ee00 0x7f141007d260 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f14040098a0 tx=0x7f1404009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.934+0000 7f1417000700 1 -- 172.21.15.112:0/2471385780 shutdown_connections 2024-05-08T05:42:07.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.934+0000 7f1417000700 1 --2- 172.21.15.112:0/2471385780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141007ee00 0x7f141007d260 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.741 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.934+0000 7f1417000700 1 -- 172.21.15.112:0/2471385780 >> 172.21.15.112:0/2471385780 conn(0x7f1410104210 msgr2=0x7f1410106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:07.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.934+0000 7f1417000700 1 -- 172.21.15.112:0/2471385780 shutdown_connections 2024-05-08T05:42:07.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.934+0000 7f1417000700 1 -- 172.21.15.112:0/2471385780 wait complete. 2024-05-08T05:42:07.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.935+0000 7f1417000700 1 Processor -- start 2024-05-08T05:42:07.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.936+0000 7f1417000700 1 -- start start 2024-05-08T05:42:07.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.936+0000 7f1417000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141007ee00 0x7f141019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.936+0000 7f1417000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f141010f240 con 0x7f141007ee00 2024-05-08T05:42:07.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.936+0000 7f1415c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141007ee00 0x7f141019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.936+0000 7f1415c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141007ee00 0x7f141019ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48094/0 (socket says 172.21.15.112:48094) 2024-05-08T05:42:07.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.936+0000 7f1415c00700 1 -- 172.21.15.112:0/544311868 learned_addr learned my addr 172.21.15.112:0/544311868 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:07.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.937+0000 7f1415c00700 1 -- 172.21.15.112:0/544311868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141019f390 con 0x7f141007ee00 2024-05-08T05:42:07.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.937+0000 7f1415c00700 1 --2- 172.21.15.112:0/544311868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141007ee00 0x7f141019ee50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1404011ac0 tx=0x7f1404011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:07.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.937+0000 7f140e000700 1 -- 172.21.15.112:0/544311868 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1404003970 con 0x7f141007ee00 2024-05-08T05:42:07.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.937+0000 7f140e000700 1 -- 172.21.15.112:0/544311868 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1404003b10 con 0x7f141007ee00 2024-05-08T05:42:07.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.937+0000 7f140e000700 1 -- 172.21.15.112:0/544311868 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1404024610 con 0x7f141007ee00 2024-05-08T05:42:07.743 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.937+0000 7f1417000700 1 -- 172.21.15.112:0/544311868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141019f620 con 0x7f141007ee00 2024-05-08T05:42:07.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.937+0000 7f1417000700 1 -- 172.21.15.112:0/544311868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14101a2310 con 0x7f141007ee00 2024-05-08T05:42:07.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.938+0000 7f140e000700 1 -- 172.21.15.112:0/544311868 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1404024bd0 con 0x7f141007ee00 2024-05-08T05:42:07.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.938+0000 7f140e000700 1 -- 172.21.15.112:0/544311868 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1404019700 con 0x7f141007ee00 2024-05-08T05:42:07.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.938+0000 7f1417000700 1 -- 172.21.15.112:0/544311868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1410064b00 con 0x7f141007ee00 2024-05-08T05:42:07.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.941+0000 7f140e000700 1 -- 172.21.15.112:0/544311868 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1404043ec0 con 0x7f141007ee00 2024-05-08T05:42:07.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.021+0000 7f1417000700 1 -- 172.21.15.112:0/544311868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f14101a24e0 con 0x7f141007ee00 2024-05-08T05:42:07.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.022+0000 7f140e000700 1 -- 172.21.15.112:0/544311868 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1404032030 con 0x7f141007ee00 2024-05-08T05:42:07.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.023+0000 7f1417000700 1 -- 172.21.15.112:0/544311868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141007ee00 msgr2=0x7f141019ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.023+0000 7f1417000700 1 --2- 172.21.15.112:0/544311868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141007ee00 0x7f141019ee50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1404011ac0 tx=0x7f1404011ba0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.023+0000 7f1417000700 1 -- 172.21.15.112:0/544311868 shutdown_connections 2024-05-08T05:42:07.745 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.023+0000 7f1417000700 1 --2- 172.21.15.112:0/544311868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141007ee00 0x7f141019ee50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.745 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.023+0000 7f1417000700 1 -- 172.21.15.112:0/544311868 >> 172.21.15.112:0/544311868 conn(0x7f1410104210 msgr2=0x7f1410104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:07.745 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.023+0000 7f1417000700 1 -- 172.21.15.112:0/544311868 shutdown_connections 2024-05-08T05:42:07.745 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.023+0000 7f1417000700 1 -- 172.21.15.112:0/544311868 wait complete. 2024-05-08T05:42:07.745 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (1/15)... 2024-05-08T05:42:08.816 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:08 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:08.461+0000 7f42150e4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:42:08.816 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:08 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:08.688+0000 7f42150e4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:42:09.284 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:08 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:08.905+0000 7f42150e4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:42:09.285 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:09 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:09.283+0000 7f42150e4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:42:09.565 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:09 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:09.405+0000 7f42150e4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:42:09.929 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:09 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:09.671+0000 7f42150e4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:42:09.930 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:09 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:09.928+0000 7f42150e4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:42:10.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:10 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2918953810' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T05:42:11.002 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:10 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:10.747+0000 7f42150e4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:42:11.012 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-08T05:42:11.013 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:42:11.013 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "7a07ff60-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:42:11.013 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T05:42:11.013 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T05:42:11.014 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T05:42:11.014 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T05:42:11.014 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:11.014 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T05:42:11.014 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T05:42:11.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-05-08T05:42:11.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:42:11.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T05:42:11.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T05:42:11.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:42:11.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-08T05:42:11.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T05:42:11.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:11.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-08T05:42:11.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T05:42:11.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:11.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T05:42:11.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:11.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T05:42:11.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T05:42:11.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T05:42:11.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T05:42:11.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T05:42:11.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T05:42:11.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:11.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T05:42:11.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T05:42:11.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T05:42:11.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T05:42:11.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T05:42:11.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T05:42:11.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T05:42:11.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T05:42:11.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T05:42:11.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:11.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T05:42:11.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:11.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T05:42:11.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T05:42:11.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:11.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T05:42:11.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T05:42:11.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T05:42:11.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T05:42:11.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T05:42:11.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T05:42:11.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T05:42:11.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:42:11.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:42:11.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:11.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T05:42:11.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:11.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T05:41:48.499480+0000", 2024-05-08T05:42:11.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:42:11.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:11.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T05:42:11.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:42:11.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.282+0000 7f8e23600700 1 Processor -- start 2024-05-08T05:42:11.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.283+0000 7f8e23600700 1 -- start start 2024-05-08T05:42:11.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.283+0000 7f8e23600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e1c108e30 0x7f8e1c109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:11.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.283+0000 7f8e23600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c109790 con 0x7f8e1c108e30 2024-05-08T05:42:11.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.283+0000 7f8e20c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e1c108e30 0x7f8e1c109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:11.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.283+0000 7f8e20c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e1c108e30 0x7f8e1c109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48102/0 (socket says 172.21.15.112:48102) 2024-05-08T05:42:11.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.284+0000 7f8e20c00700 1 -- 172.21.15.112:0/1765398682 learned_addr learned my addr 172.21.15.112:0/1765398682 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:11.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.284+0000 7f8e20c00700 1 -- 172.21.15.112:0/1765398682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1c109f90 con 0x7f8e1c108e30 2024-05-08T05:42:11.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.284+0000 7f8e20c00700 1 --2- 172.21.15.112:0/1765398682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e1c108e30 0x7f8e1c109250 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8e100098a0 tx=0x7f8e10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240d6be8ca19d2fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:11.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.285+0000 7f8e1b400700 1 -- 172.21.15.112:0/1765398682 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e10004160 con 0x7f8e1c108e30 2024-05-08T05:42:11.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.285+0000 7f8e1b400700 1 -- 172.21.15.112:0/1765398682 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8e10004300 con 0x7f8e1c108e30 2024-05-08T05:42:11.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.285+0000 7f8e1b400700 1 -- 172.21.15.112:0/1765398682 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e10011af0 con 0x7f8e1c108e30 2024-05-08T05:42:11.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.285+0000 7f8e23600700 1 -- 172.21.15.112:0/1765398682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e1c108e30 msgr2=0x7f8e1c109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:11.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.285+0000 7f8e23600700 1 --2- 172.21.15.112:0/1765398682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e1c108e30 0x7f8e1c109250 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8e100098a0 tx=0x7f8e10009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.285+0000 7f8e23600700 1 -- 172.21.15.112:0/1765398682 shutdown_connections 2024-05-08T05:42:11.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.285+0000 7f8e23600700 1 --2- 172.21.15.112:0/1765398682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e1c108e30 0x7f8e1c109250 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.285+0000 7f8e23600700 1 -- 172.21.15.112:0/1765398682 >> 172.21.15.112:0/1765398682 conn(0x7f8e1c1043f0 msgr2=0x7f8e1c106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:11.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.286+0000 7f8e23600700 1 -- 172.21.15.112:0/1765398682 shutdown_connections 2024-05-08T05:42:11.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.286+0000 7f8e23600700 1 -- 172.21.15.112:0/1765398682 wait complete. 2024-05-08T05:42:11.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.287+0000 7f8e23600700 1 Processor -- start 2024-05-08T05:42:11.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.287+0000 7f8e23600700 1 -- start start 2024-05-08T05:42:11.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.287+0000 7f8e23600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e1c108e30 0x7f8e1c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:11.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.287+0000 7f8e23600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c10d3d0 con 0x7f8e1c108e30 2024-05-08T05:42:11.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.287+0000 7f8e20c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e1c108e30 0x7f8e1c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:11.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.287+0000 7f8e20c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e1c108e30 0x7f8e1c19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48114/0 (socket says 172.21.15.112:48114) 2024-05-08T05:42:11.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.287+0000 7f8e20c00700 1 -- 172.21.15.112:0/2918953810 learned_addr learned my addr 172.21.15.112:0/2918953810 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:11.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.288+0000 7f8e20c00700 1 -- 172.21.15.112:0/2918953810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1c19f510 con 0x7f8e1c108e30 2024-05-08T05:42:11.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.288+0000 7f8e20c00700 1 --2- 172.21.15.112:0/2918953810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e1c108e30 0x7f8e1c19efd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e10011ac0 tx=0x7f8e10011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:11.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.288+0000 7f8e19600700 1 -- 172.21.15.112:0/2918953810 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e10003b20 con 0x7f8e1c108e30 2024-05-08T05:42:11.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.288+0000 7f8e19600700 1 -- 172.21.15.112:0/2918953810 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8e1001a8c0 con 0x7f8e1c108e30 2024-05-08T05:42:11.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.288+0000 7f8e19600700 1 -- 172.21.15.112:0/2918953810 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e100248d0 con 0x7f8e1c108e30 2024-05-08T05:42:11.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.288+0000 7f8e23600700 1 -- 172.21.15.112:0/2918953810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1c19f7a0 con 0x7f8e1c108e30 2024-05-08T05:42:11.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.288+0000 7f8e23600700 1 -- 172.21.15.112:0/2918953810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e1c1a2490 con 0x7f8e1c108e30 2024-05-08T05:42:11.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.289+0000 7f8e19600700 1 -- 172.21.15.112:0/2918953810 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8e10022020 con 0x7f8e1c108e30 2024-05-08T05:42:11.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.289+0000 7f8e19600700 1 -- 172.21.15.112:0/2918953810 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8e1002dde0 con 0x7f8e1c108e30 2024-05-08T05:42:11.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.290+0000 7f8e23600700 1 -- 172.21.15.112:0/2918953810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e1c064b00 con 0x7f8e1c108e30 2024-05-08T05:42:11.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.292+0000 7f8e19600700 1 -- 172.21.15.112:0/2918953810 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8e10042ec0 con 0x7f8e1c108e30 2024-05-08T05:42:11.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.370+0000 7f8e23600700 1 -- 172.21.15.112:0/2918953810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8e1c1a29d0 con 0x7f8e1c108e30 2024-05-08T05:42:11.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.371+0000 7f8e19600700 1 -- 172.21.15.112:0/2918953810 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8e10031020 con 0x7f8e1c108e30 2024-05-08T05:42:11.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.372+0000 7f8e23600700 1 -- 172.21.15.112:0/2918953810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e1c108e30 msgr2=0x7f8e1c19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:11.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.372+0000 7f8e23600700 1 --2- 172.21.15.112:0/2918953810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e1c108e30 0x7f8e1c19efd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e10011ac0 tx=0x7f8e10011ba0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.372+0000 7f8e23600700 1 -- 172.21.15.112:0/2918953810 shutdown_connections 2024-05-08T05:42:11.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.372+0000 7f8e23600700 1 --2- 172.21.15.112:0/2918953810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e1c108e30 0x7f8e1c19efd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.372+0000 7f8e23600700 1 -- 172.21.15.112:0/2918953810 >> 172.21.15.112:0/2918953810 conn(0x7f8e1c1043f0 msgr2=0x7f8e1c104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:11.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.373+0000 7f8e23600700 1 -- 172.21.15.112:0/2918953810 shutdown_connections 2024-05-08T05:42:11.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.373+0000 7f8e23600700 1 -- 172.21.15.112:0/2918953810 wait complete. 2024-05-08T05:42:11.031 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (2/15)... 2024-05-08T05:42:11.315 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:11 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:11.044+0000 7f42150e4200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:42:12.066 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:11 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:11.618+0000 7f42150e4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:42:12.066 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:11 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:11.742+0000 7f42150e4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:42:12.565 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:12 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:12.194+0000 7f42150e4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:42:13.065 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:12 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:12.636+0000 7f42150e4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:42:13.065 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:12 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:12.758+0000 7f42150e4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:42:13.391 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:13 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:13.078+0000 7f42150e4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:42:13.391 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:13 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:13.262+0000 7f42150e4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:42:13.679 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:13 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:13.392+0000 7f42150e4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:42:13.930 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:13 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:13.803+0000 7f42150e4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:42:13.931 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:13 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2097404950' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T05:42:14.200 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:13 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:13.929+0000 7f42150e4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:42:14.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-08T05:42:14.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:42:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "7a07ff60-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:42:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T05:42:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T05:42:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T05:42:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T05:42:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T05:42:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T05:42:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-05-08T05:42:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:42:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T05:42:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T05:42:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:42:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-05-08T05:42:14.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T05:42:14.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:14.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-08T05:42:14.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T05:42:14.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:14.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T05:42:14.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:14.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T05:42:14.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T05:42:14.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T05:42:14.504 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T05:42:14.505 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T05:42:14.505 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T05:42:14.505 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:14.505 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T05:42:14.505 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T05:42:14.505 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T05:42:14.505 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T05:42:14.505 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T05:42:14.506 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T05:42:14.506 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T05:42:14.506 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T05:42:14.506 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T05:42:14.506 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:14.506 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T05:42:14.506 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:14.506 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T05:42:14.506 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T05:42:14.506 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:14.507 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T05:42:14.507 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T05:42:14.507 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T05:42:14.507 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T05:42:14.507 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T05:42:14.507 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T05:42:14.507 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T05:42:14.507 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:42:14.507 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:42:14.507 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:14.507 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T05:42:14.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:14.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T05:41:48.499480+0000", 2024-05-08T05:42:14.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:42:14.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:14.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T05:42:14.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:42:14.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.695+0000 7fa717e00700 1 Processor -- start 2024-05-08T05:42:14.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.696+0000 7fa717e00700 1 -- start start 2024-05-08T05:42:14.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.696+0000 7fa717e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa71807ee60 0x7fa71807d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:14.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.696+0000 7fa717e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa71807d890 con 0x7fa71807ee60 2024-05-08T05:42:14.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.697+0000 7fa716a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa71807ee60 0x7fa71807d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:14.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.697+0000 7fa716a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa71807ee60 0x7fa71807d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48130/0 (socket says 172.21.15.112:48130) 2024-05-08T05:42:14.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.697+0000 7fa716a00700 1 -- 172.21.15.112:0/38568878 learned_addr learned my addr 172.21.15.112:0/38568878 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:14.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.697+0000 7fa716a00700 1 -- 172.21.15.112:0/38568878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71807da10 con 0x7fa71807ee60 2024-05-08T05:42:14.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.697+0000 7fa716a00700 1 --2- 172.21.15.112:0/38568878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa71807ee60 0x7fa71807d2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa7080098a0 tx=0x7fa708009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0524202efa27e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:14.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.698+0000 7fa715600700 1 -- 172.21.15.112:0/38568878 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa708004160 con 0x7fa71807ee60 2024-05-08T05:42:14.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.698+0000 7fa715600700 1 -- 172.21.15.112:0/38568878 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa708004300 con 0x7fa71807ee60 2024-05-08T05:42:14.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.698+0000 7fa715600700 1 -- 172.21.15.112:0/38568878 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa708011af0 con 0x7fa71807ee60 2024-05-08T05:42:14.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.698+0000 7fa717e00700 1 -- 172.21.15.112:0/38568878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa71807ee60 msgr2=0x7fa71807d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:14.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.698+0000 7fa717e00700 1 --2- 172.21.15.112:0/38568878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa71807ee60 0x7fa71807d2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa7080098a0 tx=0x7fa708009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.698+0000 7fa717e00700 1 -- 172.21.15.112:0/38568878 shutdown_connections 2024-05-08T05:42:14.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.698+0000 7fa717e00700 1 --2- 172.21.15.112:0/38568878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa71807ee60 0x7fa71807d2c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.698+0000 7fa717e00700 1 -- 172.21.15.112:0/38568878 >> 172.21.15.112:0/38568878 conn(0x7fa718104210 msgr2=0x7fa718106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:14.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.699+0000 7fa717e00700 1 -- 172.21.15.112:0/38568878 shutdown_connections 2024-05-08T05:42:14.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.699+0000 7fa717e00700 1 -- 172.21.15.112:0/38568878 wait complete. 2024-05-08T05:42:14.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.700+0000 7fa717e00700 1 Processor -- start 2024-05-08T05:42:14.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.700+0000 7fa717e00700 1 -- start start 2024-05-08T05:42:14.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.700+0000 7fa717e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa71807ee60 0x7fa71819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:14.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.700+0000 7fa717e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa71810f230 con 0x7fa71807ee60 2024-05-08T05:42:14.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.700+0000 7fa716a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa71807ee60 0x7fa71819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:14.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.700+0000 7fa716a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa71807ee60 0x7fa71819ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48132/0 (socket says 172.21.15.112:48132) 2024-05-08T05:42:14.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.700+0000 7fa716a00700 1 -- 172.21.15.112:0/2097404950 learned_addr learned my addr 172.21.15.112:0/2097404950 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.701+0000 7fa716a00700 1 -- 172.21.15.112:0/2097404950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71819f3b0 con 0x7fa71807ee60 2024-05-08T05:42:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.701+0000 7fa716a00700 1 --2- 172.21.15.112:0/2097404950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa71807ee60 0x7fa71819ee70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa708009fb0 tx=0x7fa708011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.701+0000 7fa70f400700 1 -- 172.21.15.112:0/2097404950 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7080036a0 con 0x7fa71807ee60 2024-05-08T05:42:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.701+0000 7fa70f400700 1 -- 172.21.15.112:0/2097404950 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa70801a430 con 0x7fa71807ee60 2024-05-08T05:42:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.701+0000 7fa717e00700 1 -- 172.21.15.112:0/2097404950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71819f640 con 0x7fa71807ee60 2024-05-08T05:42:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.701+0000 7fa70f400700 1 -- 172.21.15.112:0/2097404950 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa708019550 con 0x7fa71807ee60 2024-05-08T05:42:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.702+0000 7fa717e00700 1 -- 172.21.15.112:0/2097404950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7181a2330 con 0x7fa71807ee60 2024-05-08T05:42:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.702+0000 7fa70f400700 1 -- 172.21.15.112:0/2097404950 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa708019b90 con 0x7fa71807ee60 2024-05-08T05:42:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.702+0000 7fa70f400700 1 -- 172.21.15.112:0/2097404950 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa708024850 con 0x7fa71807ee60 2024-05-08T05:42:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.703+0000 7fa717e00700 1 -- 172.21.15.112:0/2097404950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa718064b00 con 0x7fa71807ee60 2024-05-08T05:42:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.707+0000 7fa70f400700 1 -- 172.21.15.112:0/2097404950 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa708043ec0 con 0x7fa71807ee60 2024-05-08T05:42:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.786+0000 7fa717e00700 1 -- 172.21.15.112:0/2097404950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa7181a2870 con 0x7fa71807ee60 2024-05-08T05:42:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.787+0000 7fa70f400700 1 -- 172.21.15.112:0/2097404950 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fa708032020 con 0x7fa71807ee60 2024-05-08T05:42:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.788+0000 7fa717e00700 1 -- 172.21.15.112:0/2097404950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa71807ee60 msgr2=0x7fa71819ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.788+0000 7fa717e00700 1 --2- 172.21.15.112:0/2097404950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa71807ee60 0x7fa71819ee70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa708009fb0 tx=0x7fa708011940 comp rx=0 tx=0).stop 2024-05-08T05:42:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.788+0000 7fa717e00700 1 -- 172.21.15.112:0/2097404950 shutdown_connections 2024-05-08T05:42:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.788+0000 7fa717e00700 1 --2- 172.21.15.112:0/2097404950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa71807ee60 0x7fa71819ee70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.788+0000 7fa717e00700 1 -- 172.21.15.112:0/2097404950 >> 172.21.15.112:0/2097404950 conn(0x7fa718104210 msgr2=0x7fa718104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.789+0000 7fa717e00700 1 -- 172.21.15.112:0/2097404950 shutdown_connections 2024-05-08T05:42:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.789+0000 7fa717e00700 1 -- 172.21.15.112:0/2097404950 wait complete. 2024-05-08T05:42:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (3/15)... 2024-05-08T05:42:14.816 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:14 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:14.710+0000 7f42150e4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:42:15.316 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:14 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:14.998+0000 7f42150e4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:42:15.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:15 smithi112 ceph-mon[30003]: Activating manager daemon x 2024-05-08T05:42:15.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:15 smithi112 ceph-mon[30003]: mgrmap e2: x(active, starting, since 0.0118294s) 2024-05-08T05:42:15.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:15 smithi112 ceph-mon[30003]: from='mgr.14100 172.21.15.112:0/1710698733' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:42:15.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:15 smithi112 ceph-mon[30003]: from='mgr.14100 172.21.15.112:0/1710698733' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:42:15.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:15 smithi112 ceph-mon[30003]: from='mgr.14100 172.21.15.112:0/1710698733' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:42:15.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:15 smithi112 ceph-mon[30003]: from='mgr.14100 172.21.15.112:0/1710698733' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:42:15.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:15 smithi112 ceph-mon[30003]: from='mgr.14100 172.21.15.112:0/1710698733' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:42:15.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:15 smithi112 ceph-mon[30003]: Manager daemon x is now available 2024-05-08T05:42:15.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:15 smithi112 ceph-mon[30003]: from='mgr.14100 172.21.15.112:0/1710698733' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:42:15.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:15 smithi112 ceph-mon[30003]: from='mgr.14100 172.21.15.112:0/1710698733' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:42:15.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:15 smithi112 ceph-mon[30003]: from='mgr.14100 172.21.15.112:0/1710698733' entity='mgr.x' 2024-05-08T05:42:15.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:15 smithi112 ceph-mon[30003]: from='mgr.14100 172.21.15.112:0/1710698733' entity='mgr.x' 2024-05-08T05:42:15.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:15 smithi112 ceph-mon[30003]: from='mgr.14100 172.21.15.112:0/1710698733' entity='mgr.x' 2024-05-08T05:42:17.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:17 smithi112 ceph-mon[30003]: mgrmap e3: x(active, since 1.02108s) 2024-05-08T05:42:18.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:18 smithi112 ceph-mon[30003]: mgrmap e4: x(active, since 2s) 2024-05-08T05:42:18.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:18 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2834756296' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T05:42:18.126 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-08T05:42:18.126 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:42:18.126 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "7a07ff60-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:42:18.126 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T05:42:18.126 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T05:42:18.126 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T05:42:18.126 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T05:42:18.126 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:18.126 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T05:42:18.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T05:42:18.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-05-08T05:42:18.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:42:18.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T05:42:18.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T05:42:18.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:42:18.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-08T05:42:18.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T05:42:18.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:18.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-08T05:42:18.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T05:42:18.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:18.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T05:42:18.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:18.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T05:42:18.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T05:42:18.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T05:42:18.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T05:42:18.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T05:42:18.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T05:42:18.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:18.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T05:42:18.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T05:42:18.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T05:42:18.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T05:42:18.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T05:42:18.138 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T05:42:18.138 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T05:42:18.138 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T05:42:18.138 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T05:42:18.138 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:18.138 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T05:42:18.138 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:18.138 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T05:42:18.138 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T05:42:18.138 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:18.138 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T05:42:18.138 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T05:42:18.138 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T05:42:18.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T05:42:18.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T05:42:18.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T05:42:18.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T05:42:18.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:42:18.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:42:18.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:18.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T05:42:18.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:42:18.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T05:41:48.499480+0000", 2024-05-08T05:42:18.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:42:18.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:42:18.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T05:42:18.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:42:18.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.074+0000 7f468cc00700 1 Processor -- start 2024-05-08T05:42:18.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.075+0000 7f468cc00700 1 -- start start 2024-05-08T05:42:18.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.075+0000 7f468cc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f468810ae80 0x7f468810b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:18.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.075+0000 7f468cc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f468810b7e0 con 0x7f468810ae80 2024-05-08T05:42:18.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.075+0000 7f4687400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f468810ae80 0x7f468810b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:18.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.075+0000 7f4687400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f468810ae80 0x7f468810b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44460/0 (socket says 172.21.15.112:44460) 2024-05-08T05:42:18.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.075+0000 7f4687400700 1 -- 172.21.15.112:0/2764732216 learned_addr learned my addr 172.21.15.112:0/2764732216 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:18.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.076+0000 7f4687400700 1 -- 172.21.15.112:0/2764732216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468810bfe0 con 0x7f468810ae80 2024-05-08T05:42:18.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.076+0000 7f4687400700 1 --2- 172.21.15.112:0/2764732216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f468810ae80 0x7f468810b2a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f46780098a0 tx=0x7f4678009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c6477081e1ffaa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:18.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.076+0000 7f4686000700 1 -- 172.21.15.112:0/2764732216 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4678004160 con 0x7f468810ae80 2024-05-08T05:42:18.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.076+0000 7f4686000700 1 -- 172.21.15.112:0/2764732216 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4678004300 con 0x7f468810ae80 2024-05-08T05:42:18.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.076+0000 7f4686000700 1 -- 172.21.15.112:0/2764732216 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4678011af0 con 0x7f468810ae80 2024-05-08T05:42:18.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.077+0000 7f468cc00700 1 -- 172.21.15.112:0/2764732216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f468810ae80 msgr2=0x7f468810b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:18.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.077+0000 7f468cc00700 1 --2- 172.21.15.112:0/2764732216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f468810ae80 0x7f468810b2a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f46780098a0 tx=0x7f4678009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.077+0000 7f468cc00700 1 -- 172.21.15.112:0/2764732216 shutdown_connections 2024-05-08T05:42:18.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.077+0000 7f468cc00700 1 --2- 172.21.15.112:0/2764732216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f468810ae80 0x7f468810b2a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.077+0000 7f468cc00700 1 -- 172.21.15.112:0/2764732216 >> 172.21.15.112:0/2764732216 conn(0x7f468807e250 msgr2=0x7f468807e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:18.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.077+0000 7f468cc00700 1 -- 172.21.15.112:0/2764732216 shutdown_connections 2024-05-08T05:42:18.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.077+0000 7f468cc00700 1 -- 172.21.15.112:0/2764732216 wait complete. 2024-05-08T05:42:18.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.078+0000 7f468cc00700 1 Processor -- start 2024-05-08T05:42:18.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.079+0000 7f468cc00700 1 -- start start 2024-05-08T05:42:18.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.079+0000 7f468cc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f468810ae80 0x7f468819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:18.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.079+0000 7f468cc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f468810f420 con 0x7f468810ae80 2024-05-08T05:42:18.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.079+0000 7f4687400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f468810ae80 0x7f468819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:18.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.079+0000 7f4687400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f468810ae80 0x7f468819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44470/0 (socket says 172.21.15.112:44470) 2024-05-08T05:42:18.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.079+0000 7f4687400700 1 -- 172.21.15.112:0/2834756296 learned_addr learned my addr 172.21.15.112:0/2834756296 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:18.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.080+0000 7f4687400700 1 -- 172.21.15.112:0/2834756296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468819f350 con 0x7f468810ae80 2024-05-08T05:42:18.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.080+0000 7f4687400700 1 --2- 172.21.15.112:0/2834756296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f468810ae80 0x7f468819ee10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4678011ac0 tx=0x7f4678011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:18.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.080+0000 7f467fe00700 1 -- 172.21.15.112:0/2834756296 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4678003970 con 0x7f468810ae80 2024-05-08T05:42:18.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.080+0000 7f467fe00700 1 -- 172.21.15.112:0/2834756296 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4678003b10 con 0x7f468810ae80 2024-05-08T05:42:18.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.080+0000 7f467fe00700 1 -- 172.21.15.112:0/2834756296 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4678024610 con 0x7f468810ae80 2024-05-08T05:42:18.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.080+0000 7f468cc00700 1 -- 172.21.15.112:0/2834756296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468819f5e0 con 0x7f468810ae80 2024-05-08T05:42:18.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.080+0000 7f468cc00700 1 -- 172.21.15.112:0/2834756296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46881a22d0 con 0x7f468810ae80 2024-05-08T05:42:18.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.081+0000 7f468cc00700 1 -- 172.21.15.112:0/2834756296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4688064b00 con 0x7f468810ae80 2024-05-08T05:42:18.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.082+0000 7f467fe00700 1 -- 172.21.15.112:0/2834756296 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f467801a9e0 con 0x7f468810ae80 2024-05-08T05:42:18.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.082+0000 7f467fe00700 1 --2- 172.21.15.112:0/2834756296 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f466c03e470 0x7f466c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:18.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.082+0000 7f467fe00700 1 -- 172.21.15.112:0/2834756296 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4678053840 con 0x7f468810ae80 2024-05-08T05:42:18.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.082+0000 7f4686a00700 1 --2- 172.21.15.112:0/2834756296 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f466c03e470 0x7f466c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:18.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.083+0000 7f4686a00700 1 --2- 172.21.15.112:0/2834756296 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f466c03e470 0x7f466c040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4674006fd0 tx=0x7f4674006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:18.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.089+0000 7f467fe00700 1 -- 172.21.15.112:0/2834756296 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f46780247f0 con 0x7f468810ae80 2024-05-08T05:42:18.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.409+0000 7f468cc00700 1 -- 172.21.15.112:0/2834756296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f46881a24a0 con 0x7f468810ae80 2024-05-08T05:42:18.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.410+0000 7f467fe00700 1 -- 172.21.15.112:0/2834756296 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f46881a24a0 con 0x7f468810ae80 2024-05-08T05:42:18.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.413+0000 7f468cc00700 1 -- 172.21.15.112:0/2834756296 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f466c03e470 msgr2=0x7f466c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:18.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.413+0000 7f468cc00700 1 --2- 172.21.15.112:0/2834756296 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f466c03e470 0x7f466c040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4674006fd0 tx=0x7f4674006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:18.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.413+0000 7f468cc00700 1 -- 172.21.15.112:0/2834756296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f468810ae80 msgr2=0x7f468819ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:18.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.413+0000 7f468cc00700 1 --2- 172.21.15.112:0/2834756296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f468810ae80 0x7f468819ee10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4678011ac0 tx=0x7f4678011ba0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.414+0000 7f468cc00700 1 -- 172.21.15.112:0/2834756296 shutdown_connections 2024-05-08T05:42:18.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.414+0000 7f468cc00700 1 --2- 172.21.15.112:0/2834756296 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f466c03e470 0x7f466c040930 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.414+0000 7f468cc00700 1 --2- 172.21.15.112:0/2834756296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f468810ae80 0x7f468819ee10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.414+0000 7f468cc00700 1 -- 172.21.15.112:0/2834756296 >> 172.21.15.112:0/2834756296 conn(0x7f468807e250 msgr2=0x7f4688108410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:18.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.414+0000 7f468cc00700 1 -- 172.21.15.112:0/2834756296 shutdown_connections 2024-05-08T05:42:18.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.414+0000 7f468cc00700 1 -- 172.21.15.112:0/2834756296 wait complete. 2024-05-08T05:42:18.143 INFO:teuthology.orchestra.run.smithi112.stdout:mgr is available 2024-05-08T05:42:19.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:19 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3899043407' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T05:42:19.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-08T05:42:19.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T05:42:19.674 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:42:19.675 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-08T05:42:19.675 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T05:42:19.676 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T05:42:19.676 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T05:42:19.676 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T05:42:19.676 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-08T05:42:19.676 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T05:42:19.676 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T05:42:19.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-05-08T05:42:19.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T05:42:19.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-08T05:42:19.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T05:42:19.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T05:42:19.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T05:42:19.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T05:42:19.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T05:42:19.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.710+0000 7f02f2000700 1 Processor -- start 2024-05-08T05:42:19.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.710+0000 7f02f2000700 1 -- start start 2024-05-08T05:42:19.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.711+0000 7f02f2000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02ec10fc50 0x7f02ec112040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:19.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.711+0000 7f02f2000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec0834f0 con 0x7f02ec10fc50 2024-05-08T05:42:19.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.711+0000 7f02f0c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02ec10fc50 0x7f02ec112040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:19.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.711+0000 7f02f0c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02ec10fc50 0x7f02ec112040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44474/0 (socket says 172.21.15.112:44474) 2024-05-08T05:42:19.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.711+0000 7f02f0c00700 1 -- 172.21.15.112:0/2960616812 learned_addr learned my addr 172.21.15.112:0/2960616812 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:19.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.712+0000 7f02f0c00700 1 -- 172.21.15.112:0/2960616812 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ec112580 con 0x7f02ec10fc50 2024-05-08T05:42:19.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.712+0000 7f02f0c00700 1 --2- 172.21.15.112:0/2960616812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02ec10fc50 0x7f02ec112040 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f02e00098a0 tx=0x7f02e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a583cc4249a28eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:19.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.712+0000 7f02eb400700 1 -- 172.21.15.112:0/2960616812 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02e0004160 con 0x7f02ec10fc50 2024-05-08T05:42:19.681 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.712+0000 7f02eb400700 1 -- 172.21.15.112:0/2960616812 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f02e0004300 con 0x7f02ec10fc50 2024-05-08T05:42:19.681 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.712+0000 7f02eb400700 1 -- 172.21.15.112:0/2960616812 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02e0011af0 con 0x7f02ec10fc50 2024-05-08T05:42:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.713+0000 7f02f2000700 1 -- 172.21.15.112:0/2960616812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02ec10fc50 msgr2=0x7f02ec112040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.713+0000 7f02f2000700 1 --2- 172.21.15.112:0/2960616812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02ec10fc50 0x7f02ec112040 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f02e00098a0 tx=0x7f02e0009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.713+0000 7f02f2000700 1 -- 172.21.15.112:0/2960616812 shutdown_connections 2024-05-08T05:42:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.713+0000 7f02f2000700 1 --2- 172.21.15.112:0/2960616812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02ec10fc50 0x7f02ec112040 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.713+0000 7f02f2000700 1 -- 172.21.15.112:0/2960616812 >> 172.21.15.112:0/2960616812 conn(0x7f02ec07e250 msgr2=0x7f02ec080690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.714+0000 7f02f2000700 1 -- 172.21.15.112:0/2960616812 shutdown_connections 2024-05-08T05:42:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.714+0000 7f02f2000700 1 -- 172.21.15.112:0/2960616812 wait complete. 2024-05-08T05:42:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.714+0000 7f02f2000700 1 Processor -- start 2024-05-08T05:42:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.715+0000 7f02f2000700 1 -- start start 2024-05-08T05:42:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.715+0000 7f02f2000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02ec10fc50 0x7f02ec1a3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.715+0000 7f02f2000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec113600 con 0x7f02ec10fc50 2024-05-08T05:42:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.715+0000 7f02f0c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02ec10fc50 0x7f02ec1a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.715+0000 7f02f0c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02ec10fc50 0x7f02ec1a3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44490/0 (socket says 172.21.15.112:44490) 2024-05-08T05:42:19.683 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.715+0000 7f02f0c00700 1 -- 172.21.15.112:0/3899043407 learned_addr learned my addr 172.21.15.112:0/3899043407 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:19.683 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.716+0000 7f02f0c00700 1 -- 172.21.15.112:0/3899043407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ec1a3740 con 0x7f02ec10fc50 2024-05-08T05:42:19.683 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.716+0000 7f02f0c00700 1 --2- 172.21.15.112:0/3899043407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02ec10fc50 0x7f02ec1a3200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f02e0011ac0 tx=0x7f02e0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:19.683 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.716+0000 7f02e9600700 1 -- 172.21.15.112:0/3899043407 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02e0003970 con 0x7f02ec10fc50 2024-05-08T05:42:19.683 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.716+0000 7f02e9600700 1 -- 172.21.15.112:0/3899043407 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f02e0003b10 con 0x7f02ec10fc50 2024-05-08T05:42:19.685 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.716+0000 7f02e9600700 1 -- 172.21.15.112:0/3899043407 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02e0024610 con 0x7f02ec10fc50 2024-05-08T05:42:19.685 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.716+0000 7f02f2000700 1 -- 172.21.15.112:0/3899043407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02ec1a39d0 con 0x7f02ec10fc50 2024-05-08T05:42:19.685 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.716+0000 7f02f2000700 1 -- 172.21.15.112:0/3899043407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02ec1a66c0 con 0x7f02ec10fc50 2024-05-08T05:42:19.685 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.718+0000 7f02e9600700 1 -- 172.21.15.112:0/3899043407 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f02e001a9e0 con 0x7f02ec10fc50 2024-05-08T05:42:19.685 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.718+0000 7f02f2000700 1 -- 172.21.15.112:0/3899043407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02ec064b00 con 0x7f02ec10fc50 2024-05-08T05:42:19.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.718+0000 7f02e9600700 1 --2- 172.21.15.112:0/3899043407 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f02d403e470 0x7f02d4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:19.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.718+0000 7f02e9600700 1 -- 172.21.15.112:0/3899043407 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f02e00537f0 con 0x7f02ec10fc50 2024-05-08T05:42:19.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.718+0000 7f02ebe00700 1 --2- 172.21.15.112:0/3899043407 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f02d403e470 0x7f02d4040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:19.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.719+0000 7f02ebe00700 1 --2- 172.21.15.112:0/3899043407 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f02d403e470 0x7f02d4040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f02d8006fd0 tx=0x7f02d8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:19.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.724+0000 7f02e9600700 1 -- 172.21.15.112:0/3899043407 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02e0053210 con 0x7f02ec10fc50 2024-05-08T05:42:19.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.960+0000 7f02f2000700 1 -- 172.21.15.112:0/3899043407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f02ec1a6890 con 0x7f02ec10fc50 2024-05-08T05:42:19.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.962+0000 7f02e9600700 1 -- 172.21.15.112:0/3899043407 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f02ec1a6890 con 0x7f02ec10fc50 2024-05-08T05:42:19.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.965+0000 7f02f2000700 1 -- 172.21.15.112:0/3899043407 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f02d403e470 msgr2=0x7f02d4040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:19.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.965+0000 7f02f2000700 1 --2- 172.21.15.112:0/3899043407 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f02d403e470 0x7f02d4040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f02d8006fd0 tx=0x7f02d8006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:19.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.965+0000 7f02f2000700 1 -- 172.21.15.112:0/3899043407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02ec10fc50 msgr2=0x7f02ec1a3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:19.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.965+0000 7f02f2000700 1 --2- 172.21.15.112:0/3899043407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02ec10fc50 0x7f02ec1a3200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f02e0011ac0 tx=0x7f02e0011ba0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.965+0000 7f02f2000700 1 -- 172.21.15.112:0/3899043407 shutdown_connections 2024-05-08T05:42:19.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.965+0000 7f02f2000700 1 --2- 172.21.15.112:0/3899043407 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f02d403e470 0x7f02d4040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.965+0000 7f02f2000700 1 --2- 172.21.15.112:0/3899043407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02ec10fc50 0x7f02ec1a3200 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.965+0000 7f02f2000700 1 -- 172.21.15.112:0/3899043407 >> 172.21.15.112:0/3899043407 conn(0x7f02ec07e250 msgr2=0x7f02ec07ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:19.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.965+0000 7f02f2000700 1 -- 172.21.15.112:0/3899043407 shutdown_connections 2024-05-08T05:42:19.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.965+0000 7f02f2000700 1 -- 172.21.15.112:0/3899043407 wait complete. 2024-05-08T05:42:19.687 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling cephadm module... 2024-05-08T05:42:20.935 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:20 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/911630585' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-08T05:42:20.936 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:20 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: ignoring --setuser ceph since I am not root 2024-05-08T05:42:20.936 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:20 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: ignoring --setgroup ceph since I am not root 2024-05-08T05:42:20.936 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:20 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:20.934+0000 7faeb8c55200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:42:21.312 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:21 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:21.178+0000 7faeb8c55200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:42:21.312 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:21 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:21.310+0000 7faeb8c55200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:42:21.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.301+0000 7f7cc7200700 1 Processor -- start 2024-05-08T05:42:21.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.301+0000 7f7cc7200700 1 -- start start 2024-05-08T05:42:21.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.302+0000 7f7cc7200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc010fc50 0x7f7cc0112040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:21.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.302+0000 7f7cc7200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc00834f0 con 0x7f7cc010fc50 2024-05-08T05:42:21.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.302+0000 7f7cc5e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc010fc50 0x7f7cc0112040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:21.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.302+0000 7f7cc5e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc010fc50 0x7f7cc0112040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44500/0 (socket says 172.21.15.112:44500) 2024-05-08T05:42:21.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.302+0000 7f7cc5e00700 1 -- 172.21.15.112:0/1692355524 learned_addr learned my addr 172.21.15.112:0/1692355524 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:21.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.303+0000 7f7cc5e00700 1 -- 172.21.15.112:0/1692355524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc0112580 con 0x7f7cc010fc50 2024-05-08T05:42:21.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.303+0000 7f7cc5e00700 1 --2- 172.21.15.112:0/1692355524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc010fc50 0x7f7cc0112040 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40098a0 tx=0x7f7cb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13afb5ce9ad38f09 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:21.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.303+0000 7f7cc4a00700 1 -- 172.21.15.112:0/1692355524 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cb4004160 con 0x7f7cc010fc50 2024-05-08T05:42:21.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.304+0000 7f7cc4a00700 1 -- 172.21.15.112:0/1692355524 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7cb4004300 con 0x7f7cc010fc50 2024-05-08T05:42:21.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.304+0000 7f7cc4a00700 1 -- 172.21.15.112:0/1692355524 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cb4011af0 con 0x7f7cc010fc50 2024-05-08T05:42:21.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.304+0000 7f7cc7200700 1 -- 172.21.15.112:0/1692355524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc010fc50 msgr2=0x7f7cc0112040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:21.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.304+0000 7f7cc7200700 1 --2- 172.21.15.112:0/1692355524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc010fc50 0x7f7cc0112040 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40098a0 tx=0x7f7cb4009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.304+0000 7f7cc7200700 1 -- 172.21.15.112:0/1692355524 shutdown_connections 2024-05-08T05:42:21.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.304+0000 7f7cc7200700 1 --2- 172.21.15.112:0/1692355524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc010fc50 0x7f7cc0112040 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.304+0000 7f7cc7200700 1 -- 172.21.15.112:0/1692355524 >> 172.21.15.112:0/1692355524 conn(0x7f7cc007e250 msgr2=0x7f7cc0080690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:21.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.305+0000 7f7cc7200700 1 -- 172.21.15.112:0/1692355524 shutdown_connections 2024-05-08T05:42:21.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.305+0000 7f7cc7200700 1 -- 172.21.15.112:0/1692355524 wait complete. 2024-05-08T05:42:21.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.306+0000 7f7cc7200700 1 Processor -- start 2024-05-08T05:42:21.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.306+0000 7f7cc7200700 1 -- start start 2024-05-08T05:42:21.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.306+0000 7f7cc7200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc010fc50 0x7f7cc01a3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:21.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.306+0000 7f7cc7200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc0113600 con 0x7f7cc010fc50 2024-05-08T05:42:21.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.306+0000 7f7cc5e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc010fc50 0x7f7cc01a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:21.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.306+0000 7f7cc5e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc010fc50 0x7f7cc01a3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44508/0 (socket says 172.21.15.112:44508) 2024-05-08T05:42:21.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.306+0000 7f7cc5e00700 1 -- 172.21.15.112:0/911630585 learned_addr learned my addr 172.21.15.112:0/911630585 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:21.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.307+0000 7f7cc5e00700 1 -- 172.21.15.112:0/911630585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc01a3740 con 0x7f7cc010fc50 2024-05-08T05:42:21.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.307+0000 7f7cc5e00700 1 --2- 172.21.15.112:0/911630585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc010fc50 0x7f7cc01a3200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4011ac0 tx=0x7f7cb4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:21.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.307+0000 7f7cbaa00700 1 -- 172.21.15.112:0/911630585 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cb4003970 con 0x7f7cc010fc50 2024-05-08T05:42:21.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.307+0000 7f7cbaa00700 1 -- 172.21.15.112:0/911630585 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7cb4003b10 con 0x7f7cc010fc50 2024-05-08T05:42:21.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.307+0000 7f7cbaa00700 1 -- 172.21.15.112:0/911630585 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cb4024610 con 0x7f7cc010fc50 2024-05-08T05:42:21.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.307+0000 7f7cc7200700 1 -- 172.21.15.112:0/911630585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc01a39d0 con 0x7f7cc010fc50 2024-05-08T05:42:21.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.307+0000 7f7cc7200700 1 -- 172.21.15.112:0/911630585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc01a66c0 con 0x7f7cc010fc50 2024-05-08T05:42:21.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.309+0000 7f7cc7200700 1 -- 172.21.15.112:0/911630585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc0064b00 con 0x7f7cc010fc50 2024-05-08T05:42:21.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.309+0000 7f7cbaa00700 1 -- 172.21.15.112:0/911630585 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f7cb401a9e0 con 0x7f7cc010fc50 2024-05-08T05:42:21.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.309+0000 7f7cbaa00700 1 --2- 172.21.15.112:0/911630585 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f7ca803e470 0x7f7ca8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:21.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.309+0000 7f7cbaa00700 1 -- 172.21.15.112:0/911630585 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7cb4053840 con 0x7f7cc010fc50 2024-05-08T05:42:21.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.310+0000 7f7cc5400700 1 --2- 172.21.15.112:0/911630585 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f7ca803e470 0x7f7ca8040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:21.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.310+0000 7f7cc5400700 1 --2- 172.21.15.112:0/911630585 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f7ca803e470 0x7f7ca8040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0006fd0 tx=0x7f7cb0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:21.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.317+0000 7f7cbaa00700 1 -- 172.21.15.112:0/911630585 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7cb40247f0 con 0x7f7cc010fc50 2024-05-08T05:42:21.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.608+0000 7f7cc7200700 1 -- 172.21.15.112:0/911630585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f7cc01a6890 con 0x7f7cc010fc50 2024-05-08T05:42:21.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.662+0000 7f7cbaa00700 1 -- 172.21.15.112:0/911630585 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f7cc01a6890 con 0x7f7cc010fc50 2024-05-08T05:42:21.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.666+0000 7f7cbaa00700 1 -- 172.21.15.112:0/911630585 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f7cb4055c80 con 0x7f7cc010fc50 2024-05-08T05:42:21.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.668+0000 7f7cc7200700 1 -- 172.21.15.112:0/911630585 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f7ca803e470 msgr2=0x7f7ca8040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:21.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.668+0000 7f7cc7200700 1 --2- 172.21.15.112:0/911630585 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f7ca803e470 0x7f7ca8040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0006fd0 tx=0x7f7cb0006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:21.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.668+0000 7f7cc7200700 1 -- 172.21.15.112:0/911630585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc010fc50 msgr2=0x7f7cc01a3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:21.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.668+0000 7f7cc7200700 1 --2- 172.21.15.112:0/911630585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc010fc50 0x7f7cc01a3200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4011ac0 tx=0x7f7cb4011ba0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.668+0000 7f7cc7200700 1 -- 172.21.15.112:0/911630585 shutdown_connections 2024-05-08T05:42:21.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.668+0000 7f7cc7200700 1 --2- 172.21.15.112:0/911630585 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f7ca803e470 0x7f7ca8040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.668+0000 7f7cc7200700 1 --2- 172.21.15.112:0/911630585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cc010fc50 0x7f7cc01a3200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.668+0000 7f7cc7200700 1 -- 172.21.15.112:0/911630585 >> 172.21.15.112:0/911630585 conn(0x7f7cc007e250 msgr2=0x7f7cc007ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:21.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.668+0000 7f7cc7200700 1 -- 172.21.15.112:0/911630585 shutdown_connections 2024-05-08T05:42:21.431 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.668+0000 7f7cc7200700 1 -- 172.21.15.112:0/911630585 wait complete. 2024-05-08T05:42:21.854 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:21 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:21.736+0000 7faeb8c55200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:42:21.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:21 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/911630585' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-08T05:42:21.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:21 smithi112 ceph-mon[30003]: mgrmap e5: x(active, since 5s) 2024-05-08T05:42:22.268 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:21 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:21.853+0000 7faeb8c55200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:42:23.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:22 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2822541883' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T05:42:23.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:42:23.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-08T05:42:23.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T05:42:23.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T05:42:23.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T05:42:23.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:42:23.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.059+0000 7fafc8c00700 1 Processor -- start 2024-05-08T05:42:23.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.059+0000 7fafc8c00700 1 -- start start 2024-05-08T05:42:23.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.060+0000 7fafc8c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc407ee40 0x7fafc407d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:23.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.060+0000 7fafc8c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fafc407d870 con 0x7fafc407ee40 2024-05-08T05:42:23.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.060+0000 7fafc1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc407ee40 0x7fafc407d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:23.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.060+0000 7fafc1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc407ee40 0x7fafc407d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44532/0 (socket says 172.21.15.112:44532) 2024-05-08T05:42:23.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.060+0000 7fafc1e00700 1 -- 172.21.15.112:0/3046680901 learned_addr learned my addr 172.21.15.112:0/3046680901 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:23.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.060+0000 7fafc1e00700 1 -- 172.21.15.112:0/3046680901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc407d9f0 con 0x7fafc407ee40 2024-05-08T05:42:23.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.061+0000 7fafc1e00700 1 --2- 172.21.15.112:0/3046680901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc407ee40 0x7fafc407d2a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fafb40098a0 tx=0x7fafb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b5d3f75a083e40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:23.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.061+0000 7fafc0a00700 1 -- 172.21.15.112:0/3046680901 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb4004160 con 0x7fafc407ee40 2024-05-08T05:42:23.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.061+0000 7fafc0a00700 1 -- 172.21.15.112:0/3046680901 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fafb4004300 con 0x7fafc407ee40 2024-05-08T05:42:23.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.061+0000 7fafc0a00700 1 -- 172.21.15.112:0/3046680901 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb4011af0 con 0x7fafc407ee40 2024-05-08T05:42:23.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.062+0000 7fafc8c00700 1 -- 172.21.15.112:0/3046680901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc407ee40 msgr2=0x7fafc407d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:23.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.062+0000 7fafc8c00700 1 --2- 172.21.15.112:0/3046680901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc407ee40 0x7fafc407d2a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fafb40098a0 tx=0x7fafb4009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.062+0000 7fafc8c00700 1 -- 172.21.15.112:0/3046680901 shutdown_connections 2024-05-08T05:42:23.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.062+0000 7fafc8c00700 1 --2- 172.21.15.112:0/3046680901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc407ee40 0x7fafc407d2a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.062+0000 7fafc8c00700 1 -- 172.21.15.112:0/3046680901 >> 172.21.15.112:0/3046680901 conn(0x7fafc41043f0 msgr2=0x7fafc4106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:23.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.062+0000 7fafc8c00700 1 -- 172.21.15.112:0/3046680901 shutdown_connections 2024-05-08T05:42:23.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.062+0000 7fafc8c00700 1 -- 172.21.15.112:0/3046680901 wait complete. 2024-05-08T05:42:23.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.063+0000 7fafc8c00700 1 Processor -- start 2024-05-08T05:42:23.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.063+0000 7fafc8c00700 1 -- start start 2024-05-08T05:42:23.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.064+0000 7fafc8c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc407ee40 0x7fafc419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:23.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.064+0000 7fafc8c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fafc410f3c0 con 0x7fafc407ee40 2024-05-08T05:42:23.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.064+0000 7fafc1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc407ee40 0x7fafc419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:23.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.064+0000 7fafc1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc407ee40 0x7fafc419efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44548/0 (socket says 172.21.15.112:44548) 2024-05-08T05:42:23.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.064+0000 7fafc1e00700 1 -- 172.21.15.112:0/2822541883 learned_addr learned my addr 172.21.15.112:0/2822541883 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:23.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.064+0000 7fafc1e00700 1 -- 172.21.15.112:0/2822541883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc419f510 con 0x7fafc407ee40 2024-05-08T05:42:23.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.064+0000 7fafc1e00700 1 --2- 172.21.15.112:0/2822541883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc407ee40 0x7fafc419efd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fafb4000c00 tx=0x7fafb4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:23.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.065+0000 7fafbaa00700 1 -- 172.21.15.112:0/2822541883 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb4003b20 con 0x7fafc407ee40 2024-05-08T05:42:23.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.065+0000 7fafbaa00700 1 -- 172.21.15.112:0/2822541883 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fafb401a8c0 con 0x7fafc407ee40 2024-05-08T05:42:23.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.065+0000 7fafbaa00700 1 -- 172.21.15.112:0/2822541883 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb4024940 con 0x7fafc407ee40 2024-05-08T05:42:23.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.065+0000 7fafc8c00700 1 -- 172.21.15.112:0/2822541883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc419f7a0 con 0x7fafc407ee40 2024-05-08T05:42:23.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.065+0000 7fafc8c00700 1 -- 172.21.15.112:0/2822541883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc41a2490 con 0x7fafc407ee40 2024-05-08T05:42:23.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.066+0000 7fafbaa00700 1 -- 172.21.15.112:0/2822541883 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fafb4022020 con 0x7fafc407ee40 2024-05-08T05:42:23.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.066+0000 7fafc8c00700 1 -- 172.21.15.112:0/2822541883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc4064b00 con 0x7fafc407ee40 2024-05-08T05:42:23.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.067+0000 7fafbaa00700 1 --2- 172.21.15.112:0/2822541883 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7fafa803e4c0 0x7fafa8040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:23.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.067+0000 7fafbaa00700 1 -- 172.21.15.112:0/2822541883 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fafb40533c0 con 0x7fafc407ee40 2024-05-08T05:42:23.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.067+0000 7fafc1400700 1 -- 172.21.15.112:0/2822541883 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7fafa803e4c0 msgr2=0x7fafa8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/772108158 2024-05-08T05:42:23.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.067+0000 7fafc1400700 1 --2- 172.21.15.112:0/2822541883 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7fafa803e4c0 0x7fafa8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:42:23.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.072+0000 7fafbaa00700 1 -- 172.21.15.112:0/2822541883 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fafb4029df0 con 0x7fafc407ee40 2024-05-08T05:42:23.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.268+0000 7fafc1400700 1 -- 172.21.15.112:0/2822541883 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7fafa803e4c0 msgr2=0x7fafa8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/772108158 2024-05-08T05:42:23.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.268+0000 7fafc1400700 1 --2- 172.21.15.112:0/2822541883 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7fafa803e4c0 0x7fafa8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:42:23.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.363+0000 7fafc8c00700 1 -- 172.21.15.112:0/2822541883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fafc41a2660 con 0x7fafc407ee40 2024-05-08T05:42:23.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.364+0000 7fafbaa00700 1 -- 172.21.15.112:0/2822541883 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fafc41a2660 con 0x7fafc407ee40 2024-05-08T05:42:23.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.369+0000 7fafc8c00700 1 -- 172.21.15.112:0/2822541883 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7fafa803e4c0 msgr2=0x7fafa8040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:42:23.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.369+0000 7fafc8c00700 1 --2- 172.21.15.112:0/2822541883 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7fafa803e4c0 0x7fafa8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.369+0000 7fafc8c00700 1 -- 172.21.15.112:0/2822541883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc407ee40 msgr2=0x7fafc419efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:23.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.369+0000 7fafc8c00700 1 --2- 172.21.15.112:0/2822541883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc407ee40 0x7fafc419efd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fafb4000c00 tx=0x7fafb4011b70 comp rx=0 tx=0).stop 2024-05-08T05:42:23.100 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.369+0000 7fafc8c00700 1 -- 172.21.15.112:0/2822541883 shutdown_connections 2024-05-08T05:42:23.100 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.369+0000 7fafc8c00700 1 --2- 172.21.15.112:0/2822541883 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7fafa803e4c0 0x7fafa8040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.100 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.369+0000 7fafc8c00700 1 --2- 172.21.15.112:0/2822541883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc407ee40 0x7fafc419efd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.100 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.369+0000 7fafc8c00700 1 -- 172.21.15.112:0/2822541883 >> 172.21.15.112:0/2822541883 conn(0x7fafc41043f0 msgr2=0x7fafc4104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:23.100 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.369+0000 7fafc8c00700 1 -- 172.21.15.112:0/2822541883 shutdown_connections 2024-05-08T05:42:23.100 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.369+0000 7fafc8c00700 1 -- 172.21.15.112:0/2822541883 wait complete. 2024-05-08T05:42:23.100 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-05-08T05:42:23.100 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 5... 2024-05-08T05:42:23.681 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:23 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:23.436+0000 7faeb8c55200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:42:23.681 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:23 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:23.681+0000 7faeb8c55200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:42:24.065 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:23 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:23.896+0000 7faeb8c55200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:42:24.565 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:24 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:24.267+0000 7faeb8c55200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:42:24.566 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:24 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:24.389+0000 7faeb8c55200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:42:25.066 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:24 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:24.637+0000 7faeb8c55200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:42:25.066 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:24 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:24.886+0000 7faeb8c55200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:42:25.955 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:25 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:25.669+0000 7faeb8c55200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:42:25.955 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:25 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:25.954+0000 7faeb8c55200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:42:26.816 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:26 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:26.503+0000 7faeb8c55200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:42:26.816 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:26 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:26.630+0000 7faeb8c55200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:42:27.531 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:27 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:27.087+0000 7faeb8c55200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:42:27.531 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:27 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:27.530+0000 7faeb8c55200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:42:27.815 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:27 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:27.652+0000 7faeb8c55200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:42:28.271 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:27 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:27.968+0000 7faeb8c55200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:42:28.271 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:28 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:28.143+0000 7faeb8c55200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:42:28.271 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:28 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:28.271+0000 7faeb8c55200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:42:29.066 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:28 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:28.664+0000 7faeb8c55200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:42:29.066 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:28 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:28.788+0000 7faeb8c55200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:42:29.815 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:29 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:29.532+0000 7faeb8c55200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:42:29.816 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:29 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:29.814+0000 7faeb8c55200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:42:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:29 smithi112 ceph-mon[30003]: Active manager daemon x restarted 2024-05-08T05:42:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:29 smithi112 ceph-mon[30003]: Activating manager daemon x 2024-05-08T05:42:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:29 smithi112 ceph-mon[30003]: osdmap e2: 0 total, 0 up, 0 in 2024-05-08T05:42:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:29 smithi112 ceph-mon[30003]: mgrmap e6: x(active, starting, since 0.0118055s) 2024-05-08T05:42:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:29 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:42:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:29 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:42:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:29 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:42:30.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:29 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:42:30.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:29 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:42:30.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:29 smithi112 ceph-mon[30003]: Manager daemon x is now available 2024-05-08T05:42:31.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:30 smithi112 ceph-mon[30003]: Found migration_current of "None". Setting to last migration. 2024-05-08T05:42:31.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:30 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' 2024-05-08T05:42:31.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:30 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' 2024-05-08T05:42:31.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:30 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:42:31.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:30 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:42:31.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:30 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:42:31.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:30 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:42:31.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:30 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' 2024-05-08T05:42:31.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:30 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' 2024-05-08T05:42:31.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:30 smithi112 ceph-mon[30003]: mgrmap e7: x(active, since 1.01687s) 2024-05-08T05:42:31.539 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:42:31.540 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-08T05:42:31.540 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T05:42:31.540 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:42:31.541 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.698+0000 7f5025800700 1 Processor -- start 2024-05-08T05:42:31.541 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.698+0000 7f5025800700 1 -- start start 2024-05-08T05:42:31.541 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.698+0000 7f5025800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f502007ee00 0x7f502007d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:31.544 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.698+0000 7f5025800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f502007d830 con 0x7f502007ee00 2024-05-08T05:42:31.544 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.699+0000 7f501ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f502007ee00 0x7f502007d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:31.545 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.699+0000 7f501ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f502007ee00 0x7f502007d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44552/0 (socket says 172.21.15.112:44552) 2024-05-08T05:42:31.545 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.699+0000 7f501ea00700 1 -- 172.21.15.112:0/1762663853 learned_addr learned my addr 172.21.15.112:0/1762663853 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:31.545 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.699+0000 7f501ea00700 1 -- 172.21.15.112:0/1762663853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502007d9b0 con 0x7f502007ee00 2024-05-08T05:42:31.545 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.699+0000 7f501ea00700 1 --2- 172.21.15.112:0/1762663853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f502007ee00 0x7f502007d260 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f50100098a0 tx=0x7f5010009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197db80db386b9d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:31.545 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.700+0000 7f501d600700 1 -- 172.21.15.112:0/1762663853 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5010004160 con 0x7f502007ee00 2024-05-08T05:42:31.545 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.700+0000 7f501d600700 1 -- 172.21.15.112:0/1762663853 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5010004300 con 0x7f502007ee00 2024-05-08T05:42:31.545 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.700+0000 7f501d600700 1 -- 172.21.15.112:0/1762663853 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5010011af0 con 0x7f502007ee00 2024-05-08T05:42:31.545 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.700+0000 7f5025800700 1 -- 172.21.15.112:0/1762663853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f502007ee00 msgr2=0x7f502007d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:31.545 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.700+0000 7f5025800700 1 --2- 172.21.15.112:0/1762663853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f502007ee00 0x7f502007d260 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f50100098a0 tx=0x7f5010009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:31.545 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.701+0000 7f5025800700 1 -- 172.21.15.112:0/1762663853 shutdown_connections 2024-05-08T05:42:31.545 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.701+0000 7f5025800700 1 --2- 172.21.15.112:0/1762663853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f502007ee00 0x7f502007d260 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:31.545 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.701+0000 7f5025800700 1 -- 172.21.15.112:0/1762663853 >> 172.21.15.112:0/1762663853 conn(0x7f50201043f0 msgr2=0x7f5020106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:31.545 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.701+0000 7f5025800700 1 -- 172.21.15.112:0/1762663853 shutdown_connections 2024-05-08T05:42:31.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.701+0000 7f5025800700 1 -- 172.21.15.112:0/1762663853 wait complete. 2024-05-08T05:42:31.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.702+0000 7f5025800700 1 Processor -- start 2024-05-08T05:42:31.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.702+0000 7f5025800700 1 -- start start 2024-05-08T05:42:31.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.702+0000 7f5025800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f502007ee00 0x7f502019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:31.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.702+0000 7f5025800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f502010f400 con 0x7f502007ee00 2024-05-08T05:42:31.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.703+0000 7f501ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f502007ee00 0x7f502019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:31.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.703+0000 7f501ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f502007ee00 0x7f502019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44558/0 (socket says 172.21.15.112:44558) 2024-05-08T05:42:31.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.703+0000 7f501ea00700 1 -- 172.21.15.112:0/645936714 learned_addr learned my addr 172.21.15.112:0/645936714 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:31.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.703+0000 7f501ea00700 1 -- 172.21.15.112:0/645936714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502019f520 con 0x7f502007ee00 2024-05-08T05:42:31.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.703+0000 7f501ea00700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f502007ee00 0x7f502019efe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5010011ac0 tx=0x7f5010011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:31.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.703+0000 7f5017400700 1 -- 172.21.15.112:0/645936714 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5010003b20 con 0x7f502007ee00 2024-05-08T05:42:31.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.704+0000 7f5017400700 1 -- 172.21.15.112:0/645936714 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f501001a8c0 con 0x7f502007ee00 2024-05-08T05:42:31.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.704+0000 7f5025800700 1 -- 172.21.15.112:0/645936714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502019f7b0 con 0x7f502007ee00 2024-05-08T05:42:31.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.704+0000 7f5017400700 1 -- 172.21.15.112:0/645936714 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50100248d0 con 0x7f502007ee00 2024-05-08T05:42:31.547 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.704+0000 7f5025800700 1 -- 172.21.15.112:0/645936714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50201a24a0 con 0x7f502007ee00 2024-05-08T05:42:31.550 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.705+0000 7f5017400700 1 -- 172.21.15.112:0/645936714 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f5010022020 con 0x7f502007ee00 2024-05-08T05:42:31.550 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.706+0000 7f5017400700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f500403e4c0 0x7f5004040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:31.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.706+0000 7f5017400700 1 -- 172.21.15.112:0/645936714 --> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5004041090 con 0x7f500403e4c0 2024-05-08T05:42:31.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.706+0000 7f5017400700 1 -- 172.21.15.112:0/645936714 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f50100534d0 con 0x7f502007ee00 2024-05-08T05:42:31.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.706+0000 7f501e000700 1 -- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f500403e4c0 msgr2=0x7f5004040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/772108158 2024-05-08T05:42:31.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.706+0000 7f501e000700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f500403e4c0 0x7f5004040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:42:31.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.907+0000 7f501e000700 1 -- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f500403e4c0 msgr2=0x7f5004040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/772108158 2024-05-08T05:42:31.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.907+0000 7f501e000700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f500403e4c0 0x7f5004040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:42:31.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.308+0000 7f501e000700 1 -- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f500403e4c0 msgr2=0x7f5004040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/772108158 2024-05-08T05:42:31.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.308+0000 7f501e000700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f500403e4c0 0x7f5004040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T05:42:31.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:25.109+0000 7f501e000700 1 -- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f500403e4c0 msgr2=0x7f5004040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/772108158 2024-05-08T05:42:31.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:25.109+0000 7f501e000700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f500403e4c0 0x7f5004040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T05:42:31.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.711+0000 7f501e000700 1 -- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f500403e4c0 msgr2=0x7f5004040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/772108158 2024-05-08T05:42:31.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.711+0000 7f501e000700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f500403e4c0 0x7f5004040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T05:42:31.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:29.826+0000 7f5017400700 1 -- 172.21.15.112:0/645936714 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f5010018530 con 0x7f502007ee00 2024-05-08T05:42:31.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:29.826+0000 7f5017400700 1 -- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f500403e4c0 msgr2=0x7f5004040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:42:31.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:29.826+0000 7f5017400700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f500403e4c0 0x7f5004040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:31.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.830+0000 7f5017400700 1 -- 172.21.15.112:0/645936714 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f5010054af0 con 0x7f502007ee00 2024-05-08T05:42:31.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.830+0000 7f5017400700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f5004041db0 0x7f50040441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:31.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.830+0000 7f5017400700 1 -- 172.21.15.112:0/645936714 --> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5004041090 con 0x7f5004041db0 2024-05-08T05:42:31.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.832+0000 7f501e000700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f5004041db0 0x7f50040441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:31.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.835+0000 7f501e000700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f5004041db0 0x7f50040441a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f500c003a10 tx=0x7f500c0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:31.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.837+0000 7f5017400700 1 -- 172.21.15.112:0/645936714 <== mgr.14120 v2:172.21.15.112:6800/164150456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f5004041090 con 0x7f5004041db0 2024-05-08T05:42:31.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.846+0000 7f5025800700 1 -- 172.21.15.112:0/645936714 --> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5020064b00 con 0x7f5004041db0 2024-05-08T05:42:31.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.846+0000 7f5017400700 1 -- 172.21.15.112:0/645936714 <== mgr.14120 v2:172.21.15.112:6800/164150456 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f5020064b00 con 0x7f5004041db0 2024-05-08T05:42:31.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.847+0000 7f5025800700 1 -- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f5004041db0 msgr2=0x7f50040441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:31.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.847+0000 7f5025800700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f5004041db0 0x7f50040441a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f500c003a10 tx=0x7f500c0092b0 comp rx=0 tx=0).stop 2024-05-08T05:42:31.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.847+0000 7f5025800700 1 -- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f502007ee00 msgr2=0x7f502019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:31.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.847+0000 7f5025800700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f502007ee00 0x7f502019efe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5010011ac0 tx=0x7f5010011ba0 comp rx=0 tx=0).stop 2024-05-08T05:42:31.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.847+0000 7f5025800700 1 -- 172.21.15.112:0/645936714 shutdown_connections 2024-05-08T05:42:31.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.847+0000 7f5025800700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f5004041db0 0x7f50040441a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:31.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.847+0000 7f5025800700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f502007ee00 0x7f502019efe0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:31.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.847+0000 7f5025800700 1 --2- 172.21.15.112:0/645936714 >> [v2:172.21.15.112:6800/772108158,v1:172.21.15.112:6801/772108158] conn(0x7f500403e4c0 0x7f5004040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:31.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.847+0000 7f5025800700 1 -- 172.21.15.112:0/645936714 >> 172.21.15.112:0/645936714 conn(0x7f50201043f0 msgr2=0x7f5020104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:31.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.847+0000 7f5025800700 1 -- 172.21.15.112:0/645936714 shutdown_connections 2024-05-08T05:42:31.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:30.847+0000 7f5025800700 1 -- 172.21.15.112:0/645936714 wait complete. 2024-05-08T05:42:31.553 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 5 is available 2024-05-08T05:42:31.553 INFO:teuthology.orchestra.run.smithi112.stdout:Setting orchestrator backend to cephadm... 2024-05-08T05:42:32.174 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:31 smithi112 ceph-mon[30003]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T05:42:32.174 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:31 smithi112 ceph-mon[30003]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T05:42:32.174 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:31 smithi112 ceph-mon[30003]: [08/May/2024:05:42:30] ENGINE Bus STARTING 2024-05-08T05:42:32.174 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:31 smithi112 ceph-mon[30003]: [08/May/2024:05:42:30] ENGINE Serving on http://172.21.15.112:8765 2024-05-08T05:42:32.174 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:31 smithi112 ceph-mon[30003]: [08/May/2024:05:42:31] ENGINE Serving on https://172.21.15.112:7150 2024-05-08T05:42:32.174 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:31 smithi112 ceph-mon[30003]: [08/May/2024:05:42:31] ENGINE Bus STARTED 2024-05-08T05:42:32.174 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:31 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:42:33.212 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:32 smithi112 ceph-mon[30003]: mgrmap e8: x(active, since 2s) 2024-05-08T05:42:33.212 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:32 smithi112 ceph-mon[30003]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:33.212 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:32 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' 2024-05-08T05:42:33.212 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:32 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:42:33.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.213+0000 7f864ee00700 1 Processor -- start 2024-05-08T05:42:33.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.213+0000 7f864ee00700 1 -- start start 2024-05-08T05:42:33.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.214+0000 7f864ee00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f864810b070 0x7f864810b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.214+0000 7f864ee00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f864810ba60 con 0x7f864810b070 2024-05-08T05:42:33.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.214+0000 7f8647e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f864810b070 0x7f864810b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.214+0000 7f8647e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f864810b070 0x7f864810b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39556/0 (socket says 172.21.15.112:39556) 2024-05-08T05:42:33.224 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.214+0000 7f8647e00700 1 -- 172.21.15.112:0/1258327849 learned_addr learned my addr 172.21.15.112:0/1258327849 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:33.224 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.214+0000 7f8647e00700 1 -- 172.21.15.112:0/1258327849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864810c270 con 0x7f864810b070 2024-05-08T05:42:33.224 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.215+0000 7f8647e00700 1 --2- 172.21.15.112:0/1258327849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f864810b070 0x7f864810b490 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f86380098a0 tx=0x7f8638009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bc2f30348ef1a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:33.224 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.215+0000 7f8646a00700 1 -- 172.21.15.112:0/1258327849 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8638004160 con 0x7f864810b070 2024-05-08T05:42:33.224 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.215+0000 7f8646a00700 1 -- 172.21.15.112:0/1258327849 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8638004300 con 0x7f864810b070 2024-05-08T05:42:33.224 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.215+0000 7f8646a00700 1 -- 172.21.15.112:0/1258327849 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8638011af0 con 0x7f864810b070 2024-05-08T05:42:33.224 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.216+0000 7f864ee00700 1 -- 172.21.15.112:0/1258327849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f864810b070 msgr2=0x7f864810b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.224 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.216+0000 7f864ee00700 1 --2- 172.21.15.112:0/1258327849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f864810b070 0x7f864810b490 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f86380098a0 tx=0x7f8638009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.224 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.216+0000 7f864ee00700 1 -- 172.21.15.112:0/1258327849 shutdown_connections 2024-05-08T05:42:33.224 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.216+0000 7f864ee00700 1 --2- 172.21.15.112:0/1258327849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f864810b070 0x7f864810b490 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.224 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.216+0000 7f864ee00700 1 -- 172.21.15.112:0/1258327849 >> 172.21.15.112:0/1258327849 conn(0x7f864807e440 msgr2=0x7f864807e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:33.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.216+0000 7f864ee00700 1 -- 172.21.15.112:0/1258327849 shutdown_connections 2024-05-08T05:42:33.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.216+0000 7f864ee00700 1 -- 172.21.15.112:0/1258327849 wait complete. 2024-05-08T05:42:33.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.217+0000 7f864ee00700 1 Processor -- start 2024-05-08T05:42:33.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.217+0000 7f864ee00700 1 -- start start 2024-05-08T05:42:33.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.217+0000 7f864ee00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f864810b070 0x7f864819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.217+0000 7f864ee00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f864810f620 con 0x7f864810b070 2024-05-08T05:42:33.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.218+0000 7f8647e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f864810b070 0x7f864819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.218+0000 7f8647e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f864810b070 0x7f864819f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39560/0 (socket says 172.21.15.112:39560) 2024-05-08T05:42:33.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.218+0000 7f8647e00700 1 -- 172.21.15.112:0/2999715093 learned_addr learned my addr 172.21.15.112:0/2999715093 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:33.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.218+0000 7f8647e00700 1 -- 172.21.15.112:0/2999715093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864819f560 con 0x7f864810b070 2024-05-08T05:42:33.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.218+0000 7f8647e00700 1 --2- 172.21.15.112:0/2999715093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f864810b070 0x7f864819f020 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f86380117b0 tx=0x7f8638011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:33.225 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.219+0000 7f8644c00700 1 -- 172.21.15.112:0/2999715093 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8638003af0 con 0x7f864810b070 2024-05-08T05:42:33.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.219+0000 7f8644c00700 1 -- 172.21.15.112:0/2999715093 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8638020070 con 0x7f864810b070 2024-05-08T05:42:33.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.219+0000 7f8644c00700 1 -- 172.21.15.112:0/2999715093 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8638019630 con 0x7f864810b070 2024-05-08T05:42:33.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.219+0000 7f864ee00700 1 -- 172.21.15.112:0/2999715093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864819f7f0 con 0x7f864810b070 2024-05-08T05:42:33.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.219+0000 7f864ee00700 1 -- 172.21.15.112:0/2999715093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86481a24e0 con 0x7f864810b070 2024-05-08T05:42:33.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.220+0000 7f864ee00700 1 -- 172.21.15.112:0/2999715093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8648064b00 con 0x7f864810b070 2024-05-08T05:42:33.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.220+0000 7f8644c00700 1 -- 172.21.15.112:0/2999715093 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f863801a9c0 con 0x7f864810b070 2024-05-08T05:42:33.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.221+0000 7f8644c00700 1 --2- 172.21.15.112:0/2999715093 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f862c03e4c0 0x7f862c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.221+0000 7f8644c00700 1 -- 172.21.15.112:0/2999715093 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8638053b80 con 0x7f864810b070 2024-05-08T05:42:33.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.221+0000 7f8647400700 1 --2- 172.21.15.112:0/2999715093 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f862c03e4c0 0x7f862c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.222+0000 7f8647400700 1 --2- 172.21.15.112:0/2999715093 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f862c03e4c0 0x7f862c040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8630006fd0 tx=0x7f8630006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:33.226 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.226+0000 7f8644c00700 1 -- 172.21.15.112:0/2999715093 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f86380172b0 con 0x7f864810b070 2024-05-08T05:42:33.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.474+0000 7f864ee00700 1 -- 172.21.15.112:0/2999715093 --> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f8648108bc0 con 0x7f862c03e4c0 2024-05-08T05:42:33.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.484+0000 7f8644c00700 1 -- 172.21.15.112:0/2999715093 <== mgr.14120 v2:172.21.15.112:6800/164150456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8648108bc0 con 0x7f862c03e4c0 2024-05-08T05:42:33.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.487+0000 7f864ee00700 1 -- 172.21.15.112:0/2999715093 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f862c03e4c0 msgr2=0x7f862c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.487+0000 7f864ee00700 1 --2- 172.21.15.112:0/2999715093 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f862c03e4c0 0x7f862c040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8630006fd0 tx=0x7f8630006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:33.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.487+0000 7f864ee00700 1 -- 172.21.15.112:0/2999715093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f864810b070 msgr2=0x7f864819f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.487+0000 7f864ee00700 1 --2- 172.21.15.112:0/2999715093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f864810b070 0x7f864819f020 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f86380117b0 tx=0x7f8638011b70 comp rx=0 tx=0).stop 2024-05-08T05:42:33.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.487+0000 7f864ee00700 1 -- 172.21.15.112:0/2999715093 shutdown_connections 2024-05-08T05:42:33.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.487+0000 7f864ee00700 1 --2- 172.21.15.112:0/2999715093 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f862c03e4c0 0x7f862c040980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.487+0000 7f864ee00700 1 --2- 172.21.15.112:0/2999715093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f864810b070 0x7f864819f020 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.487+0000 7f864ee00700 1 -- 172.21.15.112:0/2999715093 >> 172.21.15.112:0/2999715093 conn(0x7f864807e440 msgr2=0x7f86481084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:33.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.487+0000 7f864ee00700 1 -- 172.21.15.112:0/2999715093 shutdown_connections 2024-05-08T05:42:33.227 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:32.488+0000 7f864ee00700 1 -- 172.21.15.112:0/2999715093 wait complete. 2024-05-08T05:42:34.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-08T05:42:34.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.819+0000 7f6ad5600700 1 Processor -- start 2024-05-08T05:42:34.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.819+0000 7f6ad5600700 1 -- start start 2024-05-08T05:42:34.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.819+0000 7f6ad5600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad0107d20 0x7f6ad0108140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:34.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.819+0000 7f6ad5600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad0108710 con 0x7f6ad0107d20 2024-05-08T05:42:34.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.820+0000 7f6acea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad0107d20 0x7f6ad0108140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:34.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.820+0000 7f6acea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad0107d20 0x7f6ad0108140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33770/0 (socket says 172.21.15.112:33770) 2024-05-08T05:42:34.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.820+0000 7f6acea00700 1 -- 172.21.15.112:0/2263665295 learned_addr learned my addr 172.21.15.112:0/2263665295 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:34.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.821+0000 7f6acea00700 1 -- 172.21.15.112:0/2263665295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad0108f20 con 0x7f6ad0107d20 2024-05-08T05:42:34.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.822+0000 7f6acea00700 1 --2- 172.21.15.112:0/2263665295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad0107d20 0x7f6ad0108140 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6ac00098a0 tx=0x7f6ac0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e41f3b91eecb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:34.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.822+0000 7f6acd600700 1 -- 172.21.15.112:0/2263665295 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac0004160 con 0x7f6ad0107d20 2024-05-08T05:42:34.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.822+0000 7f6acd600700 1 -- 172.21.15.112:0/2263665295 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6ac0004300 con 0x7f6ad0107d20 2024-05-08T05:42:34.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.822+0000 7f6acd600700 1 -- 172.21.15.112:0/2263665295 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac0011af0 con 0x7f6ad0107d20 2024-05-08T05:42:34.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.823+0000 7f6ad5600700 1 -- 172.21.15.112:0/2263665295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad0107d20 msgr2=0x7f6ad0108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:34.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.823+0000 7f6ad5600700 1 --2- 172.21.15.112:0/2263665295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad0107d20 0x7f6ad0108140 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6ac00098a0 tx=0x7f6ac0009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:34.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.823+0000 7f6ad5600700 1 -- 172.21.15.112:0/2263665295 shutdown_connections 2024-05-08T05:42:34.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.823+0000 7f6ad5600700 1 --2- 172.21.15.112:0/2263665295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad0107d20 0x7f6ad0108140 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:34.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.823+0000 7f6ad5600700 1 -- 172.21.15.112:0/2263665295 >> 172.21.15.112:0/2263665295 conn(0x7f6ad01032c0 msgr2=0x7f6ad0105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:34.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.823+0000 7f6ad5600700 1 -- 172.21.15.112:0/2263665295 shutdown_connections 2024-05-08T05:42:34.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.823+0000 7f6ad5600700 1 -- 172.21.15.112:0/2263665295 wait complete. 2024-05-08T05:42:34.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.824+0000 7f6ad5600700 1 Processor -- start 2024-05-08T05:42:34.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.824+0000 7f6ad5600700 1 -- start start 2024-05-08T05:42:34.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.825+0000 7f6ad5600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad0192e50 0x7f6ad01915c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:34.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.825+0000 7f6ad5600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad010c2d0 con 0x7f6ad0192e50 2024-05-08T05:42:34.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.825+0000 7f6acea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad0192e50 0x7f6ad01915c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:34.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.825+0000 7f6acea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad0192e50 0x7f6ad01915c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33780/0 (socket says 172.21.15.112:33780) 2024-05-08T05:42:34.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.825+0000 7f6acea00700 1 -- 172.21.15.112:0/603341292 learned_addr learned my addr 172.21.15.112:0/603341292 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:34.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.825+0000 7f6acea00700 1 -- 172.21.15.112:0/603341292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad0193270 con 0x7f6ad0192e50 2024-05-08T05:42:34.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.826+0000 7f6acea00700 1 --2- 172.21.15.112:0/603341292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad0192e50 0x7f6ad01915c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0108d80 tx=0x7f6ac0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:34.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.826+0000 7f6ac7400700 1 -- 172.21.15.112:0/603341292 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac00036a0 con 0x7f6ad0192e50 2024-05-08T05:42:34.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.826+0000 7f6ac7400700 1 -- 172.21.15.112:0/603341292 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6ac001a460 con 0x7f6ad0192e50 2024-05-08T05:42:34.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.826+0000 7f6ac7400700 1 -- 172.21.15.112:0/603341292 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac001b5d0 con 0x7f6ad0192e50 2024-05-08T05:42:34.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.826+0000 7f6ad5600700 1 -- 172.21.15.112:0/603341292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad0191b00 con 0x7f6ad0192e50 2024-05-08T05:42:34.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.826+0000 7f6ad5600700 1 -- 172.21.15.112:0/603341292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad0191fe0 con 0x7f6ad0192e50 2024-05-08T05:42:34.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.827+0000 7f6ad5600700 1 -- 172.21.15.112:0/603341292 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad0064b00 con 0x7f6ad0192e50 2024-05-08T05:42:34.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.827+0000 7f6ac7400700 1 -- 172.21.15.112:0/603341292 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f6ac001a610 con 0x7f6ad0192e50 2024-05-08T05:42:34.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.828+0000 7f6ac7400700 1 --2- 172.21.15.112:0/603341292 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f6ab403e4c0 0x7f6ab4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:34.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.828+0000 7f6ac7400700 1 -- 172.21.15.112:0/603341292 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6ac0053c00 con 0x7f6ad0192e50 2024-05-08T05:42:34.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.828+0000 7f6ace000700 1 --2- 172.21.15.112:0/603341292 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f6ab403e4c0 0x7f6ab4040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:34.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.829+0000 7f6ace000700 1 --2- 172.21.15.112:0/603341292 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f6ab403e4c0 0x7f6ab4040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6abc006fd0 tx=0x7f6abc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:34.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.834+0000 7f6ac7400700 1 -- 172.21.15.112:0/603341292 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ac000f2e0 con 0x7f6ad0192e50 2024-05-08T05:42:34.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.065+0000 7f6ad5600700 1 -- 172.21.15.112:0/603341292 --> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f6ad00638e0 con 0x7f6ab403e4c0 2024-05-08T05:42:34.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.066+0000 7f6ac7400700 1 -- 172.21.15.112:0/603341292 <== mgr.14120 v2:172.21.15.112:6800/164150456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f6ad00638e0 con 0x7f6ab403e4c0 2024-05-08T05:42:34.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.069+0000 7f6ad5600700 1 -- 172.21.15.112:0/603341292 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f6ab403e4c0 msgr2=0x7f6ab4040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:34.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.069+0000 7f6ad5600700 1 --2- 172.21.15.112:0/603341292 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f6ab403e4c0 0x7f6ab4040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6abc006fd0 tx=0x7f6abc006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:34.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.069+0000 7f6ad5600700 1 -- 172.21.15.112:0/603341292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad0192e50 msgr2=0x7f6ad01915c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:34.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.069+0000 7f6ad5600700 1 --2- 172.21.15.112:0/603341292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad0192e50 0x7f6ad01915c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0108d80 tx=0x7f6ac0011940 comp rx=0 tx=0).stop 2024-05-08T05:42:34.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.069+0000 7f6ad5600700 1 -- 172.21.15.112:0/603341292 shutdown_connections 2024-05-08T05:42:34.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.069+0000 7f6ad5600700 1 --2- 172.21.15.112:0/603341292 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f6ab403e4c0 0x7f6ab4040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:34.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.069+0000 7f6ad5600700 1 --2- 172.21.15.112:0/603341292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad0192e50 0x7f6ad01915c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:34.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.069+0000 7f6ad5600700 1 -- 172.21.15.112:0/603341292 >> 172.21.15.112:0/603341292 conn(0x7f6ad01032c0 msgr2=0x7f6ad01042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:34.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.070+0000 7f6ad5600700 1 -- 172.21.15.112:0/603341292 shutdown_connections 2024-05-08T05:42:34.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.070+0000 7f6ad5600700 1 -- 172.21.15.112:0/603341292 wait complete. 2024-05-08T05:42:34.849 INFO:teuthology.orchestra.run.smithi112.stdout:Generating ssh key... 2024-05-08T05:42:35.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:34 smithi112 ceph-mon[30003]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:36.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:35 smithi112 ceph-mon[30003]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:36.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:35 smithi112 ceph-mon[30003]: Generating ssh key... 2024-05-08T05:42:36.316 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: Generating public/private rsa key pair. 2024-05-08T05:42:36.317 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: Your identification has been saved in /tmp/tmpp1yty6dx/key. 2024-05-08T05:42:36.317 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: Your public key has been saved in /tmp/tmpp1yty6dx/key.pub. 2024-05-08T05:42:36.317 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: The key fingerprint is: 2024-05-08T05:42:36.317 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: SHA256:mQZuOwPt0ZrnAgh/RoH6SAIn8tQ1b1Ok3AVzxmmW2LY ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:42:36.317 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: The key's randomart image is: 2024-05-08T05:42:36.317 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: +---[RSA 3072]----+ 2024-05-08T05:42:36.317 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: | o .o .==+o | 2024-05-08T05:42:36.317 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: |+ + o + +.=O | 2024-05-08T05:42:36.317 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: |o* .. * .+ . | 2024-05-08T05:42:36.317 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: |+.. .o + + E | 2024-05-08T05:42:36.317 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: |o= o. = S | 2024-05-08T05:42:36.317 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: |. + ++ * | 2024-05-08T05:42:36.317 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: | o .B . | 2024-05-08T05:42:36.317 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: | .= | 2024-05-08T05:42:36.317 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: | .. | 2024-05-08T05:42:36.318 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: +----[SHA256]-----+ 2024-05-08T05:42:36.791 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.459+0000 7f6855600700 1 Processor -- start 2024-05-08T05:42:36.791 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.459+0000 7f6855600700 1 -- start start 2024-05-08T05:42:36.791 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.460+0000 7f6855600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685007ee40 0x7f685007d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:36.791 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.460+0000 7f6855600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f685007d870 con 0x7f685007ee40 2024-05-08T05:42:36.791 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.460+0000 7f684ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685007ee40 0x7f685007d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:36.791 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.460+0000 7f684ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685007ee40 0x7f685007d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33790/0 (socket says 172.21.15.112:33790) 2024-05-08T05:42:36.791 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.460+0000 7f684ea00700 1 -- 172.21.15.112:0/1295378017 learned_addr learned my addr 172.21.15.112:0/1295378017 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:36.791 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.461+0000 7f684ea00700 1 -- 172.21.15.112:0/1295378017 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685007d9f0 con 0x7f685007ee40 2024-05-08T05:42:36.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.461+0000 7f684ea00700 1 --2- 172.21.15.112:0/1295378017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685007ee40 0x7f685007d2a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f68400098a0 tx=0x7f6840009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984ed996602550c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:36.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.461+0000 7f684d600700 1 -- 172.21.15.112:0/1295378017 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6840004160 con 0x7f685007ee40 2024-05-08T05:42:36.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.461+0000 7f684d600700 1 -- 172.21.15.112:0/1295378017 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6840004300 con 0x7f685007ee40 2024-05-08T05:42:36.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.461+0000 7f684d600700 1 -- 172.21.15.112:0/1295378017 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6840011af0 con 0x7f685007ee40 2024-05-08T05:42:36.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.462+0000 7f6855600700 1 -- 172.21.15.112:0/1295378017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685007ee40 msgr2=0x7f685007d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:36.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.462+0000 7f6855600700 1 --2- 172.21.15.112:0/1295378017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685007ee40 0x7f685007d2a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f68400098a0 tx=0x7f6840009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:36.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.462+0000 7f6855600700 1 -- 172.21.15.112:0/1295378017 shutdown_connections 2024-05-08T05:42:36.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.462+0000 7f6855600700 1 --2- 172.21.15.112:0/1295378017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685007ee40 0x7f685007d2a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:36.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.462+0000 7f6855600700 1 -- 172.21.15.112:0/1295378017 >> 172.21.15.112:0/1295378017 conn(0x7f68501043f0 msgr2=0x7f6850106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:36.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.462+0000 7f6855600700 1 -- 172.21.15.112:0/1295378017 shutdown_connections 2024-05-08T05:42:36.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.462+0000 7f6855600700 1 -- 172.21.15.112:0/1295378017 wait complete. 2024-05-08T05:42:36.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.463+0000 7f6855600700 1 Processor -- start 2024-05-08T05:42:36.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.463+0000 7f6855600700 1 -- start start 2024-05-08T05:42:36.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.464+0000 7f6855600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685007ee40 0x7f685019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:36.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.464+0000 7f6855600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f685010f3c0 con 0x7f685007ee40 2024-05-08T05:42:36.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.464+0000 7f684ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685007ee40 0x7f685019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:36.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.464+0000 7f684ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685007ee40 0x7f685019efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33798/0 (socket says 172.21.15.112:33798) 2024-05-08T05:42:36.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.464+0000 7f684ea00700 1 -- 172.21.15.112:0/689715317 learned_addr learned my addr 172.21.15.112:0/689715317 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:36.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.464+0000 7f684ea00700 1 -- 172.21.15.112:0/689715317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685019f510 con 0x7f685007ee40 2024-05-08T05:42:36.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.465+0000 7f684ea00700 1 --2- 172.21.15.112:0/689715317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685007ee40 0x7f685019efd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6840000c00 tx=0x7f6840011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:36.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.465+0000 7f6847400700 1 -- 172.21.15.112:0/689715317 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6840003b20 con 0x7f685007ee40 2024-05-08T05:42:36.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.465+0000 7f6847400700 1 -- 172.21.15.112:0/689715317 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f684001a8c0 con 0x7f685007ee40 2024-05-08T05:42:36.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.465+0000 7f6855600700 1 -- 172.21.15.112:0/689715317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685019f7a0 con 0x7f685007ee40 2024-05-08T05:42:36.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.465+0000 7f6847400700 1 -- 172.21.15.112:0/689715317 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6840024940 con 0x7f685007ee40 2024-05-08T05:42:36.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.465+0000 7f6855600700 1 -- 172.21.15.112:0/689715317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68501a2490 con 0x7f685007ee40 2024-05-08T05:42:36.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.466+0000 7f6855600700 1 -- 172.21.15.112:0/689715317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6850064b00 con 0x7f685007ee40 2024-05-08T05:42:36.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.466+0000 7f6847400700 1 -- 172.21.15.112:0/689715317 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f6840022020 con 0x7f685007ee40 2024-05-08T05:42:36.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.467+0000 7f6847400700 1 --2- 172.21.15.112:0/689715317 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f683403e4c0 0x7f6834040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:36.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.467+0000 7f6847400700 1 -- 172.21.15.112:0/689715317 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6840053740 con 0x7f685007ee40 2024-05-08T05:42:36.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.467+0000 7f684e000700 1 --2- 172.21.15.112:0/689715317 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f683403e4c0 0x7f6834040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:36.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.468+0000 7f684e000700 1 --2- 172.21.15.112:0/689715317 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f683403e4c0 0x7f6834040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f683c006fd0 tx=0x7f683c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:36.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.472+0000 7f6847400700 1 -- 172.21.15.112:0/689715317 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f684001aa70 con 0x7f685007ee40 2024-05-08T05:42:36.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.714+0000 7f6855600700 1 -- 172.21.15.112:0/689715317 --> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f6850105660 con 0x7f683403e4c0 2024-05-08T05:42:36.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.025+0000 7f6847400700 1 -- 172.21.15.112:0/689715317 <== mgr.14120 v2:172.21.15.112:6800/164150456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6850105660 con 0x7f683403e4c0 2024-05-08T05:42:36.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.030+0000 7f6855600700 1 -- 172.21.15.112:0/689715317 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f683403e4c0 msgr2=0x7f6834040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:36.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.030+0000 7f6855600700 1 --2- 172.21.15.112:0/689715317 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f683403e4c0 0x7f6834040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f683c006fd0 tx=0x7f683c006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:36.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.030+0000 7f6855600700 1 -- 172.21.15.112:0/689715317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685007ee40 msgr2=0x7f685019efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:36.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.030+0000 7f6855600700 1 --2- 172.21.15.112:0/689715317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685007ee40 0x7f685019efd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6840000c00 tx=0x7f6840011b70 comp rx=0 tx=0).stop 2024-05-08T05:42:36.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.030+0000 7f6855600700 1 -- 172.21.15.112:0/689715317 shutdown_connections 2024-05-08T05:42:36.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.030+0000 7f6855600700 1 --2- 172.21.15.112:0/689715317 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f683403e4c0 0x7f6834040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:36.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.030+0000 7f6855600700 1 --2- 172.21.15.112:0/689715317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f685007ee40 0x7f685019efd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:36.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.030+0000 7f6855600700 1 -- 172.21.15.112:0/689715317 >> 172.21.15.112:0/689715317 conn(0x7f68501043f0 msgr2=0x7f6850104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:36.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.030+0000 7f6855600700 1 -- 172.21.15.112:0/689715317 shutdown_connections 2024-05-08T05:42:36.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.030+0000 7f6855600700 1 -- 172.21.15.112:0/689715317 wait complete. 2024-05-08T05:42:37.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:37 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' 2024-05-08T05:42:37.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:37 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' 2024-05-08T05:42:38.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:38 smithi112 ceph-mon[30003]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:38.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWR6ZAu8e+ffNgTQdsp2CLBJsPd4Ycek0h0c4u/yRD4ZCZW1INIMUSw6NWygDqH2wC+MKVO0HtrBnX37KinGtkpi5506m+uvRRPbr97eps4BmXM9AgTXofBx45PGseaZwiEmwWQ9aXJm4+s+1GakK9KIL43K5mPAS5gnVfa1Q0QqyA2O+iVU1qEIKxrbiJ0zrQDYKw8y0zyvKiJa8oalj+4a12+ySeKyTIsDf6FXZRnwrGB2aOmvX34KC4DPXhmByKTzgRrPTG8igsvIfmc73ourCe13IvI+92DoHXMvfYlbWyHm9ydyTBSkF0TQJ6g2VUVRbbQhTbO7mP8jIQb35wXH5nyTWRPZzpMZsfuyVoEgQTXS7DsLmXIQ1SyWo9TnUjY8eK5RoyA/E7cKt6aFoK3pWD5bLzTPMM29pMC/S+6HcDWOmnrhl0cmsWAAYrKkWRR+V6qsElBjg6ZAzGss66P1bHs+fN0aw6Q1ZRZqurUqzWWaNNyqXN1Kw978oYht8= ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:42:38.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.433+0000 7f7db9200700 1 Processor -- start 2024-05-08T05:42:38.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.433+0000 7f7db9200700 1 -- start start 2024-05-08T05:42:38.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.434+0000 7f7db9200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db4107730 0x7f7db4107b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:38.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.434+0000 7f7db9200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7db407cdd0 con 0x7f7db4107730 2024-05-08T05:42:38.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.434+0000 7f7db1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db4107730 0x7f7db4107b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:38.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.434+0000 7f7db1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db4107730 0x7f7db4107b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33800/0 (socket says 172.21.15.112:33800) 2024-05-08T05:42:38.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.434+0000 7f7db1e00700 1 -- 172.21.15.112:0/3281647104 learned_addr learned my addr 172.21.15.112:0/3281647104 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:38.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.435+0000 7f7db1e00700 1 -- 172.21.15.112:0/3281647104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db4108090 con 0x7f7db4107730 2024-05-08T05:42:38.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.435+0000 7f7db1e00700 1 --2- 172.21.15.112:0/3281647104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db4107730 0x7f7db4107b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7da40098a0 tx=0x7f7da4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8e441079f68859 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:38.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.435+0000 7f7db0a00700 1 -- 172.21.15.112:0/3281647104 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7da4004160 con 0x7f7db4107730 2024-05-08T05:42:38.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.435+0000 7f7db0a00700 1 -- 172.21.15.112:0/3281647104 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7da4004300 con 0x7f7db4107730 2024-05-08T05:42:38.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.435+0000 7f7db0a00700 1 -- 172.21.15.112:0/3281647104 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7da4011af0 con 0x7f7db4107730 2024-05-08T05:42:38.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.436+0000 7f7db9200700 1 -- 172.21.15.112:0/3281647104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db4107730 msgr2=0x7f7db4107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:38.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.436+0000 7f7db9200700 1 --2- 172.21.15.112:0/3281647104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db4107730 0x7f7db4107b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7da40098a0 tx=0x7f7da4009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:38.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.436+0000 7f7db9200700 1 -- 172.21.15.112:0/3281647104 shutdown_connections 2024-05-08T05:42:38.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.436+0000 7f7db9200700 1 --2- 172.21.15.112:0/3281647104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db4107730 0x7f7db4107b50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:38.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.436+0000 7f7db9200700 1 -- 172.21.15.112:0/3281647104 >> 172.21.15.112:0/3281647104 conn(0x7f7db4103310 msgr2=0x7f7db4105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:38.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.436+0000 7f7db9200700 1 -- 172.21.15.112:0/3281647104 shutdown_connections 2024-05-08T05:42:38.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.436+0000 7f7db9200700 1 -- 172.21.15.112:0/3281647104 wait complete. 2024-05-08T05:42:38.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.437+0000 7f7db9200700 1 Processor -- start 2024-05-08T05:42:38.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.438+0000 7f7db9200700 1 -- start start 2024-05-08T05:42:38.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.438+0000 7f7db9200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db4107730 0x7f7db419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:38.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.438+0000 7f7db9200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7db41090c0 con 0x7f7db4107730 2024-05-08T05:42:38.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.438+0000 7f7db1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db4107730 0x7f7db419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:38.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.438+0000 7f7db1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db4107730 0x7f7db419efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33802/0 (socket says 172.21.15.112:33802) 2024-05-08T05:42:38.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.438+0000 7f7db1e00700 1 -- 172.21.15.112:0/30575620 learned_addr learned my addr 172.21.15.112:0/30575620 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:38.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.439+0000 7f7db1e00700 1 -- 172.21.15.112:0/30575620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db419f520 con 0x7f7db4107730 2024-05-08T05:42:38.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.439+0000 7f7db1e00700 1 --2- 172.21.15.112:0/30575620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db4107730 0x7f7db419efe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7da4011ac0 tx=0x7f7da4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:38.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.439+0000 7f7daaa00700 1 -- 172.21.15.112:0/30575620 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7da4003970 con 0x7f7db4107730 2024-05-08T05:42:38.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.439+0000 7f7daaa00700 1 -- 172.21.15.112:0/30575620 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7da4003b10 con 0x7f7db4107730 2024-05-08T05:42:38.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.439+0000 7f7daaa00700 1 -- 172.21.15.112:0/30575620 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7da4024610 con 0x7f7db4107730 2024-05-08T05:42:38.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.439+0000 7f7db9200700 1 -- 172.21.15.112:0/30575620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db419f7b0 con 0x7f7db4107730 2024-05-08T05:42:38.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.440+0000 7f7db9200700 1 -- 172.21.15.112:0/30575620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db41a24a0 con 0x7f7db4107730 2024-05-08T05:42:38.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.441+0000 7f7db9200700 1 -- 172.21.15.112:0/30575620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db4064b00 con 0x7f7db4107730 2024-05-08T05:42:38.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.441+0000 7f7daaa00700 1 -- 172.21.15.112:0/30575620 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7da401a9e0 con 0x7f7db4107730 2024-05-08T05:42:38.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.441+0000 7f7daaa00700 1 --2- 172.21.15.112:0/30575620 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f7d9803e4c0 0x7f7d98040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:38.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.441+0000 7f7daaa00700 1 -- 172.21.15.112:0/30575620 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7da4053910 con 0x7f7db4107730 2024-05-08T05:42:38.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.442+0000 7f7db1400700 1 --2- 172.21.15.112:0/30575620 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f7d9803e4c0 0x7f7d98040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:38.348 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.442+0000 7f7db1400700 1 --2- 172.21.15.112:0/30575620 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f7d9803e4c0 0x7f7d98040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7da0006fd0 tx=0x7f7da0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:38.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.447+0000 7f7daaa00700 1 -- 172.21.15.112:0/30575620 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7da4053260 con 0x7f7db4107730 2024-05-08T05:42:38.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.682+0000 7f7db9200700 1 -- 172.21.15.112:0/30575620 --> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f7db40008d0 con 0x7f7d9803e4c0 2024-05-08T05:42:38.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.684+0000 7f7daaa00700 1 -- 172.21.15.112:0/30575620 <== mgr.14120 v2:172.21.15.112:6800/164150456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f7db40008d0 con 0x7f7d9803e4c0 2024-05-08T05:42:38.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.688+0000 7f7db9200700 1 -- 172.21.15.112:0/30575620 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f7d9803e4c0 msgr2=0x7f7d98040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:38.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.688+0000 7f7db9200700 1 --2- 172.21.15.112:0/30575620 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f7d9803e4c0 0x7f7d98040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7da0006fd0 tx=0x7f7da0006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:38.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.688+0000 7f7db9200700 1 -- 172.21.15.112:0/30575620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db4107730 msgr2=0x7f7db419efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:38.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.688+0000 7f7db9200700 1 --2- 172.21.15.112:0/30575620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db4107730 0x7f7db419efe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7da4011ac0 tx=0x7f7da4011ba0 comp rx=0 tx=0).stop 2024-05-08T05:42:38.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.688+0000 7f7db9200700 1 -- 172.21.15.112:0/30575620 shutdown_connections 2024-05-08T05:42:38.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.688+0000 7f7db9200700 1 --2- 172.21.15.112:0/30575620 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f7d9803e4c0 0x7f7d98040980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:38.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.688+0000 7f7db9200700 1 --2- 172.21.15.112:0/30575620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7db4107730 0x7f7db419efe0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:38.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.688+0000 7f7db9200700 1 -- 172.21.15.112:0/30575620 >> 172.21.15.112:0/30575620 conn(0x7f7db4103310 msgr2=0x7f7db4195ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:38.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.688+0000 7f7db9200700 1 -- 172.21.15.112:0/30575620 shutdown_connections 2024-05-08T05:42:38.349 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.688+0000 7f7db9200700 1 -- 172.21.15.112:0/30575620 wait complete. 2024-05-08T05:42:38.349 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-08T05:42:38.350 INFO:teuthology.orchestra.run.smithi112.stdout:Adding key to root@localhost authorized_keys... 2024-05-08T05:42:38.350 INFO:teuthology.orchestra.run.smithi112.stdout:Adding host smithi112... 2024-05-08T05:42:40.162 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:39 smithi112 ceph-mon[30003]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi112", "addr": "172.21.15.112", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:41.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:40 smithi112 ceph-mon[30003]: Deploying cephadm binary to smithi112 2024-05-08T05:42:45.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Added host 'smithi112' with addr '172.21.15.112' 2024-05-08T05:42:45.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.958+0000 7f3591a00700 1 Processor -- start 2024-05-08T05:42:45.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.959+0000 7f3591a00700 1 -- start start 2024-05-08T05:42:45.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.959+0000 7f3591a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f358c10b070 0x7f358c10b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.959+0000 7f3591a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f358c10ba60 con 0x7f358c10b070 2024-05-08T05:42:45.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.959+0000 7f358aa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f358c10b070 0x7f358c10b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.959+0000 7f358aa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f358c10b070 0x7f358c10b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33812/0 (socket says 172.21.15.112:33812) 2024-05-08T05:42:45.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.959+0000 7f358aa00700 1 -- 172.21.15.112:0/3785257948 learned_addr learned my addr 172.21.15.112:0/3785257948 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:45.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.960+0000 7f358aa00700 1 -- 172.21.15.112:0/3785257948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358c10c270 con 0x7f358c10b070 2024-05-08T05:42:45.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.960+0000 7f358aa00700 1 --2- 172.21.15.112:0/3785257948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f358c10b070 0x7f358c10b490 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f357c0098a0 tx=0x7f357c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f7a84f48bad3e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:45.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.961+0000 7f3589600700 1 -- 172.21.15.112:0/3785257948 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f357c004160 con 0x7f358c10b070 2024-05-08T05:42:45.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.961+0000 7f3589600700 1 -- 172.21.15.112:0/3785257948 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f357c004300 con 0x7f358c10b070 2024-05-08T05:42:45.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.961+0000 7f3589600700 1 -- 172.21.15.112:0/3785257948 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f357c011af0 con 0x7f358c10b070 2024-05-08T05:42:45.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.961+0000 7f3591a00700 1 -- 172.21.15.112:0/3785257948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f358c10b070 msgr2=0x7f358c10b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.961+0000 7f3591a00700 1 --2- 172.21.15.112:0/3785257948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f358c10b070 0x7f358c10b490 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f357c0098a0 tx=0x7f357c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.961+0000 7f3591a00700 1 -- 172.21.15.112:0/3785257948 shutdown_connections 2024-05-08T05:42:45.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.961+0000 7f3591a00700 1 --2- 172.21.15.112:0/3785257948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f358c10b070 0x7f358c10b490 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.961+0000 7f3591a00700 1 -- 172.21.15.112:0/3785257948 >> 172.21.15.112:0/3785257948 conn(0x7f358c07e440 msgr2=0x7f358c07e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:45.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.961+0000 7f3591a00700 1 -- 172.21.15.112:0/3785257948 shutdown_connections 2024-05-08T05:42:45.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.961+0000 7f3591a00700 1 -- 172.21.15.112:0/3785257948 wait complete. 2024-05-08T05:42:45.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.962+0000 7f3591a00700 1 Processor -- start 2024-05-08T05:42:45.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.963+0000 7f3591a00700 1 -- start start 2024-05-08T05:42:45.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.963+0000 7f3591a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f358c10b070 0x7f358c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.963+0000 7f3591a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f358c10f620 con 0x7f358c10b070 2024-05-08T05:42:45.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.963+0000 7f358aa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f358c10b070 0x7f358c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.963+0000 7f358aa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f358c10b070 0x7f358c19f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33820/0 (socket says 172.21.15.112:33820) 2024-05-08T05:42:45.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.963+0000 7f358aa00700 1 -- 172.21.15.112:0/2411697943 learned_addr learned my addr 172.21.15.112:0/2411697943 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:45.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.964+0000 7f358aa00700 1 -- 172.21.15.112:0/2411697943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358c19f560 con 0x7f358c10b070 2024-05-08T05:42:45.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.964+0000 7f358aa00700 1 --2- 172.21.15.112:0/2411697943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f358c10b070 0x7f358c19f020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f357c0117b0 tx=0x7f357c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:45.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.964+0000 7f3583400700 1 -- 172.21.15.112:0/2411697943 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f357c003af0 con 0x7f358c10b070 2024-05-08T05:42:45.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.964+0000 7f3583400700 1 -- 172.21.15.112:0/2411697943 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f357c020070 con 0x7f358c10b070 2024-05-08T05:42:45.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.964+0000 7f3583400700 1 -- 172.21.15.112:0/2411697943 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f357c019630 con 0x7f358c10b070 2024-05-08T05:42:45.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.964+0000 7f3591a00700 1 -- 172.21.15.112:0/2411697943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358c19f7f0 con 0x7f358c10b070 2024-05-08T05:42:45.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.964+0000 7f3591a00700 1 -- 172.21.15.112:0/2411697943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358c1a24e0 con 0x7f358c10b070 2024-05-08T05:42:45.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.966+0000 7f3591a00700 1 -- 172.21.15.112:0/2411697943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f358c064b00 con 0x7f358c10b070 2024-05-08T05:42:45.847 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.966+0000 7f3583400700 1 -- 172.21.15.112:0/2411697943 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f357c01a9c0 con 0x7f358c10b070 2024-05-08T05:42:45.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.966+0000 7f3583400700 1 --2- 172.21.15.112:0/2411697943 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f357003e4c0 0x7f3570040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.966+0000 7f3583400700 1 -- 172.21.15.112:0/2411697943 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f357c053ae0 con 0x7f358c10b070 2024-05-08T05:42:45.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.967+0000 7f358a000700 1 --2- 172.21.15.112:0/2411697943 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f357003e4c0 0x7f3570040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.967+0000 7f358a000700 1 --2- 172.21.15.112:0/2411697943 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f357003e4c0 0x7f3570040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3578006fd0 tx=0x7f3578006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:45.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.972+0000 7f3583400700 1 -- 172.21.15.112:0/2411697943 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f357c053470 con 0x7f358c10b070 2024-05-08T05:42:45.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:39.209+0000 7f3591a00700 1 -- 172.21.15.112:0/2411697943 --> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi112", "addr": "172.21.15.112", "target": ["mon-mgr", ""]}) -- 0x7f358c108c00 con 0x7f357003e4c0 2024-05-08T05:42:45.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:45.148+0000 7f3583400700 1 -- 172.21.15.112:0/2411697943 <== mgr.14120 v2:172.21.15.112:6800/164150456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f358c108c00 con 0x7f357003e4c0 2024-05-08T05:42:45.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:45.151+0000 7f3591a00700 1 -- 172.21.15.112:0/2411697943 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f357003e4c0 msgr2=0x7f3570040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:45.151+0000 7f3591a00700 1 --2- 172.21.15.112:0/2411697943 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f357003e4c0 0x7f3570040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3578006fd0 tx=0x7f3578006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:45.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:45.151+0000 7f3591a00700 1 -- 172.21.15.112:0/2411697943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f358c10b070 msgr2=0x7f358c19f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:45.151+0000 7f3591a00700 1 --2- 172.21.15.112:0/2411697943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f358c10b070 0x7f358c19f020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f357c0117b0 tx=0x7f357c011b70 comp rx=0 tx=0).stop 2024-05-08T05:42:45.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:45.151+0000 7f3591a00700 1 -- 172.21.15.112:0/2411697943 shutdown_connections 2024-05-08T05:42:45.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:45.151+0000 7f3591a00700 1 --2- 172.21.15.112:0/2411697943 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f357003e4c0 0x7f3570040980 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:45.151+0000 7f3591a00700 1 --2- 172.21.15.112:0/2411697943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f358c10b070 0x7f358c19f020 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:45.151+0000 7f3591a00700 1 -- 172.21.15.112:0/2411697943 >> 172.21.15.112:0/2411697943 conn(0x7f358c07e440 msgr2=0x7f358c1084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:45.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:45.152+0000 7f3591a00700 1 -- 172.21.15.112:0/2411697943 shutdown_connections 2024-05-08T05:42:45.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:45.152+0000 7f3591a00700 1 -- 172.21.15.112:0/2411697943 wait complete. 2024-05-08T05:42:45.849 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying unmanaged mon service... 2024-05-08T05:42:46.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:46 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' 2024-05-08T05:42:46.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:46 smithi112 ceph-mon[30003]: Added host smithi112 2024-05-08T05:42:46.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:46 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:42:47.918 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:47 smithi112 ceph-mon[30003]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:47.918 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:47 smithi112 ceph-mon[30003]: Saving service mon spec with placement count:5 2024-05-08T05:42:47.918 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:47 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' 2024-05-08T05:42:47.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-08T05:42:47.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.526+0000 7f0e2a200700 1 Processor -- start 2024-05-08T05:42:47.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.526+0000 7f0e2a200700 1 -- start start 2024-05-08T05:42:47.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.527+0000 7f0e2a200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2410b050 0x7f0e2410b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.527+0000 7f0e2a200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2410ba40 con 0x7f0e2410b050 2024-05-08T05:42:47.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.527+0000 7f0e23400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2410b050 0x7f0e2410b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.527+0000 7f0e23400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2410b050 0x7f0e2410b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49538/0 (socket says 172.21.15.112:49538) 2024-05-08T05:42:47.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.527+0000 7f0e23400700 1 -- 172.21.15.112:0/1441449946 learned_addr learned my addr 172.21.15.112:0/1441449946 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:47.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.528+0000 7f0e23400700 1 -- 172.21.15.112:0/1441449946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2410c250 con 0x7f0e2410b050 2024-05-08T05:42:47.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.528+0000 7f0e23400700 1 --2- 172.21.15.112:0/1441449946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2410b050 0x7f0e2410b470 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e140098a0 tx=0x7f0e14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62551cbb7a4fedcf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:47.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.528+0000 7f0e22000700 1 -- 172.21.15.112:0/1441449946 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e14004160 con 0x7f0e2410b050 2024-05-08T05:42:47.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.529+0000 7f0e22000700 1 -- 172.21.15.112:0/1441449946 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0e14004300 con 0x7f0e2410b050 2024-05-08T05:42:47.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.529+0000 7f0e22000700 1 -- 172.21.15.112:0/1441449946 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e14011af0 con 0x7f0e2410b050 2024-05-08T05:42:47.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.529+0000 7f0e2a200700 1 -- 172.21.15.112:0/1441449946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2410b050 msgr2=0x7f0e2410b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:47.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.529+0000 7f0e2a200700 1 --2- 172.21.15.112:0/1441449946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2410b050 0x7f0e2410b470 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e140098a0 tx=0x7f0e14009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.529+0000 7f0e2a200700 1 -- 172.21.15.112:0/1441449946 shutdown_connections 2024-05-08T05:42:47.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.529+0000 7f0e2a200700 1 --2- 172.21.15.112:0/1441449946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2410b050 0x7f0e2410b470 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.529+0000 7f0e2a200700 1 -- 172.21.15.112:0/1441449946 >> 172.21.15.112:0/1441449946 conn(0x7f0e2407e460 msgr2=0x7f0e2407e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:47.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.529+0000 7f0e2a200700 1 -- 172.21.15.112:0/1441449946 shutdown_connections 2024-05-08T05:42:47.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.529+0000 7f0e2a200700 1 -- 172.21.15.112:0/1441449946 wait complete. 2024-05-08T05:42:47.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.530+0000 7f0e2a200700 1 Processor -- start 2024-05-08T05:42:47.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.531+0000 7f0e2a200700 1 -- start start 2024-05-08T05:42:47.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.531+0000 7f0e2a200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2410b050 0x7f0e2419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.531+0000 7f0e2a200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2410f600 con 0x7f0e2410b050 2024-05-08T05:42:47.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.531+0000 7f0e23400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2410b050 0x7f0e2419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.531+0000 7f0e23400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2410b050 0x7f0e2419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49548/0 (socket says 172.21.15.112:49548) 2024-05-08T05:42:47.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.531+0000 7f0e23400700 1 -- 172.21.15.112:0/1258653035 learned_addr learned my addr 172.21.15.112:0/1258653035 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:47.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.532+0000 7f0e23400700 1 -- 172.21.15.112:0/1258653035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2419f4e0 con 0x7f0e2410b050 2024-05-08T05:42:47.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.532+0000 7f0e23400700 1 --2- 172.21.15.112:0/1258653035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2410b050 0x7f0e2419efa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e14000c00 tx=0x7f0e14011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:47.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.532+0000 7f0e1be00700 1 -- 172.21.15.112:0/1258653035 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e14003b20 con 0x7f0e2410b050 2024-05-08T05:42:47.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.532+0000 7f0e1be00700 1 -- 172.21.15.112:0/1258653035 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0e1401a8c0 con 0x7f0e2410b050 2024-05-08T05:42:47.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.532+0000 7f0e1be00700 1 -- 172.21.15.112:0/1258653035 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e14024940 con 0x7f0e2410b050 2024-05-08T05:42:47.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.532+0000 7f0e2a200700 1 -- 172.21.15.112:0/1258653035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e2419f770 con 0x7f0e2410b050 2024-05-08T05:42:47.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.532+0000 7f0e2a200700 1 -- 172.21.15.112:0/1258653035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e241a2460 con 0x7f0e2410b050 2024-05-08T05:42:47.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.533+0000 7f0e1be00700 1 -- 172.21.15.112:0/1258653035 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0e14022020 con 0x7f0e2410b050 2024-05-08T05:42:47.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.533+0000 7f0e2a200700 1 -- 172.21.15.112:0/1258653035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e24064b00 con 0x7f0e2410b050 2024-05-08T05:42:47.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.534+0000 7f0e1be00700 1 --2- 172.21.15.112:0/1258653035 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f0e0803e4c0 0x7f0e08040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.534+0000 7f0e1be00700 1 -- 172.21.15.112:0/1258653035 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0e14053740 con 0x7f0e2410b050 2024-05-08T05:42:47.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.534+0000 7f0e22a00700 1 --2- 172.21.15.112:0/1258653035 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f0e0803e4c0 0x7f0e08040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.535+0000 7f0e22a00700 1 --2- 172.21.15.112:0/1258653035 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f0e0803e4c0 0x7f0e08040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0e10006fd0 tx=0x7f0e10006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:47.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.540+0000 7f0e1be00700 1 -- 172.21.15.112:0/1258653035 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e140042a0 con 0x7f0e2410b050 2024-05-08T05:42:47.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.825+0000 7f0e2a200700 1 -- 172.21.15.112:0/1258653035 --> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0e24108b20 con 0x7f0e0803e4c0 2024-05-08T05:42:47.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.833+0000 7f0e1be00700 1 -- 172.21.15.112:0/1258653035 <== mgr.14120 v2:172.21.15.112:6800/164150456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0e24108b20 con 0x7f0e0803e4c0 2024-05-08T05:42:47.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.836+0000 7f0e2a200700 1 -- 172.21.15.112:0/1258653035 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f0e0803e4c0 msgr2=0x7f0e08040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:47.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.836+0000 7f0e2a200700 1 --2- 172.21.15.112:0/1258653035 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f0e0803e4c0 0x7f0e08040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0e10006fd0 tx=0x7f0e10006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:47.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.836+0000 7f0e2a200700 1 -- 172.21.15.112:0/1258653035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2410b050 msgr2=0x7f0e2419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:47.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.836+0000 7f0e2a200700 1 --2- 172.21.15.112:0/1258653035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2410b050 0x7f0e2419efa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e14000c00 tx=0x7f0e14011b70 comp rx=0 tx=0).stop 2024-05-08T05:42:47.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.836+0000 7f0e2a200700 1 -- 172.21.15.112:0/1258653035 shutdown_connections 2024-05-08T05:42:47.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.836+0000 7f0e2a200700 1 --2- 172.21.15.112:0/1258653035 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f0e0803e4c0 0x7f0e08040980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.836+0000 7f0e2a200700 1 --2- 172.21.15.112:0/1258653035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2410b050 0x7f0e2419efa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.836+0000 7f0e2a200700 1 -- 172.21.15.112:0/1258653035 >> 172.21.15.112:0/1258653035 conn(0x7f0e2407e460 msgr2=0x7f0e24108410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:47.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.836+0000 7f0e2a200700 1 -- 172.21.15.112:0/1258653035 shutdown_connections 2024-05-08T05:42:47.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:46.836+0000 7f0e2a200700 1 -- 172.21.15.112:0/1258653035 wait complete. 2024-05-08T05:42:47.931 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying unmanaged mgr service... 2024-05-08T05:42:49.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-08T05:42:49.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.651+0000 7fa4aba00700 1 Processor -- start 2024-05-08T05:42:49.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.651+0000 7fa4aba00700 1 -- start start 2024-05-08T05:42:49.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.652+0000 7fa4aba00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a407ee40 0x7fa4a407d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:49.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.652+0000 7fa4aba00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a407d870 con 0x7fa4a407ee40 2024-05-08T05:42:49.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.652+0000 7fa4a9000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a407ee40 0x7fa4a407d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:49.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.652+0000 7fa4a9000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a407ee40 0x7fa4a407d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49554/0 (socket says 172.21.15.112:49554) 2024-05-08T05:42:49.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.652+0000 7fa4a9000700 1 -- 172.21.15.112:0/1236621038 learned_addr learned my addr 172.21.15.112:0/1236621038 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:49.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.653+0000 7fa4a9000700 1 -- 172.21.15.112:0/1236621038 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a407d9f0 con 0x7fa4a407ee40 2024-05-08T05:42:49.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.653+0000 7fa4a9000700 1 --2- 172.21.15.112:0/1236621038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a407ee40 0x7fa4a407d2a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa4880098a0 tx=0x7fa488009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84f37fc50903554 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:49.644 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.653+0000 7fa4a3e00700 1 -- 172.21.15.112:0/1236621038 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa488004160 con 0x7fa4a407ee40 2024-05-08T05:42:49.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.653+0000 7fa4a3e00700 1 -- 172.21.15.112:0/1236621038 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa488004300 con 0x7fa4a407ee40 2024-05-08T05:42:49.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.653+0000 7fa4a3e00700 1 -- 172.21.15.112:0/1236621038 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa488011af0 con 0x7fa4a407ee40 2024-05-08T05:42:49.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.654+0000 7fa4aba00700 1 -- 172.21.15.112:0/1236621038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a407ee40 msgr2=0x7fa4a407d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:49.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.654+0000 7fa4aba00700 1 --2- 172.21.15.112:0/1236621038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a407ee40 0x7fa4a407d2a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa4880098a0 tx=0x7fa488009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.654+0000 7fa4aba00700 1 -- 172.21.15.112:0/1236621038 shutdown_connections 2024-05-08T05:42:49.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.654+0000 7fa4aba00700 1 --2- 172.21.15.112:0/1236621038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a407ee40 0x7fa4a407d2a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.654+0000 7fa4aba00700 1 -- 172.21.15.112:0/1236621038 >> 172.21.15.112:0/1236621038 conn(0x7fa4a41043f0 msgr2=0x7fa4a4106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:49.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.654+0000 7fa4aba00700 1 -- 172.21.15.112:0/1236621038 shutdown_connections 2024-05-08T05:42:49.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.654+0000 7fa4aba00700 1 -- 172.21.15.112:0/1236621038 wait complete. 2024-05-08T05:42:49.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.655+0000 7fa4aba00700 1 Processor -- start 2024-05-08T05:42:49.645 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.656+0000 7fa4aba00700 1 -- start start 2024-05-08T05:42:49.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.656+0000 7fa4aba00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a407ee40 0x7fa4a419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:49.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.656+0000 7fa4aba00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a410f3c0 con 0x7fa4a407ee40 2024-05-08T05:42:49.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.656+0000 7fa4a9000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a407ee40 0x7fa4a419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:49.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.656+0000 7fa4a9000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a407ee40 0x7fa4a419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49558/0 (socket says 172.21.15.112:49558) 2024-05-08T05:42:49.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.656+0000 7fa4a9000700 1 -- 172.21.15.112:0/3612685577 learned_addr learned my addr 172.21.15.112:0/3612685577 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:49.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.657+0000 7fa4a9000700 1 -- 172.21.15.112:0/3612685577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a419f4e0 con 0x7fa4a407ee40 2024-05-08T05:42:49.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.657+0000 7fa4a9000700 1 --2- 172.21.15.112:0/3612685577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a407ee40 0x7fa4a419efa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa488000c00 tx=0x7fa488011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:49.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.657+0000 7fa4a2000700 1 -- 172.21.15.112:0/3612685577 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa488003b20 con 0x7fa4a407ee40 2024-05-08T05:42:49.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.657+0000 7fa4a2000700 1 -- 172.21.15.112:0/3612685577 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa48801a8c0 con 0x7fa4a407ee40 2024-05-08T05:42:49.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.657+0000 7fa4a2000700 1 -- 172.21.15.112:0/3612685577 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa488024940 con 0x7fa4a407ee40 2024-05-08T05:42:49.646 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.657+0000 7fa4aba00700 1 -- 172.21.15.112:0/3612685577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a419f770 con 0x7fa4a407ee40 2024-05-08T05:42:49.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.657+0000 7fa4aba00700 1 -- 172.21.15.112:0/3612685577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a41a2460 con 0x7fa4a407ee40 2024-05-08T05:42:49.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.659+0000 7fa4aba00700 1 -- 172.21.15.112:0/3612685577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a4064b00 con 0x7fa4a407ee40 2024-05-08T05:42:49.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.659+0000 7fa4a2000700 1 -- 172.21.15.112:0/3612685577 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fa488022020 con 0x7fa4a407ee40 2024-05-08T05:42:49.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.659+0000 7fa4a2000700 1 --2- 172.21.15.112:0/3612685577 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7fa48003e4c0 0x7fa480040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:49.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.659+0000 7fa4a2000700 1 -- 172.21.15.112:0/3612685577 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa488053740 con 0x7fa4a407ee40 2024-05-08T05:42:49.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.659+0000 7fa49be00700 1 --2- 172.21.15.112:0/3612685577 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7fa48003e4c0 0x7fa480040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:49.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.660+0000 7fa49be00700 1 --2- 172.21.15.112:0/3612685577 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7fa48003e4c0 0x7fa480040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa490006fd0 tx=0x7fa490006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:49.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.665+0000 7fa4a2000700 1 -- 172.21.15.112:0/3612685577 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa4880042a0 con 0x7fa4a407ee40 2024-05-08T05:42:49.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.916+0000 7fa4aba00700 1 -- 172.21.15.112:0/3612685577 --> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa4a4105650 con 0x7fa48003e4c0 2024-05-08T05:42:49.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.928+0000 7fa4a2000700 1 -- 172.21.15.112:0/3612685577 <== mgr.14120 v2:172.21.15.112:6800/164150456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa4a4105650 con 0x7fa48003e4c0 2024-05-08T05:42:49.647 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.931+0000 7fa4aba00700 1 -- 172.21.15.112:0/3612685577 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7fa48003e4c0 msgr2=0x7fa480040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:49.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.931+0000 7fa4aba00700 1 --2- 172.21.15.112:0/3612685577 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7fa48003e4c0 0x7fa480040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa490006fd0 tx=0x7fa490006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:49.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.931+0000 7fa4aba00700 1 -- 172.21.15.112:0/3612685577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a407ee40 msgr2=0x7fa4a419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:49.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.931+0000 7fa4aba00700 1 --2- 172.21.15.112:0/3612685577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a407ee40 0x7fa4a419efa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa488000c00 tx=0x7fa488011b70 comp rx=0 tx=0).stop 2024-05-08T05:42:49.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.931+0000 7fa4aba00700 1 -- 172.21.15.112:0/3612685577 shutdown_connections 2024-05-08T05:42:49.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.931+0000 7fa4aba00700 1 --2- 172.21.15.112:0/3612685577 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7fa48003e4c0 0x7fa480040980 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.931+0000 7fa4aba00700 1 --2- 172.21.15.112:0/3612685577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4a407ee40 0x7fa4a419efa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.931+0000 7fa4aba00700 1 -- 172.21.15.112:0/3612685577 >> 172.21.15.112:0/3612685577 conn(0x7fa4a41043f0 msgr2=0x7fa4a4104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:49.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.931+0000 7fa4aba00700 1 -- 172.21.15.112:0/3612685577 shutdown_connections 2024-05-08T05:42:49.648 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:48.931+0000 7fa4aba00700 1 -- 172.21.15.112:0/3612685577 wait complete. 2024-05-08T05:42:50.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:49 smithi112 ceph-mon[30003]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:50.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:49 smithi112 ceph-mon[30003]: Saving service mgr spec with placement count:2 2024-05-08T05:42:50.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:49 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' 2024-05-08T05:42:50.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:49 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' 2024-05-08T05:42:51.177 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:51 smithi112 ceph-mon[30003]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:42:51.177 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:51 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' 2024-05-08T05:42:51.177 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:51 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1460372907' entity='client.admin' 2024-05-08T05:42:51.187 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.224+0000 7f490b800700 1 Processor -- start 2024-05-08T05:42:51.187 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.224+0000 7f490b800700 1 -- start start 2024-05-08T05:42:51.187 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.225+0000 7f490b800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4904109890 0x7f490410bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.187 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.225+0000 7f490b800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f490407ce30 con 0x7f4904109890 2024-05-08T05:42:51.188 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.225+0000 7f4908e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4904109890 0x7f490410bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.188 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.225+0000 7f4908e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4904109890 0x7f490410bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49570/0 (socket says 172.21.15.112:49570) 2024-05-08T05:42:51.188 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.225+0000 7f4908e00700 1 -- 172.21.15.112:0/2625666818 learned_addr learned my addr 172.21.15.112:0/2625666818 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:51.188 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.226+0000 7f4908e00700 1 -- 172.21.15.112:0/2625666818 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490410c1c0 con 0x7f4904109890 2024-05-08T05:42:51.188 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.226+0000 7f4908e00700 1 --2- 172.21.15.112:0/2625666818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4904109890 0x7f490410bc80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f48f80098a0 tx=0x7f48f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3829a2bd8a51853 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:51.188 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.227+0000 7f4903400700 1 -- 172.21.15.112:0/2625666818 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48f8004160 con 0x7f4904109890 2024-05-08T05:42:51.188 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.227+0000 7f4903400700 1 -- 172.21.15.112:0/2625666818 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48f8004300 con 0x7f4904109890 2024-05-08T05:42:51.188 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.227+0000 7f4903400700 1 -- 172.21.15.112:0/2625666818 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48f8011a20 con 0x7f4904109890 2024-05-08T05:42:51.188 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.227+0000 7f490b800700 1 -- 172.21.15.112:0/2625666818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4904109890 msgr2=0x7f490410bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.188 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.227+0000 7f490b800700 1 --2- 172.21.15.112:0/2625666818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4904109890 0x7f490410bc80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f48f80098a0 tx=0x7f48f8009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.188 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.228+0000 7f490b800700 1 -- 172.21.15.112:0/2625666818 shutdown_connections 2024-05-08T05:42:51.188 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.228+0000 7f490b800700 1 --2- 172.21.15.112:0/2625666818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4904109890 0x7f490410bc80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.188 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.228+0000 7f490b800700 1 -- 172.21.15.112:0/2625666818 >> 172.21.15.112:0/2625666818 conn(0x7f49041032c0 msgr2=0x7f4904105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:51.188 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.228+0000 7f490b800700 1 -- 172.21.15.112:0/2625666818 shutdown_connections 2024-05-08T05:42:51.189 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.228+0000 7f490b800700 1 -- 172.21.15.112:0/2625666818 wait complete. 2024-05-08T05:42:51.189 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.229+0000 7f490b800700 1 Processor -- start 2024-05-08T05:42:51.189 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.229+0000 7f490b800700 1 -- start start 2024-05-08T05:42:51.189 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.230+0000 7f490b800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4904109890 0x7f490419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.189 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.230+0000 7f490b800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f490410d1d0 con 0x7f4904109890 2024-05-08T05:42:51.189 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.230+0000 7f4908e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4904109890 0x7f490419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.189 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.230+0000 7f4908e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4904109890 0x7f490419efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49582/0 (socket says 172.21.15.112:49582) 2024-05-08T05:42:51.192 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.230+0000 7f4908e00700 1 -- 172.21.15.112:0/1460372907 learned_addr learned my addr 172.21.15.112:0/1460372907 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:51.192 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.230+0000 7f4908e00700 1 -- 172.21.15.112:0/1460372907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490419f500 con 0x7f4904109890 2024-05-08T05:42:51.192 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.230+0000 7f4908e00700 1 --2- 172.21.15.112:0/1460372907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4904109890 0x7f490419efc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f48f8000c00 tx=0x7f48f8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:51.192 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.231+0000 7f4901600700 1 -- 172.21.15.112:0/1460372907 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48f8003aa0 con 0x7f4904109890 2024-05-08T05:42:51.192 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.231+0000 7f4901600700 1 -- 172.21.15.112:0/1460372907 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48f801b8f0 con 0x7f4904109890 2024-05-08T05:42:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.231+0000 7f490b800700 1 -- 172.21.15.112:0/1460372907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490419f790 con 0x7f4904109890 2024-05-08T05:42:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.231+0000 7f4901600700 1 -- 172.21.15.112:0/1460372907 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48f8023870 con 0x7f4904109890 2024-05-08T05:42:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.231+0000 7f490b800700 1 -- 172.21.15.112:0/1460372907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49041a2430 con 0x7f4904109890 2024-05-08T05:42:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.232+0000 7f4901600700 1 -- 172.21.15.112:0/1460372907 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f48f8023b70 con 0x7f4904109890 2024-05-08T05:42:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.232+0000 7f490b800700 1 -- 172.21.15.112:0/1460372907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4904064b00 con 0x7f4904109890 2024-05-08T05:42:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.233+0000 7f4901600700 1 --2- 172.21.15.112:0/1460372907 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f48ec03e470 0x7f48ec040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.233+0000 7f4901600700 1 -- 172.21.15.112:0/1460372907 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f48f8053e90 con 0x7f4904109890 2024-05-08T05:42:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.234+0000 7f4903e00700 1 --2- 172.21.15.112:0/1460372907 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f48ec03e470 0x7f48ec040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.234+0000 7f4903e00700 1 --2- 172.21.15.112:0/1460372907 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f48ec03e470 0x7f48ec040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f48f0006fd0 tx=0x7f48f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.238+0000 7f4901600700 1 -- 172.21.15.112:0/1460372907 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f48f801f030 con 0x7f4904109890 2024-05-08T05:42:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.467+0000 7f490b800700 1 -- 172.21.15.112:0/1460372907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f49041a2a20 con 0x7f4904109890 2024-05-08T05:42:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.475+0000 7f4901600700 1 -- 172.21.15.112:0/1460372907 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f48f8055070 con 0x7f4904109890 2024-05-08T05:42:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.479+0000 7f490b800700 1 -- 172.21.15.112:0/1460372907 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f48ec03e470 msgr2=0x7f48ec040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.479+0000 7f490b800700 1 --2- 172.21.15.112:0/1460372907 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f48ec03e470 0x7f48ec040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f48f0006fd0 tx=0x7f48f0006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.479+0000 7f490b800700 1 -- 172.21.15.112:0/1460372907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4904109890 msgr2=0x7f490419efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.479+0000 7f490b800700 1 --2- 172.21.15.112:0/1460372907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4904109890 0x7f490419efc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f48f8000c00 tx=0x7f48f8011aa0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.480+0000 7f490b800700 1 -- 172.21.15.112:0/1460372907 shutdown_connections 2024-05-08T05:42:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.480+0000 7f490b800700 1 --2- 172.21.15.112:0/1460372907 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f48ec03e470 0x7f48ec040930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.480+0000 7f490b800700 1 --2- 172.21.15.112:0/1460372907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4904109890 0x7f490419efc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.480+0000 7f490b800700 1 -- 172.21.15.112:0/1460372907 >> 172.21.15.112:0/1460372907 conn(0x7f49041032c0 msgr2=0x7f4904103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.480+0000 7f490b800700 1 -- 172.21.15.112:0/1460372907 shutdown_connections 2024-05-08T05:42:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:50.480+0000 7f490b800700 1 -- 172.21.15.112:0/1460372907 wait complete. 2024-05-08T05:42:52.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.766+0000 7f7847400700 1 Processor -- start 2024-05-08T05:42:52.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.766+0000 7f7847400700 1 -- start start 2024-05-08T05:42:52.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.766+0000 7f7847400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7840108e30 0x7f7840109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.766+0000 7f7847400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7840109790 con 0x7f7840108e30 2024-05-08T05:42:52.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.767+0000 7f7844a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7840108e30 0x7f7840109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.767+0000 7f7844a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7840108e30 0x7f7840109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49596/0 (socket says 172.21.15.112:49596) 2024-05-08T05:42:52.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.767+0000 7f7844a00700 1 -- 172.21.15.112:0/84426674 learned_addr learned my addr 172.21.15.112:0/84426674 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:52.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.767+0000 7f7844a00700 1 -- 172.21.15.112:0/84426674 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7840109f90 con 0x7f7840108e30 2024-05-08T05:42:52.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.768+0000 7f7844a00700 1 --2- 172.21.15.112:0/84426674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7840108e30 0x7f7840109250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f78340098a0 tx=0x7f7834009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee7fa1de40d7117 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:52.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.768+0000 7f783f400700 1 -- 172.21.15.112:0/84426674 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7834004160 con 0x7f7840108e30 2024-05-08T05:42:52.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.768+0000 7f783f400700 1 -- 172.21.15.112:0/84426674 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7834004300 con 0x7f7840108e30 2024-05-08T05:42:52.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.768+0000 7f783f400700 1 -- 172.21.15.112:0/84426674 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7834011a20 con 0x7f7840108e30 2024-05-08T05:42:52.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.769+0000 7f7847400700 1 -- 172.21.15.112:0/84426674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7840108e30 msgr2=0x7f7840109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:52.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.769+0000 7f7847400700 1 --2- 172.21.15.112:0/84426674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7840108e30 0x7f7840109250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f78340098a0 tx=0x7f7834009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.770+0000 7f7847400700 1 -- 172.21.15.112:0/84426674 shutdown_connections 2024-05-08T05:42:52.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.770+0000 7f7847400700 1 --2- 172.21.15.112:0/84426674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7840108e30 0x7f7840109250 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.770+0000 7f7847400700 1 -- 172.21.15.112:0/84426674 >> 172.21.15.112:0/84426674 conn(0x7f78401043f0 msgr2=0x7f7840106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:52.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.770+0000 7f7847400700 1 -- 172.21.15.112:0/84426674 shutdown_connections 2024-05-08T05:42:52.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.770+0000 7f7847400700 1 -- 172.21.15.112:0/84426674 wait complete. 2024-05-08T05:42:52.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.771+0000 7f7847400700 1 Processor -- start 2024-05-08T05:42:52.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.771+0000 7f7847400700 1 -- start start 2024-05-08T05:42:52.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.772+0000 7f7847400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7840108e30 0x7f784019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.772+0000 7f7847400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f784010d3d0 con 0x7f7840108e30 2024-05-08T05:42:52.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.772+0000 7f7844a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7840108e30 0x7f784019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.772+0000 7f7844a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7840108e30 0x7f784019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49598/0 (socket says 172.21.15.112:49598) 2024-05-08T05:42:52.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.772+0000 7f7844a00700 1 -- 172.21.15.112:0/3502869078 learned_addr learned my addr 172.21.15.112:0/3502869078 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:52.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.772+0000 7f7844a00700 1 -- 172.21.15.112:0/3502869078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784019d280 con 0x7f7840108e30 2024-05-08T05:42:52.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.773+0000 7f7844a00700 1 --2- 172.21.15.112:0/3502869078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7840108e30 0x7f784019cd40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f78340116e0 tx=0x7f7834011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:52.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.773+0000 7f783d600700 1 -- 172.21.15.112:0/3502869078 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7834003a60 con 0x7f7840108e30 2024-05-08T05:42:52.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.773+0000 7f783d600700 1 -- 172.21.15.112:0/3502869078 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783401b8f0 con 0x7f7840108e30 2024-05-08T05:42:52.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.773+0000 7f7847400700 1 -- 172.21.15.112:0/3502869078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784019d510 con 0x7f7840108e30 2024-05-08T05:42:52.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.773+0000 7f783d600700 1 -- 172.21.15.112:0/3502869078 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7834019860 con 0x7f7840108e30 2024-05-08T05:42:52.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.773+0000 7f7847400700 1 -- 172.21.15.112:0/3502869078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78401a0200 con 0x7f7840108e30 2024-05-08T05:42:52.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.774+0000 7f7847400700 1 -- 172.21.15.112:0/3502869078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f784010d550 con 0x7f7840108e30 2024-05-08T05:42:52.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.775+0000 7f783d600700 1 -- 172.21.15.112:0/3502869078 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f783401b440 con 0x7f7840108e30 2024-05-08T05:42:52.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.775+0000 7f783d600700 1 --2- 172.21.15.112:0/3502869078 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f782803e470 0x7f7828040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.775+0000 7f783d600700 1 -- 172.21.15.112:0/3502869078 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7834053c90 con 0x7f7840108e30 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.775+0000 7f783fe00700 1 --2- 172.21.15.112:0/3502869078 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f782803e470 0x7f7828040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.776+0000 7f783fe00700 1 --2- 172.21.15.112:0/3502869078 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f782803e470 0x7f7828040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f782c006fd0 tx=0x7f782c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:51.780+0000 7f783d600700 1 -- 172.21.15.112:0/3502869078 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f783401f030 con 0x7f7840108e30 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:52.015+0000 7f7847400700 1 -- 172.21.15.112:0/3502869078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f78401a0820 con 0x7f7840108e30 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:52.018+0000 7f783d600700 1 -- 172.21.15.112:0/3502869078 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f78340223b0 con 0x7f7840108e30 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:52.022+0000 7f7847400700 1 -- 172.21.15.112:0/3502869078 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f782803e470 msgr2=0x7f7828040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:52.022+0000 7f7847400700 1 --2- 172.21.15.112:0/3502869078 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f782803e470 0x7f7828040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f782c006fd0 tx=0x7f782c006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:52.022+0000 7f7847400700 1 -- 172.21.15.112:0/3502869078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7840108e30 msgr2=0x7f784019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:52.022+0000 7f7847400700 1 --2- 172.21.15.112:0/3502869078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7840108e30 0x7f784019cd40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f78340116e0 tx=0x7f7834011aa0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:52.022+0000 7f7847400700 1 -- 172.21.15.112:0/3502869078 shutdown_connections 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:52.022+0000 7f7847400700 1 --2- 172.21.15.112:0/3502869078 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f782803e470 0x7f7828040930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:52.022+0000 7f7847400700 1 --2- 172.21.15.112:0/3502869078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7840108e30 0x7f784019cd40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:52.022+0000 7f7847400700 1 -- 172.21.15.112:0/3502869078 >> 172.21.15.112:0/3502869078 conn(0x7f78401043f0 msgr2=0x7f7840104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:52.023+0000 7f7847400700 1 -- 172.21.15.112:0/3502869078 shutdown_connections 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:52.023+0000 7f7847400700 1 -- 172.21.15.112:0/3502869078 wait complete. 2024-05-08T05:42:52.993 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling the dashboard module... 2024-05-08T05:42:53.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:53 smithi112 ceph-mon[30003]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:42:53.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:53 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3502869078' entity='client.admin' 2024-05-08T05:42:53.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:53 smithi112 ceph-mon[30003]: from='mgr.14120 172.21.15.112:0/342358141' entity='mgr.x' 2024-05-08T05:42:54.131 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:54 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1509309783' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-08T05:42:54.464 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:54 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: ignoring --setuser ceph since I am not root 2024-05-08T05:42:54.464 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:54 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: ignoring --setgroup ceph since I am not root 2024-05-08T05:42:54.464 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:54 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:54.351+0000 7fa179b2e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:42:54.748 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:54 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:54.610+0000 7fa179b2e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:42:54.748 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:54 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:54.747+0000 7fa179b2e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:42:54.823 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.550+0000 7fd269a00700 1 Processor -- start 2024-05-08T05:42:54.823 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.550+0000 7fd269a00700 1 -- start start 2024-05-08T05:42:54.823 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.551+0000 7fd269a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd264108e20 0x7fd264109240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:54.823 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.551+0000 7fd269a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd264109780 con 0x7fd264108e20 2024-05-08T05:42:54.823 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.551+0000 7fd262a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd264108e20 0x7fd264109240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:54.823 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.551+0000 7fd262a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd264108e20 0x7fd264109240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49610/0 (socket says 172.21.15.112:49610) 2024-05-08T05:42:54.823 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.551+0000 7fd262a00700 1 -- 172.21.15.112:0/3010056412 learned_addr learned my addr 172.21.15.112:0/3010056412 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:54.823 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.552+0000 7fd262a00700 1 -- 172.21.15.112:0/3010056412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd264109f80 con 0x7fd264108e20 2024-05-08T05:42:54.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.552+0000 7fd262a00700 1 --2- 172.21.15.112:0/3010056412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd264108e20 0x7fd264109240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2540098a0 tx=0x7fd254009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22674b1a4d4aaa1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:54.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.552+0000 7fd261600700 1 -- 172.21.15.112:0/3010056412 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd254004160 con 0x7fd264108e20 2024-05-08T05:42:54.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.552+0000 7fd261600700 1 -- 172.21.15.112:0/3010056412 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd254004300 con 0x7fd264108e20 2024-05-08T05:42:54.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.552+0000 7fd261600700 1 -- 172.21.15.112:0/3010056412 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd254011a20 con 0x7fd264108e20 2024-05-08T05:42:54.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.553+0000 7fd269a00700 1 -- 172.21.15.112:0/3010056412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd264108e20 msgr2=0x7fd264109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:54.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.553+0000 7fd269a00700 1 --2- 172.21.15.112:0/3010056412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd264108e20 0x7fd264109240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2540098a0 tx=0x7fd254009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.553+0000 7fd269a00700 1 -- 172.21.15.112:0/3010056412 shutdown_connections 2024-05-08T05:42:54.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.553+0000 7fd269a00700 1 --2- 172.21.15.112:0/3010056412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd264108e20 0x7fd264109240 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.553+0000 7fd269a00700 1 -- 172.21.15.112:0/3010056412 >> 172.21.15.112:0/3010056412 conn(0x7fd2641043a0 msgr2=0x7fd264106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:54.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.553+0000 7fd269a00700 1 -- 172.21.15.112:0/3010056412 shutdown_connections 2024-05-08T05:42:54.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.553+0000 7fd269a00700 1 -- 172.21.15.112:0/3010056412 wait complete. 2024-05-08T05:42:54.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.554+0000 7fd269a00700 1 Processor -- start 2024-05-08T05:42:54.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.555+0000 7fd269a00700 1 -- start start 2024-05-08T05:42:54.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.555+0000 7fd269a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd264108e20 0x7fd26419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:54.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.555+0000 7fd269a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd26410d3c0 con 0x7fd264108e20 2024-05-08T05:42:54.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.555+0000 7fd262a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd264108e20 0x7fd26419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:54.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.555+0000 7fd262a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd264108e20 0x7fd26419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49616/0 (socket says 172.21.15.112:49616) 2024-05-08T05:42:54.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.555+0000 7fd262a00700 1 -- 172.21.15.112:0/1509309783 learned_addr learned my addr 172.21.15.112:0/1509309783 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:54.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.555+0000 7fd262a00700 1 -- 172.21.15.112:0/1509309783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26419f460 con 0x7fd264108e20 2024-05-08T05:42:54.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.556+0000 7fd262a00700 1 --2- 172.21.15.112:0/1509309783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd264108e20 0x7fd26419ef20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd254000c00 tx=0x7fd254011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:54.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.556+0000 7fd25b400700 1 -- 172.21.15.112:0/1509309783 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd254003aa0 con 0x7fd264108e20 2024-05-08T05:42:54.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.556+0000 7fd25b400700 1 -- 172.21.15.112:0/1509309783 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd25401b8f0 con 0x7fd264108e20 2024-05-08T05:42:54.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.556+0000 7fd25b400700 1 -- 172.21.15.112:0/1509309783 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd254023870 con 0x7fd264108e20 2024-05-08T05:42:54.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.556+0000 7fd269a00700 1 -- 172.21.15.112:0/1509309783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26419f6f0 con 0x7fd264108e20 2024-05-08T05:42:54.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.556+0000 7fd269a00700 1 -- 172.21.15.112:0/1509309783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2641a23e0 con 0x7fd264108e20 2024-05-08T05:42:54.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.557+0000 7fd25b400700 1 -- 172.21.15.112:0/1509309783 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd25401b440 con 0x7fd264108e20 2024-05-08T05:42:54.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.557+0000 7fd269a00700 1 -- 172.21.15.112:0/1509309783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd26410d540 con 0x7fd264108e20 2024-05-08T05:42:54.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.558+0000 7fd25b400700 1 --2- 172.21.15.112:0/1509309783 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7fd24803e4c0 0x7fd248040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.558+0000 7fd25b400700 1 -- 172.21.15.112:0/1509309783 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd254053da0 con 0x7fd264108e20 2024-05-08T05:42:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.558+0000 7fd262000700 1 --2- 172.21.15.112:0/1509309783 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7fd24803e4c0 0x7fd248040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.559+0000 7fd262000700 1 --2- 172.21.15.112:0/1509309783 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7fd24803e4c0 0x7fd248040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd250006fd0 tx=0x7fd250006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.564+0000 7fd25b400700 1 -- 172.21.15.112:0/1509309783 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd25401f030 con 0x7fd264108e20 2024-05-08T05:42:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:53.867+0000 7fd269a00700 1 -- 172.21.15.112:0/1509309783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fd2641a29c0 con 0x7fd264108e20 2024-05-08T05:42:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:54.028+0000 7fd25b400700 1 -- 172.21.15.112:0/1509309783 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fd25400be50 con 0x7fd264108e20 2024-05-08T05:42:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:54.033+0000 7fd269a00700 1 -- 172.21.15.112:0/1509309783 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7fd24803e4c0 msgr2=0x7fd248040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:54.033+0000 7fd269a00700 1 --2- 172.21.15.112:0/1509309783 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7fd24803e4c0 0x7fd248040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd250006fd0 tx=0x7fd250006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:54.033+0000 7fd269a00700 1 -- 172.21.15.112:0/1509309783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd264108e20 msgr2=0x7fd26419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:54.033+0000 7fd25b400700 1 -- 172.21.15.112:0/1509309783 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fd25400b3e0 con 0x7fd264108e20 2024-05-08T05:42:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:54.033+0000 7fd269a00700 1 --2- 172.21.15.112:0/1509309783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd264108e20 0x7fd26419ef20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd254000c00 tx=0x7fd254011aa0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:54.034+0000 7fd269a00700 1 -- 172.21.15.112:0/1509309783 shutdown_connections 2024-05-08T05:42:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:54.034+0000 7fd269a00700 1 --2- 172.21.15.112:0/1509309783 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7fd24803e4c0 0x7fd248040980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:54.034+0000 7fd269a00700 1 --2- 172.21.15.112:0/1509309783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd264108e20 0x7fd26419ef20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.827 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:54.034+0000 7fd269a00700 1 -- 172.21.15.112:0/1509309783 >> 172.21.15.112:0/1509309783 conn(0x7fd2641043a0 msgr2=0x7fd264104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:54.827 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:54.034+0000 7fd269a00700 1 -- 172.21.15.112:0/1509309783 shutdown_connections 2024-05-08T05:42:54.827 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:54.034+0000 7fd269a00700 1 -- 172.21.15.112:0/1509309783 wait complete. 2024-05-08T05:42:55.271 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:55 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:55.151+0000 7fa179b2e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:42:55.271 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:55 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1509309783' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-08T05:42:55.271 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:55 smithi112 ceph-mon[30003]: mgrmap e9: x(active, since 24s) 2024-05-08T05:42:55.565 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:55 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:55.269+0000 7fa179b2e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:42:56.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:42:56 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/429560416' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T05:42:56.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:42:56.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-08T05:42:56.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T05:42:56.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T05:42:56.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T05:42:56.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:42:56.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.464+0000 7f2708c00700 1 Processor -- start 2024-05-08T05:42:56.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.464+0000 7f2708c00700 1 -- start start 2024-05-08T05:42:56.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.464+0000 7f2708c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2704107500 0x7f2704107920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:56.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.464+0000 7f2708c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f270407cdf0 con 0x7f2704107500 2024-05-08T05:42:56.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.465+0000 7f2703400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2704107500 0x7f2704107920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:56.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.465+0000 7f2703400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2704107500 0x7f2704107920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43062/0 (socket says 172.21.15.112:43062) 2024-05-08T05:42:56.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.465+0000 7f2703400700 1 -- 172.21.15.112:0/670194175 learned_addr learned my addr 172.21.15.112:0/670194175 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:56.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.465+0000 7f2703400700 1 -- 172.21.15.112:0/670194175 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2704107e60 con 0x7f2704107500 2024-05-08T05:42:56.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.466+0000 7f2703400700 1 --2- 172.21.15.112:0/670194175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2704107500 0x7f2704107920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009e80 tx=0x7f26f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d79a6164533f36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:56.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.466+0000 7f2702a00700 1 -- 172.21.15.112:0/670194175 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26f4017070 con 0x7f2704107500 2024-05-08T05:42:56.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.466+0000 7f2702a00700 1 -- 172.21.15.112:0/670194175 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26f400b8c0 con 0x7f2704107500 2024-05-08T05:42:56.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.466+0000 7f2702a00700 1 -- 172.21.15.112:0/670194175 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26f4004490 con 0x7f2704107500 2024-05-08T05:42:56.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.467+0000 7f2708c00700 1 -- 172.21.15.112:0/670194175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2704107500 msgr2=0x7f2704107920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:56.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.467+0000 7f2708c00700 1 --2- 172.21.15.112:0/670194175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2704107500 0x7f2704107920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009e80 tx=0x7f26f400b330 comp rx=0 tx=0).stop 2024-05-08T05:42:56.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.467+0000 7f2708c00700 1 -- 172.21.15.112:0/670194175 shutdown_connections 2024-05-08T05:42:56.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.467+0000 7f2708c00700 1 --2- 172.21.15.112:0/670194175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2704107500 0x7f2704107920 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.467+0000 7f2708c00700 1 -- 172.21.15.112:0/670194175 >> 172.21.15.112:0/670194175 conn(0x7f27041030c0 msgr2=0x7f2704105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:56.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.467+0000 7f2708c00700 1 -- 172.21.15.112:0/670194175 shutdown_connections 2024-05-08T05:42:56.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.467+0000 7f2708c00700 1 -- 172.21.15.112:0/670194175 wait complete. 2024-05-08T05:42:56.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.468+0000 7f2708c00700 1 Processor -- start 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.468+0000 7f2708c00700 1 -- start start 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.468+0000 7f2708c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2704107500 0x7f270419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.468+0000 7f2708c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2704108ed0 con 0x7f2704107500 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.469+0000 7f2703400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2704107500 0x7f270419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.469+0000 7f2703400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2704107500 0x7f270419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43074/0 (socket says 172.21.15.112:43074) 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.469+0000 7f2703400700 1 -- 172.21.15.112:0/429560416 learned_addr learned my addr 172.21.15.112:0/429560416 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.469+0000 7f2703400700 1 -- 172.21.15.112:0/429560416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270419f2e0 con 0x7f2704107500 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.469+0000 7f2703400700 1 --2- 172.21.15.112:0/429560416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2704107500 0x7f270419eda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f26f4004220 tx=0x7f26f40093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.469+0000 7f2700c00700 1 -- 172.21.15.112:0/429560416 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26f4020020 con 0x7f2704107500 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.469+0000 7f2700c00700 1 -- 172.21.15.112:0/429560416 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26f4003b50 con 0x7f2704107500 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.469+0000 7f2708c00700 1 -- 172.21.15.112:0/429560416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270419f570 con 0x7f2704107500 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.470+0000 7f2700c00700 1 -- 172.21.15.112:0/429560416 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26f4017040 con 0x7f2704107500 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.470+0000 7f2708c00700 1 -- 172.21.15.112:0/429560416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27041a2260 con 0x7f2704107500 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.471+0000 7f2708c00700 1 -- 172.21.15.112:0/429560416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2704064b00 con 0x7f2704107500 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.471+0000 7f2700c00700 1 -- 172.21.15.112:0/429560416 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f26f4023020 con 0x7f2704107500 2024-05-08T05:42:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.472+0000 7f2700c00700 1 --2- 172.21.15.112:0/429560416 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f26e803e4c0 0x7f26e8040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:56.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.472+0000 7f2700c00700 1 -- 172.21.15.112:0/429560416 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f26f400f050 con 0x7f2704107500 2024-05-08T05:42:56.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.472+0000 7f26faa00700 1 -- 172.21.15.112:0/429560416 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f26e803e4c0 msgr2=0x7f26e8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/164150456 2024-05-08T05:42:56.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.472+0000 7f26faa00700 1 --2- 172.21.15.112:0/429560416 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f26e803e4c0 0x7f26e8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:42:56.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.480+0000 7f2700c00700 1 -- 172.21.15.112:0/429560416 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26f402a020 con 0x7f2704107500 2024-05-08T05:42:56.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.673+0000 7f26faa00700 1 -- 172.21.15.112:0/429560416 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f26e803e4c0 msgr2=0x7f26e8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/164150456 2024-05-08T05:42:56.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.673+0000 7f26faa00700 1 --2- 172.21.15.112:0/429560416 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f26e803e4c0 0x7f26e8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:42:56.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.773+0000 7f2708c00700 1 -- 172.21.15.112:0/429560416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f270402e850 con 0x7f2704107500 2024-05-08T05:42:56.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.774+0000 7f2700c00700 1 -- 172.21.15.112:0/429560416 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f26f402b260 con 0x7f2704107500 2024-05-08T05:42:56.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.777+0000 7f2708c00700 1 -- 172.21.15.112:0/429560416 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f26e803e4c0 msgr2=0x7f26e8040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:42:56.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.777+0000 7f2708c00700 1 --2- 172.21.15.112:0/429560416 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f26e803e4c0 0x7f26e8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.777+0000 7f2708c00700 1 -- 172.21.15.112:0/429560416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2704107500 msgr2=0x7f270419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:56.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.777+0000 7f2708c00700 1 --2- 172.21.15.112:0/429560416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2704107500 0x7f270419eda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f26f4004220 tx=0x7f26f40093e0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.777+0000 7f2708c00700 1 -- 172.21.15.112:0/429560416 shutdown_connections 2024-05-08T05:42:56.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.777+0000 7f2708c00700 1 --2- 172.21.15.112:0/429560416 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f26e803e4c0 0x7f26e8040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.777+0000 7f2708c00700 1 --2- 172.21.15.112:0/429560416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2704107500 0x7f270419eda0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.777+0000 7f2708c00700 1 -- 172.21.15.112:0/429560416 >> 172.21.15.112:0/429560416 conn(0x7f27041030c0 msgr2=0x7f2704195a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:56.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.777+0000 7f2708c00700 1 -- 172.21.15.112:0/429560416 shutdown_connections 2024-05-08T05:42:56.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:55.777+0000 7f2708c00700 1 -- 172.21.15.112:0/429560416 wait complete. 2024-05-08T05:42:56.427 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-05-08T05:42:56.427 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 9... 2024-05-08T05:42:56.983 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:56 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:56.848+0000 7fa179b2e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:42:57.298 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:57 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:57.083+0000 7fa179b2e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:42:57.565 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:57 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:57.297+0000 7fa179b2e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:42:58.043 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:57 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:57.673+0000 7fa179b2e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:42:58.043 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:57 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:57.793+0000 7fa179b2e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:42:58.294 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:58 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:58.042+0000 7fa179b2e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:42:58.565 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:58 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:58.293+0000 7fa179b2e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:42:59.360 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:59 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:59.073+0000 7fa179b2e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:42:59.815 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:59 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:59.359+0000 7fa179b2e200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:43:00.316 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:42:59 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:42:59.914+0000 7fa179b2e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:43:00.316 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:43:00 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:43:00.042+0000 7fa179b2e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:43:00.815 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:43:00 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:43:00.529+0000 7fa179b2e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:43:01.316 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:43:00 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:43:00.986+0000 7fa179b2e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:43:01.316 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:43:01 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:43:01.110+0000 7fa179b2e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:43:01.747 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:43:01 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:43:01.436+0000 7fa179b2e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:43:01.747 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:43:01 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:43:01.619+0000 7fa179b2e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:43:01.747 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:43:01 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:43:01.747+0000 7fa179b2e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:43:02.565 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:43:02 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:43:02.144+0000 7fa179b2e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:43:02.565 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:43:02 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:43:02.268+0000 7fa179b2e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:43:03.309 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:43:03 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:43:03.018+0000 7fa179b2e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:43:03.565 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:43:03 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:43:03.308+0000 7fa179b2e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:43:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:03 smithi112 ceph-mon[30003]: Active manager daemon x restarted 2024-05-08T05:43:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:03 smithi112 ceph-mon[30003]: Activating manager daemon x 2024-05-08T05:43:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:03 smithi112 ceph-mon[30003]: osdmap e3: 0 total, 0 up, 0 in 2024-05-08T05:43:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:03 smithi112 ceph-mon[30003]: mgrmap e10: x(active, starting, since 0.0139595s) 2024-05-08T05:43:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:03 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:43:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:03 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:43:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:03 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:43:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:03 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:43:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:03 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:43:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:03 smithi112 ceph-mon[30003]: Manager daemon x is now available 2024-05-08T05:43:04.690 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:04.690 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:43:04.691 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:43:04.691 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:04 smithi112 ceph-mon[30003]: [08/May/2024:05:43:03] ENGINE Bus STARTING 2024-05-08T05:43:04.691 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:04 smithi112 ceph-mon[30003]: [08/May/2024:05:43:03] ENGINE Serving on https://172.21.15.112:7150 2024-05-08T05:43:04.691 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:04 smithi112 ceph-mon[30003]: [08/May/2024:05:43:04] ENGINE Serving on http://172.21.15.112:8765 2024-05-08T05:43:04.691 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:04 smithi112 ceph-mon[30003]: [08/May/2024:05:43:04] ENGINE Bus STARTED 2024-05-08T05:43:04.691 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:04 smithi112 ceph-mon[30003]: mgrmap e11: x(active, since 1.01794s) 2024-05-08T05:43:05.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:43:05.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-08T05:43:05.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T05:43:05.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:43:05.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.982+0000 7f1fd3600700 1 Processor -- start 2024-05-08T05:43:05.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.982+0000 7f1fd3600700 1 -- start start 2024-05-08T05:43:05.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.982+0000 7f1fd3600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fcc108e30 0x7f1fcc109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:05.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.982+0000 7f1fd3600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc109790 con 0x7f1fcc108e30 2024-05-08T05:43:05.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.982+0000 7f1fd0c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fcc108e30 0x7f1fcc109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:05.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.983+0000 7f1fd0c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fcc108e30 0x7f1fcc109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43086/0 (socket says 172.21.15.112:43086) 2024-05-08T05:43:05.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.983+0000 7f1fd0c00700 1 -- 172.21.15.112:0/1103609182 learned_addr learned my addr 172.21.15.112:0/1103609182 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:05.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.983+0000 7f1fd0c00700 1 -- 172.21.15.112:0/1103609182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc109f90 con 0x7f1fcc108e30 2024-05-08T05:43:05.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.983+0000 7f1fd0c00700 1 --2- 172.21.15.112:0/1103609182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fcc108e30 0x7f1fcc109250 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00098a0 tx=0x7f1fc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8395ded7ba868e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:05.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.984+0000 7f1fcb400700 1 -- 172.21.15.112:0/1103609182 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fc0004160 con 0x7f1fcc108e30 2024-05-08T05:43:05.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.984+0000 7f1fcb400700 1 -- 172.21.15.112:0/1103609182 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fc0004300 con 0x7f1fcc108e30 2024-05-08T05:43:05.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.984+0000 7f1fcb400700 1 -- 172.21.15.112:0/1103609182 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fc0011a20 con 0x7f1fcc108e30 2024-05-08T05:43:05.063 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.984+0000 7f1fd3600700 1 -- 172.21.15.112:0/1103609182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fcc108e30 msgr2=0x7f1fcc109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:05.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.984+0000 7f1fd3600700 1 --2- 172.21.15.112:0/1103609182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fcc108e30 0x7f1fcc109250 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00098a0 tx=0x7f1fc0009bb0 comp rx=0 tx=0).stop 2024-05-08T05:43:05.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.985+0000 7f1fd3600700 1 -- 172.21.15.112:0/1103609182 shutdown_connections 2024-05-08T05:43:05.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.985+0000 7f1fd3600700 1 --2- 172.21.15.112:0/1103609182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fcc108e30 0x7f1fcc109250 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:05.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.985+0000 7f1fd3600700 1 -- 172.21.15.112:0/1103609182 >> 172.21.15.112:0/1103609182 conn(0x7f1fcc1043f0 msgr2=0x7f1fcc106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:05.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.985+0000 7f1fd3600700 1 -- 172.21.15.112:0/1103609182 shutdown_connections 2024-05-08T05:43:05.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.985+0000 7f1fd3600700 1 -- 172.21.15.112:0/1103609182 wait complete. 2024-05-08T05:43:05.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.986+0000 7f1fd3600700 1 Processor -- start 2024-05-08T05:43:05.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.986+0000 7f1fd3600700 1 -- start start 2024-05-08T05:43:05.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.986+0000 7f1fd3600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fcc108e30 0x7f1fcc19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:05.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.986+0000 7f1fd3600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc10d3d0 con 0x7f1fcc108e30 2024-05-08T05:43:05.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.987+0000 7f1fd0c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fcc108e30 0x7f1fcc19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:05.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.987+0000 7f1fd0c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fcc108e30 0x7f1fcc19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43092/0 (socket says 172.21.15.112:43092) 2024-05-08T05:43:05.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.987+0000 7f1fd0c00700 1 -- 172.21.15.112:0/543892775 learned_addr learned my addr 172.21.15.112:0/543892775 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:05.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.987+0000 7f1fd0c00700 1 -- 172.21.15.112:0/543892775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc19f450 con 0x7f1fcc108e30 2024-05-08T05:43:05.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.987+0000 7f1fd0c00700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fcc108e30 0x7f1fcc19ef10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0000c00 tx=0x7f1fc0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:05.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.988+0000 7f1fc9600700 1 -- 172.21.15.112:0/543892775 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fc0003aa0 con 0x7f1fcc108e30 2024-05-08T05:43:05.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.988+0000 7f1fc9600700 1 -- 172.21.15.112:0/543892775 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fc001b8f0 con 0x7f1fcc108e30 2024-05-08T05:43:05.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.988+0000 7f1fc9600700 1 -- 172.21.15.112:0/543892775 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fc0023870 con 0x7f1fcc108e30 2024-05-08T05:43:05.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.988+0000 7f1fd3600700 1 -- 172.21.15.112:0/543892775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fcc19f6e0 con 0x7f1fcc108e30 2024-05-08T05:43:05.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.988+0000 7f1fd3600700 1 -- 172.21.15.112:0/543892775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fcc1a23d0 con 0x7f1fcc108e30 2024-05-08T05:43:05.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.989+0000 7f1fc9600700 1 -- 172.21.15.112:0/543892775 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f1fc001b440 con 0x7f1fcc108e30 2024-05-08T05:43:05.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.990+0000 7f1fc9600700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 0x7f1fb40409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:05.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.990+0000 7f1fc9600700 1 -- 172.21.15.112:0/543892775 --> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1fb40410f0 con 0x7f1fb403e520 2024-05-08T05:43:05.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.990+0000 7f1fc9600700 1 -- 172.21.15.112:0/543892775 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1fc0053dd0 con 0x7f1fcc108e30 2024-05-08T05:43:05.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.990+0000 7f1fcbe00700 1 -- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 msgr2=0x7f1fb40409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/164150456 2024-05-08T05:43:05.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:56.990+0000 7f1fcbe00700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 0x7f1fb40409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:43:05.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:57.191+0000 7f1fcbe00700 1 -- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 msgr2=0x7f1fb40409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/164150456 2024-05-08T05:43:05.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:57.191+0000 7f1fcbe00700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 0x7f1fb40409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:43:05.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:57.592+0000 7f1fcbe00700 1 -- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 msgr2=0x7f1fb40409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/164150456 2024-05-08T05:43:05.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:57.592+0000 7f1fcbe00700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 0x7f1fb40409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T05:43:05.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:58.393+0000 7f1fcbe00700 1 -- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 msgr2=0x7f1fb40409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/164150456 2024-05-08T05:43:05.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:58.393+0000 7f1fcbe00700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 0x7f1fb40409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T05:43:05.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:59.995+0000 7f1fcbe00700 1 -- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 msgr2=0x7f1fb40409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/164150456 2024-05-08T05:43:05.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:59.995+0000 7f1fcbe00700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 0x7f1fb40409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T05:43:05.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:03.200+0000 7f1fcbe00700 1 -- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 msgr2=0x7f1fb40409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/164150456 2024-05-08T05:43:05.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:03.200+0000 7f1fcbe00700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 0x7f1fb40409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-08T05:43:05.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:03.323+0000 7f1fc9600700 1 -- 172.21.15.112:0/543892775 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f1fc000b3e0 con 0x7f1fcc108e30 2024-05-08T05:43:05.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:03.324+0000 7f1fc9600700 1 -- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 msgr2=0x7f1fb40409e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:43:05.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:03.324+0000 7f1fc9600700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 0x7f1fb40409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:05.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.324+0000 7f1fc9600700 1 -- 172.21.15.112:0/543892775 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f1fc0055520 con 0x7f1fcc108e30 2024-05-08T05:43:05.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.325+0000 7f1fc9600700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1fb4041fa0 0x7f1fb4044390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:05.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.325+0000 7f1fc9600700 1 -- 172.21.15.112:0/543892775 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1fc0024e70 con 0x7f1fb4041fa0 2024-05-08T05:43:05.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.327+0000 7f1fcbe00700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1fb4041fa0 0x7f1fb4044390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.329+0000 7f1fcbe00700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1fb4041fa0 0x7f1fb4044390 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8003a10 tx=0x7f1fb8009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.330+0000 7f1fc9600700 1 -- 172.21.15.112:0/543892775 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f1fc0024e70 con 0x7f1fb4041fa0 2024-05-08T05:43:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.340+0000 7f1fd3600700 1 -- 172.21.15.112:0/543892775 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f1fcc10d550 con 0x7f1fb4041fa0 2024-05-08T05:43:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.341+0000 7f1fc9600700 1 -- 172.21.15.112:0/543892775 <== mgr.14152 v2:172.21.15.112:6800/1211679607 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f1fcc10d550 con 0x7f1fb4041fa0 2024-05-08T05:43:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.342+0000 7f1fd3600700 1 -- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1fb4041fa0 msgr2=0x7f1fb4044390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.342+0000 7f1fd3600700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1fb4041fa0 0x7f1fb4044390 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8003a10 tx=0x7f1fb8009270 comp rx=0 tx=0).stop 2024-05-08T05:43:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.342+0000 7f1fd3600700 1 -- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fcc108e30 msgr2=0x7f1fcc19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.342+0000 7f1fd3600700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fcc108e30 0x7f1fcc19ef10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0000c00 tx=0x7f1fc0011aa0 comp rx=0 tx=0).stop 2024-05-08T05:43:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.342+0000 7f1fd3600700 1 -- 172.21.15.112:0/543892775 shutdown_connections 2024-05-08T05:43:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.342+0000 7f1fd3600700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1fb4041fa0 0x7f1fb4044390 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.342+0000 7f1fd3600700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fcc108e30 0x7f1fcc19ef10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.342+0000 7f1fd3600700 1 --2- 172.21.15.112:0/543892775 >> [v2:172.21.15.112:6800/164150456,v1:172.21.15.112:6801/164150456] conn(0x7f1fb403e520 0x7f1fb40409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.342+0000 7f1fd3600700 1 -- 172.21.15.112:0/543892775 >> 172.21.15.112:0/543892775 conn(0x7f1fcc1043f0 msgr2=0x7f1fcc104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.342+0000 7f1fd3600700 1 -- 172.21.15.112:0/543892775 shutdown_connections 2024-05-08T05:43:05.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:04.342+0000 7f1fd3600700 1 -- 172.21.15.112:0/543892775 wait complete. 2024-05-08T05:43:05.068 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 9 is available 2024-05-08T05:43:05.068 INFO:teuthology.orchestra.run.smithi112.stdout:Generating a dashboard self-signed certificate... 2024-05-08T05:43:05.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:05 smithi112 ceph-mon[30003]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T05:43:05.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:05 smithi112 ceph-mon[30003]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T05:43:06.749 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:06 smithi112 ceph-mon[30003]: mgrmap e12: x(active, since 2s) 2024-05-08T05:43:06.749 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:06 smithi112 ceph-mon[30003]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:07.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-08T05:43:07.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.915+0000 7ff494200700 1 Processor -- start 2024-05-08T05:43:07.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.916+0000 7ff494200700 1 -- start start 2024-05-08T05:43:07.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.916+0000 7ff494200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff48c10b070 0x7ff48c10b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:07.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.916+0000 7ff494200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c10ba60 con 0x7ff48c10b070 2024-05-08T05:43:07.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.916+0000 7ff491800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff48c10b070 0x7ff48c10b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:07.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.916+0000 7ff491800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff48c10b070 0x7ff48c10b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45248/0 (socket says 172.21.15.112:45248) 2024-05-08T05:43:07.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.916+0000 7ff491800700 1 -- 172.21.15.112:0/2692633668 learned_addr learned my addr 172.21.15.112:0/2692633668 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:07.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.917+0000 7ff491800700 1 -- 172.21.15.112:0/2692633668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48c10c270 con 0x7ff48c10b070 2024-05-08T05:43:07.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.917+0000 7ff491800700 1 --2- 172.21.15.112:0/2692633668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff48c10b070 0x7ff48c10b490 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff4800098a0 tx=0x7ff480009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0935179ed49bd94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:07.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.917+0000 7ff48be00700 1 -- 172.21.15.112:0/2692633668 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff480004160 con 0x7ff48c10b070 2024-05-08T05:43:07.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.917+0000 7ff48be00700 1 -- 172.21.15.112:0/2692633668 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff480004300 con 0x7ff48c10b070 2024-05-08T05:43:07.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.917+0000 7ff48be00700 1 -- 172.21.15.112:0/2692633668 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff480011a20 con 0x7ff48c10b070 2024-05-08T05:43:07.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.918+0000 7ff494200700 1 -- 172.21.15.112:0/2692633668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff48c10b070 msgr2=0x7ff48c10b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:07.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.918+0000 7ff494200700 1 --2- 172.21.15.112:0/2692633668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff48c10b070 0x7ff48c10b490 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff4800098a0 tx=0x7ff480009bb0 comp rx=0 tx=0).stop 2024-05-08T05:43:07.073 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.918+0000 7ff494200700 1 -- 172.21.15.112:0/2692633668 shutdown_connections 2024-05-08T05:43:07.073 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.918+0000 7ff494200700 1 --2- 172.21.15.112:0/2692633668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff48c10b070 0x7ff48c10b490 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:07.073 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.918+0000 7ff494200700 1 -- 172.21.15.112:0/2692633668 >> 172.21.15.112:0/2692633668 conn(0x7ff48c07e440 msgr2=0x7ff48c07e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:07.073 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.918+0000 7ff494200700 1 -- 172.21.15.112:0/2692633668 shutdown_connections 2024-05-08T05:43:07.073 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.918+0000 7ff494200700 1 -- 172.21.15.112:0/2692633668 wait complete. 2024-05-08T05:43:07.073 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.919+0000 7ff494200700 1 Processor -- start 2024-05-08T05:43:07.073 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.920+0000 7ff494200700 1 -- start start 2024-05-08T05:43:07.073 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.920+0000 7ff494200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff48c10b070 0x7ff48c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:07.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.920+0000 7ff494200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c10f620 con 0x7ff48c10b070 2024-05-08T05:43:07.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.920+0000 7ff491800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff48c10b070 0x7ff48c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:07.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.920+0000 7ff491800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff48c10b070 0x7ff48c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45252/0 (socket says 172.21.15.112:45252) 2024-05-08T05:43:07.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.920+0000 7ff491800700 1 -- 172.21.15.112:0/2089709632 learned_addr learned my addr 172.21.15.112:0/2089709632 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:07.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.920+0000 7ff491800700 1 -- 172.21.15.112:0/2089709632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48c19f500 con 0x7ff48c10b070 2024-05-08T05:43:07.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.921+0000 7ff491800700 1 --2- 172.21.15.112:0/2089709632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff48c10b070 0x7ff48c19efc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff480000c00 tx=0x7ff480011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:07.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.921+0000 7ff48a000700 1 -- 172.21.15.112:0/2089709632 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff480003aa0 con 0x7ff48c10b070 2024-05-08T05:43:07.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.921+0000 7ff48a000700 1 -- 172.21.15.112:0/2089709632 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff48001b8f0 con 0x7ff48c10b070 2024-05-08T05:43:07.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.921+0000 7ff494200700 1 -- 172.21.15.112:0/2089709632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48c19f790 con 0x7ff48c10b070 2024-05-08T05:43:07.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.921+0000 7ff48a000700 1 -- 172.21.15.112:0/2089709632 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff480023890 con 0x7ff48c10b070 2024-05-08T05:43:07.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.921+0000 7ff494200700 1 -- 172.21.15.112:0/2089709632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48c1a2480 con 0x7ff48c10b070 2024-05-08T05:43:07.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.922+0000 7ff48a000700 1 -- 172.21.15.112:0/2089709632 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7ff48001b440 con 0x7ff48c10b070 2024-05-08T05:43:07.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.922+0000 7ff494200700 1 -- 172.21.15.112:0/2089709632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff48c064b00 con 0x7ff48c10b070 2024-05-08T05:43:07.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.923+0000 7ff48a000700 1 --2- 172.21.15.112:0/2089709632 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff47403e520 0x7ff4740409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:07.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.923+0000 7ff48a000700 1 -- 172.21.15.112:0/2089709632 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff480053fc0 con 0x7ff48c10b070 2024-05-08T05:43:07.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.923+0000 7ff490e00700 1 --2- 172.21.15.112:0/2089709632 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff47403e520 0x7ff4740409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:07.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.924+0000 7ff490e00700 1 --2- 172.21.15.112:0/2089709632 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff47403e520 0x7ff4740409e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff478006fd0 tx=0x7ff478006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:07.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:05.929+0000 7ff48a000700 1 -- 172.21.15.112:0/2089709632 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff48001f030 con 0x7ff48c10b070 2024-05-08T05:43:07.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:06.168+0000 7ff494200700 1 -- 172.21.15.112:0/2089709632 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7ff48c108b20 con 0x7ff47403e520 2024-05-08T05:43:07.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:06.402+0000 7ff48a000700 1 -- 172.21.15.112:0/2089709632 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7ff48c108b20 con 0x7ff47403e520 2024-05-08T05:43:07.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:06.405+0000 7ff494200700 1 -- 172.21.15.112:0/2089709632 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff47403e520 msgr2=0x7ff4740409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:07.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:06.405+0000 7ff494200700 1 --2- 172.21.15.112:0/2089709632 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff47403e520 0x7ff4740409e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff478006fd0 tx=0x7ff478006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:07.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:06.405+0000 7ff494200700 1 -- 172.21.15.112:0/2089709632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff48c10b070 msgr2=0x7ff48c19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:07.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:06.405+0000 7ff494200700 1 --2- 172.21.15.112:0/2089709632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff48c10b070 0x7ff48c19efc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff480000c00 tx=0x7ff480011aa0 comp rx=0 tx=0).stop 2024-05-08T05:43:07.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:06.405+0000 7ff494200700 1 -- 172.21.15.112:0/2089709632 shutdown_connections 2024-05-08T05:43:07.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:06.405+0000 7ff494200700 1 --2- 172.21.15.112:0/2089709632 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff47403e520 0x7ff4740409e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:07.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:06.405+0000 7ff494200700 1 --2- 172.21.15.112:0/2089709632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff48c10b070 0x7ff48c19efc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:07.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:06.405+0000 7ff494200700 1 -- 172.21.15.112:0/2089709632 >> 172.21.15.112:0/2089709632 conn(0x7ff48c07e440 msgr2=0x7ff48c108410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:07.075 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:06.405+0000 7ff494200700 1 -- 172.21.15.112:0/2089709632 shutdown_connections 2024-05-08T05:43:07.076 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:06.406+0000 7ff494200700 1 -- 172.21.15.112:0/2089709632 wait complete. 2024-05-08T05:43:07.076 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial admin user... 2024-05-08T05:43:07.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:07.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:07.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:08.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:08 smithi112 ceph-mon[30003]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:08.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:09.039 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$W85W.RfStaN6OdX5Vl/WXOm7.XnRs.lrjgsGJzeLVOQIhkKIwama6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715146988, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-08T05:43:09.039 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.690+0000 7fc2f1800700 1 Processor -- start 2024-05-08T05:43:09.039 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.690+0000 7fc2f1800700 1 -- start start 2024-05-08T05:43:09.039 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.691+0000 7fc2f1800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec108e30 0x7fc2ec109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.039 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.691+0000 7fc2f1800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec109790 con 0x7fc2ec108e30 2024-05-08T05:43:09.039 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.691+0000 7fc2eaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec108e30 0x7fc2ec109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.039 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.691+0000 7fc2eaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec108e30 0x7fc2ec109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45266/0 (socket says 172.21.15.112:45266) 2024-05-08T05:43:09.039 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.691+0000 7fc2eaa00700 1 -- 172.21.15.112:0/1231303547 learned_addr learned my addr 172.21.15.112:0/1231303547 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:09.039 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.692+0000 7fc2eaa00700 1 -- 172.21.15.112:0/1231303547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ec109f90 con 0x7fc2ec108e30 2024-05-08T05:43:09.039 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.692+0000 7fc2eaa00700 1 --2- 172.21.15.112:0/1231303547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec108e30 0x7fc2ec109250 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0098a0 tx=0x7fc2dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b385a0ad45e57d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.039 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.692+0000 7fc2e9600700 1 -- 172.21.15.112:0/1231303547 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc004160 con 0x7fc2ec108e30 2024-05-08T05:43:09.039 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.692+0000 7fc2e9600700 1 -- 172.21.15.112:0/1231303547 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2dc004300 con 0x7fc2ec108e30 2024-05-08T05:43:09.039 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.692+0000 7fc2e9600700 1 -- 172.21.15.112:0/1231303547 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc011a20 con 0x7fc2ec108e30 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.693+0000 7fc2f1800700 1 -- 172.21.15.112:0/1231303547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec108e30 msgr2=0x7fc2ec109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.693+0000 7fc2f1800700 1 --2- 172.21.15.112:0/1231303547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec108e30 0x7fc2ec109250 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0098a0 tx=0x7fc2dc009bb0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.693+0000 7fc2f1800700 1 -- 172.21.15.112:0/1231303547 shutdown_connections 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.693+0000 7fc2f1800700 1 --2- 172.21.15.112:0/1231303547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec108e30 0x7fc2ec109250 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.693+0000 7fc2f1800700 1 -- 172.21.15.112:0/1231303547 >> 172.21.15.112:0/1231303547 conn(0x7fc2ec1043f0 msgr2=0x7fc2ec106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.693+0000 7fc2f1800700 1 -- 172.21.15.112:0/1231303547 shutdown_connections 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.693+0000 7fc2f1800700 1 -- 172.21.15.112:0/1231303547 wait complete. 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.695+0000 7fc2f1800700 1 Processor -- start 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.695+0000 7fc2f1800700 1 -- start start 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.696+0000 7fc2f1800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec108e30 0x7fc2ec07eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.696+0000 7fc2f1800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec10d3d0 con 0x7fc2ec108e30 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.696+0000 7fc2eaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec108e30 0x7fc2ec07eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.696+0000 7fc2eaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec108e30 0x7fc2ec07eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45280/0 (socket says 172.21.15.112:45280) 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.696+0000 7fc2eaa00700 1 -- 172.21.15.112:0/1803052170 learned_addr learned my addr 172.21.15.112:0/1803052170 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.696+0000 7fc2eaa00700 1 -- 172.21.15.112:0/1803052170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ec07f030 con 0x7fc2ec108e30 2024-05-08T05:43:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.697+0000 7fc2eaa00700 1 --2- 172.21.15.112:0/1803052170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec108e30 0x7fc2ec07eaf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc000c00 tx=0x7fc2dc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.697+0000 7fc2e3400700 1 -- 172.21.15.112:0/1803052170 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc003aa0 con 0x7fc2ec108e30 2024-05-08T05:43:09.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.697+0000 7fc2e3400700 1 -- 172.21.15.112:0/1803052170 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2dc01b8f0 con 0x7fc2ec108e30 2024-05-08T05:43:09.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.697+0000 7fc2f1800700 1 -- 172.21.15.112:0/1803052170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ec07d220 con 0x7fc2ec108e30 2024-05-08T05:43:09.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.697+0000 7fc2e3400700 1 -- 172.21.15.112:0/1803052170 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc023870 con 0x7fc2ec108e30 2024-05-08T05:43:09.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.697+0000 7fc2f1800700 1 -- 172.21.15.112:0/1803052170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ec07d760 con 0x7fc2ec108e30 2024-05-08T05:43:09.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.698+0000 7fc2f1800700 1 -- 172.21.15.112:0/1803052170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ec064b00 con 0x7fc2ec108e30 2024-05-08T05:43:09.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.699+0000 7fc2e3400700 1 -- 172.21.15.112:0/1803052170 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fc2dc023b70 con 0x7fc2ec108e30 2024-05-08T05:43:09.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.699+0000 7fc2e3400700 1 --2- 172.21.15.112:0/1803052170 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc2d003e4c0 0x7fc2d0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.699+0000 7fc2e3400700 1 -- 172.21.15.112:0/1803052170 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc2dc053fd0 con 0x7fc2ec108e30 2024-05-08T05:43:09.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.700+0000 7fc2ea000700 1 --2- 172.21.15.112:0/1803052170 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc2d003e4c0 0x7fc2d0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.700+0000 7fc2ea000700 1 --2- 172.21.15.112:0/1803052170 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc2d003e4c0 0x7fc2d0040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8006fd0 tx=0x7fc2d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.704+0000 7fc2e3400700 1 -- 172.21.15.112:0/1803052170 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc2dc01f030 con 0x7fc2ec108e30 2024-05-08T05:43:09.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:07.958+0000 7fc2f1800700 1 -- 172.21.15.112:0/1803052170 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fc2ec07e0d0 con 0x7fc2d003e4c0 2024-05-08T05:43:09.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:08.207+0000 7fc2e3400700 1 -- 172.21.15.112:0/1803052170 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fc2d800a040 con 0x7fc2d003e4c0 2024-05-08T05:43:09.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:08.210+0000 7fc2f1800700 1 -- 172.21.15.112:0/1803052170 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc2d003e4c0 msgr2=0x7fc2d0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:08.210+0000 7fc2f1800700 1 --2- 172.21.15.112:0/1803052170 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc2d003e4c0 0x7fc2d0040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8006fd0 tx=0x7fc2d8006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:09.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:08.210+0000 7fc2f1800700 1 -- 172.21.15.112:0/1803052170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec108e30 msgr2=0x7fc2ec07eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:08.210+0000 7fc2f1800700 1 --2- 172.21.15.112:0/1803052170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec108e30 0x7fc2ec07eaf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc000c00 tx=0x7fc2dc011aa0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:08.210+0000 7fc2f1800700 1 -- 172.21.15.112:0/1803052170 shutdown_connections 2024-05-08T05:43:09.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:08.210+0000 7fc2f1800700 1 --2- 172.21.15.112:0/1803052170 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc2d003e4c0 0x7fc2d0040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:08.210+0000 7fc2f1800700 1 --2- 172.21.15.112:0/1803052170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec108e30 0x7fc2ec07eaf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:08.210+0000 7fc2f1800700 1 -- 172.21.15.112:0/1803052170 >> 172.21.15.112:0/1803052170 conn(0x7fc2ec1043f0 msgr2=0x7fc2ec104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:09.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:08.210+0000 7fc2f1800700 1 -- 172.21.15.112:0/1803052170 shutdown_connections 2024-05-08T05:43:09.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:08.210+0000 7fc2f1800700 1 -- 172.21.15.112:0/1803052170 wait complete. 2024-05-08T05:43:09.042 INFO:teuthology.orchestra.run.smithi112.stdout:Fetching dashboard port number... 2024-05-08T05:43:10.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:10.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:10 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1153060544' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-08T05:43:11.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:11 smithi112 ceph-mon[30003]: mgrmap e13: x(active, since 6s) 2024-05-08T05:43:11.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 8443 2024-05-08T05:43:11.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.675+0000 7fea5ea00700 1 Processor -- start 2024-05-08T05:43:11.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.675+0000 7fea5ea00700 1 -- start start 2024-05-08T05:43:11.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.675+0000 7fea5ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea58108e30 0x7fea58109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.675+0000 7fea5ea00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea58109790 con 0x7fea58108e30 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.676+0000 7fea57e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea58108e30 0x7fea58109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.676+0000 7fea57e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea58108e30 0x7fea58109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45286/0 (socket says 172.21.15.112:45286) 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.676+0000 7fea57e00700 1 -- 172.21.15.112:0/4019028406 learned_addr learned my addr 172.21.15.112:0/4019028406 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.676+0000 7fea57e00700 1 -- 172.21.15.112:0/4019028406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea58109f90 con 0x7fea58108e30 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.677+0000 7fea57e00700 1 --2- 172.21.15.112:0/4019028406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea58108e30 0x7fea58109250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea480098a0 tx=0x7fea48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d466b395f3ac0fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.677+0000 7fea56a00700 1 -- 172.21.15.112:0/4019028406 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea48004160 con 0x7fea58108e30 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.677+0000 7fea56a00700 1 -- 172.21.15.112:0/4019028406 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea48004300 con 0x7fea58108e30 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.677+0000 7fea56a00700 1 -- 172.21.15.112:0/4019028406 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea48011a20 con 0x7fea58108e30 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.678+0000 7fea5ea00700 1 -- 172.21.15.112:0/4019028406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea58108e30 msgr2=0x7fea58109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.678+0000 7fea5ea00700 1 --2- 172.21.15.112:0/4019028406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea58108e30 0x7fea58109250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea480098a0 tx=0x7fea48009bb0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.678+0000 7fea5ea00700 1 -- 172.21.15.112:0/4019028406 shutdown_connections 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.678+0000 7fea5ea00700 1 --2- 172.21.15.112:0/4019028406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea58108e30 0x7fea58109250 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.678+0000 7fea5ea00700 1 -- 172.21.15.112:0/4019028406 >> 172.21.15.112:0/4019028406 conn(0x7fea581043f0 msgr2=0x7fea58106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.678+0000 7fea5ea00700 1 -- 172.21.15.112:0/4019028406 shutdown_connections 2024-05-08T05:43:11.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.678+0000 7fea5ea00700 1 -- 172.21.15.112:0/4019028406 wait complete. 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.679+0000 7fea5ea00700 1 Processor -- start 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.679+0000 7fea5ea00700 1 -- start start 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.680+0000 7fea5ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea58108e30 0x7fea5819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.680+0000 7fea5ea00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea5810d3d0 con 0x7fea58108e30 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.680+0000 7fea57e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea58108e30 0x7fea5819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.680+0000 7fea57e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea58108e30 0x7fea5819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45294/0 (socket says 172.21.15.112:45294) 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.680+0000 7fea57e00700 1 -- 172.21.15.112:0/1153060544 learned_addr learned my addr 172.21.15.112:0/1153060544 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.680+0000 7fea57e00700 1 -- 172.21.15.112:0/1153060544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5819d2d0 con 0x7fea58108e30 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.680+0000 7fea57e00700 1 --2- 172.21.15.112:0/1153060544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea58108e30 0x7fea5819cd90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fea48000c00 tx=0x7fea48011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.681+0000 7fea54c00700 1 -- 172.21.15.112:0/1153060544 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea48003aa0 con 0x7fea58108e30 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.681+0000 7fea54c00700 1 -- 172.21.15.112:0/1153060544 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea4801b8f0 con 0x7fea58108e30 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.681+0000 7fea5ea00700 1 -- 172.21.15.112:0/1153060544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5819d560 con 0x7fea58108e30 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.681+0000 7fea54c00700 1 -- 172.21.15.112:0/1153060544 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea48023870 con 0x7fea58108e30 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.681+0000 7fea5ea00700 1 -- 172.21.15.112:0/1153060544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea581a0210 con 0x7fea58108e30 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.682+0000 7fea54c00700 1 -- 172.21.15.112:0/1153060544 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fea48023b70 con 0x7fea58108e30 2024-05-08T05:43:11.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.682+0000 7fea5ea00700 1 -- 172.21.15.112:0/1153060544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5810d550 con 0x7fea58108e30 2024-05-08T05:43:11.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.683+0000 7fea54c00700 1 --2- 172.21.15.112:0/1153060544 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fea3c03e4c0 0x7fea3c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.683+0000 7fea54c00700 1 -- 172.21.15.112:0/1153060544 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fea48054040 con 0x7fea58108e30 2024-05-08T05:43:11.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.683+0000 7fea57400700 1 --2- 172.21.15.112:0/1153060544 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fea3c03e4c0 0x7fea3c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.684+0000 7fea57400700 1 --2- 172.21.15.112:0/1153060544 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fea3c03e4c0 0x7fea3c040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea44006fd0 tx=0x7fea44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.689+0000 7fea54c00700 1 -- 172.21.15.112:0/1153060544 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea4801f030 con 0x7fea58108e30 2024-05-08T05:43:11.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.938+0000 7fea5ea00700 1 -- 172.21.15.112:0/1153060544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fea58064b00 con 0x7fea58108e30 2024-05-08T05:43:11.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.939+0000 7fea54c00700 1 -- 172.21.15.112:0/1153060544 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fea4800edc0 con 0x7fea58108e30 2024-05-08T05:43:11.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.942+0000 7fea5ea00700 1 -- 172.21.15.112:0/1153060544 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fea3c03e4c0 msgr2=0x7fea3c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.942+0000 7fea5ea00700 1 --2- 172.21.15.112:0/1153060544 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fea3c03e4c0 0x7fea3c040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea44006fd0 tx=0x7fea44006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:11.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.942+0000 7fea5ea00700 1 -- 172.21.15.112:0/1153060544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea58108e30 msgr2=0x7fea5819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.942+0000 7fea5ea00700 1 --2- 172.21.15.112:0/1153060544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea58108e30 0x7fea5819cd90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fea48000c00 tx=0x7fea48011aa0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.942+0000 7fea5ea00700 1 -- 172.21.15.112:0/1153060544 shutdown_connections 2024-05-08T05:43:11.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.942+0000 7fea5ea00700 1 --2- 172.21.15.112:0/1153060544 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fea3c03e4c0 0x7fea3c040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.942+0000 7fea5ea00700 1 --2- 172.21.15.112:0/1153060544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea58108e30 0x7fea5819cd90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.607 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.942+0000 7fea5ea00700 1 -- 172.21.15.112:0/1153060544 >> 172.21.15.112:0/1153060544 conn(0x7fea581043f0 msgr2=0x7fea58104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:11.607 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.942+0000 7fea5ea00700 1 -- 172.21.15.112:0/1153060544 shutdown_connections 2024-05-08T05:43:11.607 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:09.942+0000 7fea5ea00700 1 -- 172.21.15.112:0/1153060544 wait complete. 2024-05-08T05:43:11.631 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:43:11.632 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-05-08T05:43:11.653 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:43:11.654 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-05-08T05:43:11.654 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-05-08T05:43:11.654 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-08T05:43:11.661 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph Dashboard is now available at: 2024-05-08T05:43:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:43:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: URL: https://smithi112.front.sepia.ceph.com:8443/ 2024-05-08T05:43:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: User: admin 2024-05-08T05:43:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: Password: 5n336ja3bo 2024-05-08T05:43:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:43:11.662 INFO:teuthology.orchestra.run.smithi112.stdout:Saving cluster configuration to /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config directory 2024-05-08T05:43:11.664 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling autotune for osd_memory_target 2024-05-08T05:43:13.158 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.253+0000 7f7d46e00700 1 Processor -- start 2024-05-08T05:43:13.158 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.253+0000 7f7d46e00700 1 -- start start 2024-05-08T05:43:13.158 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.254+0000 7f7d46e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4010ae80 0x7f7d4010b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:13.158 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.254+0000 7f7d46e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4010b7e0 con 0x7f7d4010ae80 2024-05-08T05:43:13.158 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.254+0000 7f7d45a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4010ae80 0x7f7d4010b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:13.158 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.255+0000 7f7d45a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4010ae80 0x7f7d4010b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45302/0 (socket says 172.21.15.112:45302) 2024-05-08T05:43:13.158 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.255+0000 7f7d45a00700 1 -- 172.21.15.112:0/2956536889 learned_addr learned my addr 172.21.15.112:0/2956536889 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:13.158 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.255+0000 7f7d45a00700 1 -- 172.21.15.112:0/2956536889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4010bfe0 con 0x7f7d4010ae80 2024-05-08T05:43:13.159 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.255+0000 7f7d45a00700 1 --2- 172.21.15.112:0/2956536889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4010ae80 0x7f7d4010b2a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d340098a0 tx=0x7f7d34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c321e86ec1c11416 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:13.159 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.255+0000 7f7d3fe00700 1 -- 172.21.15.112:0/2956536889 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d34004160 con 0x7f7d4010ae80 2024-05-08T05:43:13.159 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.256+0000 7f7d3fe00700 1 -- 172.21.15.112:0/2956536889 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d34004300 con 0x7f7d4010ae80 2024-05-08T05:43:13.159 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.256+0000 7f7d3fe00700 1 -- 172.21.15.112:0/2956536889 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d34011a20 con 0x7f7d4010ae80 2024-05-08T05:43:13.159 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.256+0000 7f7d46e00700 1 -- 172.21.15.112:0/2956536889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4010ae80 msgr2=0x7f7d4010b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:13.159 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.256+0000 7f7d46e00700 1 --2- 172.21.15.112:0/2956536889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4010ae80 0x7f7d4010b2a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d340098a0 tx=0x7f7d34009bb0 comp rx=0 tx=0).stop 2024-05-08T05:43:13.159 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.256+0000 7f7d46e00700 1 -- 172.21.15.112:0/2956536889 shutdown_connections 2024-05-08T05:43:13.159 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.256+0000 7f7d46e00700 1 --2- 172.21.15.112:0/2956536889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4010ae80 0x7f7d4010b2a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:13.159 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.256+0000 7f7d46e00700 1 -- 172.21.15.112:0/2956536889 >> 172.21.15.112:0/2956536889 conn(0x7f7d4007e250 msgr2=0x7f7d4007e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:13.159 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.256+0000 7f7d46e00700 1 -- 172.21.15.112:0/2956536889 shutdown_connections 2024-05-08T05:43:13.159 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.256+0000 7f7d46e00700 1 -- 172.21.15.112:0/2956536889 wait complete. 2024-05-08T05:43:13.159 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.257+0000 7f7d46e00700 1 Processor -- start 2024-05-08T05:43:13.159 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.257+0000 7f7d46e00700 1 -- start start 2024-05-08T05:43:13.159 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.258+0000 7f7d46e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4010ae80 0x7f7d4019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:13.159 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.258+0000 7f7d46e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4010f420 con 0x7f7d4010ae80 2024-05-08T05:43:13.160 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.258+0000 7f7d45a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4010ae80 0x7f7d4019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:13.160 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.258+0000 7f7d45a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4010ae80 0x7f7d4019edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45312/0 (socket says 172.21.15.112:45312) 2024-05-08T05:43:13.160 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.258+0000 7f7d45a00700 1 -- 172.21.15.112:0/2229717181 learned_addr learned my addr 172.21.15.112:0/2229717181 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:13.160 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.258+0000 7f7d45a00700 1 -- 172.21.15.112:0/2229717181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4019f310 con 0x7f7d4010ae80 2024-05-08T05:43:13.160 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.259+0000 7f7d45a00700 1 --2- 172.21.15.112:0/2229717181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4010ae80 0x7f7d4019edd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d34004660 tx=0x7f7d34011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:13.160 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.259+0000 7f7d3e000700 1 -- 172.21.15.112:0/2229717181 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d34003aa0 con 0x7f7d4010ae80 2024-05-08T05:43:13.160 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.259+0000 7f7d3e000700 1 -- 172.21.15.112:0/2229717181 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d3401b8f0 con 0x7f7d4010ae80 2024-05-08T05:43:13.160 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.259+0000 7f7d3e000700 1 -- 172.21.15.112:0/2229717181 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d34023900 con 0x7f7d4010ae80 2024-05-08T05:43:13.160 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.259+0000 7f7d46e00700 1 -- 172.21.15.112:0/2229717181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4019f5a0 con 0x7f7d4010ae80 2024-05-08T05:43:13.160 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.259+0000 7f7d46e00700 1 -- 172.21.15.112:0/2229717181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d401a2290 con 0x7f7d4010ae80 2024-05-08T05:43:13.160 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.260+0000 7f7d46e00700 1 -- 172.21.15.112:0/2229717181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d4010f5a0 con 0x7f7d4010ae80 2024-05-08T05:43:13.160 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.260+0000 7f7d3e000700 1 -- 172.21.15.112:0/2229717181 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f7d3401baa0 con 0x7f7d4010ae80 2024-05-08T05:43:13.160 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.261+0000 7f7d3e000700 1 --2- 172.21.15.112:0/2229717181 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7d2803e5c0 0x7f7d28040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.261+0000 7f7d3e000700 1 -- 172.21.15.112:0/2229717181 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7d34053f50 con 0x7f7d4010ae80 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.261+0000 7f7d45000700 1 --2- 172.21.15.112:0/2229717181 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7d2803e5c0 0x7f7d28040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.262+0000 7f7d45000700 1 --2- 172.21.15.112:0/2229717181 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7d2803e5c0 0x7f7d28040a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c006fd0 tx=0x7f7d2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.266+0000 7f7d3e000700 1 -- 172.21.15.112:0/2229717181 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d3401f030 con 0x7f7d4010ae80 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.493+0000 7f7d46e00700 1 -- 172.21.15.112:0/2229717181 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f7d401a2900 con 0x7f7d4010ae80 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.494+0000 7f7d3e000700 1 -- 172.21.15.112:0/2229717181 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f7d34018350 con 0x7f7d4010ae80 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.497+0000 7f7d46e00700 1 -- 172.21.15.112:0/2229717181 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7d2803e5c0 msgr2=0x7f7d28040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.497+0000 7f7d46e00700 1 --2- 172.21.15.112:0/2229717181 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7d2803e5c0 0x7f7d28040a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c006fd0 tx=0x7f7d2c006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.497+0000 7f7d46e00700 1 -- 172.21.15.112:0/2229717181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4010ae80 msgr2=0x7f7d4019edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.497+0000 7f7d46e00700 1 --2- 172.21.15.112:0/2229717181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4010ae80 0x7f7d4019edd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d34004660 tx=0x7f7d34011aa0 comp rx=0 tx=0).stop 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.498+0000 7f7d46e00700 1 -- 172.21.15.112:0/2229717181 shutdown_connections 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.498+0000 7f7d46e00700 1 --2- 172.21.15.112:0/2229717181 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7d2803e5c0 0x7f7d28040a80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.498+0000 7f7d46e00700 1 --2- 172.21.15.112:0/2229717181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d4010ae80 0x7f7d4019edd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.498+0000 7f7d46e00700 1 -- 172.21.15.112:0/2229717181 >> 172.21.15.112:0/2229717181 conn(0x7f7d4007e250 msgr2=0x7f7d40108400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.498+0000 7f7d46e00700 1 -- 172.21.15.112:0/2229717181 shutdown_connections 2024-05-08T05:43:13.161 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:12.498+0000 7f7d46e00700 1 -- 172.21.15.112:0/2229717181 wait complete. 2024-05-08T05:43:14.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.768+0000 7fc61a200700 1 Processor -- start 2024-05-08T05:43:14.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.769+0000 7fc61a200700 1 -- start start 2024-05-08T05:43:14.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.769+0000 7fc61a200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc614109910 0x7fc61410bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:14.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.769+0000 7fc61a200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc61407cdb0 con 0x7fc614109910 2024-05-08T05:43:14.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.769+0000 7fc613400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc614109910 0x7fc61410bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:14.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.770+0000 7fc613400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc614109910 0x7fc61410bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45316/0 (socket says 172.21.15.112:45316) 2024-05-08T05:43:14.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.770+0000 7fc613400700 1 -- 172.21.15.112:0/2477145387 learned_addr learned my addr 172.21.15.112:0/2477145387 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:14.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.770+0000 7fc613400700 1 -- 172.21.15.112:0/2477145387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61410c240 con 0x7fc614109910 2024-05-08T05:43:14.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.770+0000 7fc613400700 1 --2- 172.21.15.112:0/2477145387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc614109910 0x7fc61410bd00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6040098a0 tx=0x7fc604009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d2e83431ccc359 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:14.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.771+0000 7fc612000700 1 -- 172.21.15.112:0/2477145387 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc604004160 con 0x7fc614109910 2024-05-08T05:43:14.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.771+0000 7fc612000700 1 -- 172.21.15.112:0/2477145387 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc604004300 con 0x7fc614109910 2024-05-08T05:43:14.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.771+0000 7fc612000700 1 -- 172.21.15.112:0/2477145387 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc604011a20 con 0x7fc614109910 2024-05-08T05:43:14.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.771+0000 7fc61a200700 1 -- 172.21.15.112:0/2477145387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc614109910 msgr2=0x7fc61410bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:14.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.771+0000 7fc61a200700 1 --2- 172.21.15.112:0/2477145387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc614109910 0x7fc61410bd00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6040098a0 tx=0x7fc604009bb0 comp rx=0 tx=0).stop 2024-05-08T05:43:14.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.772+0000 7fc61a200700 1 -- 172.21.15.112:0/2477145387 shutdown_connections 2024-05-08T05:43:14.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.772+0000 7fc61a200700 1 --2- 172.21.15.112:0/2477145387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc614109910 0x7fc61410bd00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:14.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.772+0000 7fc61a200700 1 -- 172.21.15.112:0/2477145387 >> 172.21.15.112:0/2477145387 conn(0x7fc6141032c0 msgr2=0x7fc614105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:14.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.772+0000 7fc61a200700 1 -- 172.21.15.112:0/2477145387 shutdown_connections 2024-05-08T05:43:14.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.772+0000 7fc61a200700 1 -- 172.21.15.112:0/2477145387 wait complete. 2024-05-08T05:43:14.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.773+0000 7fc61a200700 1 Processor -- start 2024-05-08T05:43:14.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.773+0000 7fc61a200700 1 -- start start 2024-05-08T05:43:14.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.773+0000 7fc61a200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc614109910 0x7fc61419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:14.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.773+0000 7fc61a200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc61410d1e0 con 0x7fc614109910 2024-05-08T05:43:14.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.773+0000 7fc613400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc614109910 0x7fc61419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:14.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.773+0000 7fc613400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc614109910 0x7fc61419cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45324/0 (socket says 172.21.15.112:45324) 2024-05-08T05:43:14.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.773+0000 7fc613400700 1 -- 172.21.15.112:0/2053889291 learned_addr learned my addr 172.21.15.112:0/2053889291 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:14.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.774+0000 7fc613400700 1 -- 172.21.15.112:0/2053889291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61419d250 con 0x7fc614109910 2024-05-08T05:43:14.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.774+0000 7fc613400700 1 --2- 172.21.15.112:0/2053889291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc614109910 0x7fc61419cd10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc6040116e0 tx=0x7fc604011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:14.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.774+0000 7fc60be00700 1 -- 172.21.15.112:0/2053889291 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc604003a60 con 0x7fc614109910 2024-05-08T05:43:14.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.774+0000 7fc60be00700 1 -- 172.21.15.112:0/2053889291 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc60401b8f0 con 0x7fc614109910 2024-05-08T05:43:14.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.774+0000 7fc61a200700 1 -- 172.21.15.112:0/2053889291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61419d4e0 con 0x7fc614109910 2024-05-08T05:43:14.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.774+0000 7fc60be00700 1 -- 172.21.15.112:0/2053889291 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc604019860 con 0x7fc614109910 2024-05-08T05:43:14.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.775+0000 7fc61a200700 1 -- 172.21.15.112:0/2053889291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6141a01c0 con 0x7fc614109910 2024-05-08T05:43:14.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.776+0000 7fc61a200700 1 -- 172.21.15.112:0/2053889291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc614064b00 con 0x7fc614109910 2024-05-08T05:43:14.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.776+0000 7fc60be00700 1 -- 172.21.15.112:0/2053889291 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fc604019b60 con 0x7fc614109910 2024-05-08T05:43:14.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.776+0000 7fc60be00700 1 --2- 172.21.15.112:0/2053889291 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc5f803e510 0x7fc5f80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:14.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.776+0000 7fc60be00700 1 -- 172.21.15.112:0/2053889291 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc604053f40 con 0x7fc614109910 2024-05-08T05:43:14.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.777+0000 7fc612a00700 1 --2- 172.21.15.112:0/2053889291 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc5f803e510 0x7fc5f80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:14.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.777+0000 7fc612a00700 1 --2- 172.21.15.112:0/2053889291 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc5f803e510 0x7fc5f80409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc600006fd0 tx=0x7fc600006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:14.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:13.783+0000 7fc60be00700 1 -- 172.21.15.112:0/2053889291 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc60401f030 con 0x7fc614109910 2024-05-08T05:43:14.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:14.134+0000 7fc61a200700 1 -- 172.21.15.112:0/2053889291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fc6141a0770 con 0x7fc614109910 2024-05-08T05:43:14.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:14.140+0000 7fc60be00700 1 -- 172.21.15.112:0/2053889291 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fc604028de0 con 0x7fc614109910 2024-05-08T05:43:14.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:14.143+0000 7fc61a200700 1 -- 172.21.15.112:0/2053889291 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc5f803e510 msgr2=0x7fc5f80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:14.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:14.143+0000 7fc61a200700 1 --2- 172.21.15.112:0/2053889291 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc5f803e510 0x7fc5f80409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc600006fd0 tx=0x7fc600006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:14.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:14.143+0000 7fc61a200700 1 -- 172.21.15.112:0/2053889291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc614109910 msgr2=0x7fc61419cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:14.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:14.143+0000 7fc61a200700 1 --2- 172.21.15.112:0/2053889291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc614109910 0x7fc61419cd10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc6040116e0 tx=0x7fc604011aa0 comp rx=0 tx=0).stop 2024-05-08T05:43:14.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:14.143+0000 7fc61a200700 1 -- 172.21.15.112:0/2053889291 shutdown_connections 2024-05-08T05:43:14.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:14.143+0000 7fc61a200700 1 --2- 172.21.15.112:0/2053889291 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc5f803e510 0x7fc5f80409d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:14.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:14.143+0000 7fc61a200700 1 --2- 172.21.15.112:0/2053889291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc614109910 0x7fc61419cd10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:14.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:14.143+0000 7fc61a200700 1 -- 172.21.15.112:0/2053889291 >> 172.21.15.112:0/2053889291 conn(0x7fc6141032c0 msgr2=0x7fc614103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:14.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:14.143+0000 7fc61a200700 1 -- 172.21.15.112:0/2053889291 shutdown_connections 2024-05-08T05:43:14.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-05-08T05:43:14.144+0000 7fc61a200700 1 -- 172.21.15.112:0/2053889291 wait complete. 2024-05-08T05:43:14.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-08T05:43:14.839 INFO:teuthology.orchestra.run.smithi112.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-08T05:43:14.839 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:43:14.839 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:14.839 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:43:14.839 INFO:teuthology.orchestra.run.smithi112.stdout:Or, if you are only running a single cluster on this host: 2024-05-08T05:43:14.840 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:43:14.840 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-08T05:43:14.840 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:43:14.840 INFO:teuthology.orchestra.run.smithi112.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-08T05:43:14.840 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:43:14.840 INFO:teuthology.orchestra.run.smithi112.stdout: ceph telemetry on 2024-05-08T05:43:14.840 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:43:14.840 INFO:teuthology.orchestra.run.smithi112.stdout:For more information see: 2024-05-08T05:43:14.840 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:43:14.840 INFO:teuthology.orchestra.run.smithi112.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-08T05:43:14.840 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:43:14.840 INFO:teuthology.orchestra.run.smithi112.stdout:Bootstrap complete. 2024-05-08T05:43:14.902 INFO:tasks.cephadm:Fetching config... 2024-05-08T05:43:14.902 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:43:14.903 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-08T05:43:14.946 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-08T05:43:14.946 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:43:14.947 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-08T05:43:15.026 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-08T05:43:15.026 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:43:15.027 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-08T05:43:15.145 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-08T05:43:15.145 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:43:15.145 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-08T05:43:15.239 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-08T05:43:15.240 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWR6ZAu8e+ffNgTQdsp2CLBJsPd4Ycek0h0c4u/yRD4ZCZW1INIMUSw6NWygDqH2wC+MKVO0HtrBnX37KinGtkpi5506m+uvRRPbr97eps4BmXM9AgTXofBx45PGseaZwiEmwWQ9aXJm4+s+1GakK9KIL43K5mPAS5gnVfa1Q0QqyA2O+iVU1qEIKxrbiJ0zrQDYKw8y0zyvKiJa8oalj+4a12+ySeKyTIsDf6FXZRnwrGB2aOmvX34KC4DPXhmByKTzgRrPTG8igsvIfmc73ourCe13IvI+92DoHXMvfYlbWyHm9ydyTBSkF0TQJ6g2VUVRbbQhTbO7mP8jIQb35wXH5nyTWRPZzpMZsfuyVoEgQTXS7DsLmXIQ1SyWo9TnUjY8eK5RoyA/E7cKt6aFoK3pWD5bLzTPMM29pMC/S+6HcDWOmnrhl0cmsWAAYrKkWRR+V6qsElBjg6ZAzGss66P1bHs+fN0aw6Q1ZRZqurUqzWWaNNyqXN1Kw978oYht8= ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T05:43:15.359 INFO:teuthology.orchestra.run.smithi112.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWR6ZAu8e+ffNgTQdsp2CLBJsPd4Ycek0h0c4u/yRD4ZCZW1INIMUSw6NWygDqH2wC+MKVO0HtrBnX37KinGtkpi5506m+uvRRPbr97eps4BmXM9AgTXofBx45PGseaZwiEmwWQ9aXJm4+s+1GakK9KIL43K5mPAS5gnVfa1Q0QqyA2O+iVU1qEIKxrbiJ0zrQDYKw8y0zyvKiJa8oalj+4a12+ySeKyTIsDf6FXZRnwrGB2aOmvX34KC4DPXhmByKTzgRrPTG8igsvIfmc73ourCe13IvI+92DoHXMvfYlbWyHm9ydyTBSkF0TQJ6g2VUVRbbQhTbO7mP8jIQb35wXH5nyTWRPZzpMZsfuyVoEgQTXS7DsLmXIQ1SyWo9TnUjY8eK5RoyA/E7cKt6aFoK3pWD5bLzTPMM29pMC/S+6HcDWOmnrhl0cmsWAAYrKkWRR+V6qsElBjg6ZAzGss66P1bHs+fN0aw6Q1ZRZqurUqzWWaNNyqXN1Kw978oYht8= ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:43:15.360 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:15 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2053889291' entity='client.admin' 2024-05-08T05:43:15.397 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWR6ZAu8e+ffNgTQdsp2CLBJsPd4Ycek0h0c4u/yRD4ZCZW1INIMUSw6NWygDqH2wC+MKVO0HtrBnX37KinGtkpi5506m+uvRRPbr97eps4BmXM9AgTXofBx45PGseaZwiEmwWQ9aXJm4+s+1GakK9KIL43K5mPAS5gnVfa1Q0QqyA2O+iVU1qEIKxrbiJ0zrQDYKw8y0zyvKiJa8oalj+4a12+ySeKyTIsDf6FXZRnwrGB2aOmvX34KC4DPXhmByKTzgRrPTG8igsvIfmc73ourCe13IvI+92DoHXMvfYlbWyHm9ydyTBSkF0TQJ6g2VUVRbbQhTbO7mP8jIQb35wXH5nyTWRPZzpMZsfuyVoEgQTXS7DsLmXIQ1SyWo9TnUjY8eK5RoyA/E7cKt6aFoK3pWD5bLzTPMM29pMC/S+6HcDWOmnrhl0cmsWAAYrKkWRR+V6qsElBjg6ZAzGss66P1bHs+fN0aw6Q1ZRZqurUqzWWaNNyqXN1Kw978oYht8= ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T05:43:15.439 INFO:teuthology.orchestra.run.smithi150.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWR6ZAu8e+ffNgTQdsp2CLBJsPd4Ycek0h0c4u/yRD4ZCZW1INIMUSw6NWygDqH2wC+MKVO0HtrBnX37KinGtkpi5506m+uvRRPbr97eps4BmXM9AgTXofBx45PGseaZwiEmwWQ9aXJm4+s+1GakK9KIL43K5mPAS5gnVfa1Q0QqyA2O+iVU1qEIKxrbiJ0zrQDYKw8y0zyvKiJa8oalj+4a12+ySeKyTIsDf6FXZRnwrGB2aOmvX34KC4DPXhmByKTzgRrPTG8igsvIfmc73ourCe13IvI+92DoHXMvfYlbWyHm9ydyTBSkF0TQJ6g2VUVRbbQhTbO7mP8jIQb35wXH5nyTWRPZzpMZsfuyVoEgQTXS7DsLmXIQ1SyWo9TnUjY8eK5RoyA/E7cKt6aFoK3pWD5bLzTPMM29pMC/S+6HcDWOmnrhl0cmsWAAYrKkWRR+V6qsElBjg6ZAzGss66P1bHs+fN0aw6Q1ZRZqurUqzWWaNNyqXN1Kw978oYht8= ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:43:15.451 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWR6ZAu8e+ffNgTQdsp2CLBJsPd4Ycek0h0c4u/yRD4ZCZW1INIMUSw6NWygDqH2wC+MKVO0HtrBnX37KinGtkpi5506m+uvRRPbr97eps4BmXM9AgTXofBx45PGseaZwiEmwWQ9aXJm4+s+1GakK9KIL43K5mPAS5gnVfa1Q0QqyA2O+iVU1qEIKxrbiJ0zrQDYKw8y0zyvKiJa8oalj+4a12+ySeKyTIsDf6FXZRnwrGB2aOmvX34KC4DPXhmByKTzgRrPTG8igsvIfmc73ourCe13IvI+92DoHXMvfYlbWyHm9ydyTBSkF0TQJ6g2VUVRbbQhTbO7mP8jIQb35wXH5nyTWRPZzpMZsfuyVoEgQTXS7DsLmXIQ1SyWo9TnUjY8eK5RoyA/E7cKt6aFoK3pWD5bLzTPMM29pMC/S+6HcDWOmnrhl0cmsWAAYrKkWRR+V6qsElBjg6ZAzGss66P1bHs+fN0aw6Q1ZRZqurUqzWWaNNyqXN1Kw978oYht8= ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T05:43:15.492 INFO:teuthology.orchestra.run.smithi162.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWR6ZAu8e+ffNgTQdsp2CLBJsPd4Ycek0h0c4u/yRD4ZCZW1INIMUSw6NWygDqH2wC+MKVO0HtrBnX37KinGtkpi5506m+uvRRPbr97eps4BmXM9AgTXofBx45PGseaZwiEmwWQ9aXJm4+s+1GakK9KIL43K5mPAS5gnVfa1Q0QqyA2O+iVU1qEIKxrbiJ0zrQDYKw8y0zyvKiJa8oalj+4a12+ySeKyTIsDf6FXZRnwrGB2aOmvX34KC4DPXhmByKTzgRrPTG8igsvIfmc73ourCe13IvI+92DoHXMvfYlbWyHm9ydyTBSkF0TQJ6g2VUVRbbQhTbO7mP8jIQb35wXH5nyTWRPZzpMZsfuyVoEgQTXS7DsLmXIQ1SyWo9TnUjY8eK5RoyA/E7cKt6aFoK3pWD5bLzTPMM29pMC/S+6HcDWOmnrhl0cmsWAAYrKkWRR+V6qsElBjg6ZAzGss66P1bHs+fN0aw6Q1ZRZqurUqzWWaNNyqXN1Kw978oYht8= ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:43:15.504 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-08T05:43:15.911 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:43:18.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.520+0000 7faf4be00700 1 -- 172.21.15.112:0/2358205779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf4c104f10 msgr2=0x7faf4c105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:18.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.520+0000 7faf4be00700 1 --2- 172.21.15.112:0/2358205779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf4c104f10 0x7faf4c105350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009910 tx=0x7faf3c009c20 comp rx=0 tx=0).stop 2024-05-08T05:43:18.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.521+0000 7faf4be00700 1 -- 172.21.15.112:0/2358205779 shutdown_connections 2024-05-08T05:43:18.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.521+0000 7faf4be00700 1 --2- 172.21.15.112:0/2358205779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf4c104f10 0x7faf4c105350 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:18.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.521+0000 7faf4be00700 1 -- 172.21.15.112:0/2358205779 >> 172.21.15.112:0/2358205779 conn(0x7faf4c100500 msgr2=0x7faf4c102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:18.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.521+0000 7faf4be00700 1 -- 172.21.15.112:0/2358205779 shutdown_connections 2024-05-08T05:43:18.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.521+0000 7faf4be00700 1 -- 172.21.15.112:0/2358205779 wait complete. 2024-05-08T05:43:18.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.522+0000 7faf4be00700 1 Processor -- start 2024-05-08T05:43:18.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.522+0000 7faf4be00700 1 -- start start 2024-05-08T05:43:18.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.523+0000 7faf4be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf4c104f10 0x7faf4c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:18.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.523+0000 7faf4be00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c10ae00 con 0x7faf4c104f10 2024-05-08T05:43:18.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.523+0000 7faf4aa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf4c104f10 0x7faf4c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:18.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.523+0000 7faf4aa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf4c104f10 0x7faf4c19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57978/0 (socket says 172.21.15.112:57978) 2024-05-08T05:43:18.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.523+0000 7faf4aa00700 1 -- 172.21.15.112:0/4101376776 learned_addr learned my addr 172.21.15.112:0/4101376776 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:18.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.523+0000 7faf4aa00700 1 -- 172.21.15.112:0/4101376776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4c19ab10 con 0x7faf4c104f10 2024-05-08T05:43:18.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.524+0000 7faf4aa00700 1 --2- 172.21.15.112:0/4101376776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf4c104f10 0x7faf4c19a5d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf3c005800 tx=0x7faf3c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:18.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.524+0000 7faf50a00700 1 -- 172.21.15.112:0/4101376776 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf3c027070 con 0x7faf4c104f10 2024-05-08T05:43:18.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.524+0000 7faf4be00700 1 -- 172.21.15.112:0/4101376776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4c19ada0 con 0x7faf4c104f10 2024-05-08T05:43:18.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.524+0000 7faf50a00700 1 -- 172.21.15.112:0/4101376776 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf3c00be10 con 0x7faf4c104f10 2024-05-08T05:43:18.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.524+0000 7faf50a00700 1 -- 172.21.15.112:0/4101376776 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf3c00f460 con 0x7faf4c104f10 2024-05-08T05:43:18.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.524+0000 7faf4be00700 1 -- 172.21.15.112:0/4101376776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf4c19da40 con 0x7faf4c104f10 2024-05-08T05:43:18.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.525+0000 7faf4be00700 1 -- 172.21.15.112:0/4101376776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf4c10b580 con 0x7faf4c104f10 2024-05-08T05:43:18.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.525+0000 7faf50a00700 1 -- 172.21.15.112:0/4101376776 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7faf3c02a070 con 0x7faf4c104f10 2024-05-08T05:43:18.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.526+0000 7faf50a00700 1 --2- 172.21.15.112:0/4101376776 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7faf3003e510 0x7faf300409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:18.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.526+0000 7faf50a00700 1 -- 172.21.15.112:0/4101376776 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faf3c01e030 con 0x7faf4c104f10 2024-05-08T05:43:18.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.526+0000 7faf4a000700 1 --2- 172.21.15.112:0/4101376776 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7faf3003e510 0x7faf300409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:18.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.527+0000 7faf4a000700 1 --2- 172.21.15.112:0/4101376776 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7faf3003e510 0x7faf300409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faf38006fd0 tx=0x7faf38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:18.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.532+0000 7faf50a00700 1 -- 172.21.15.112:0/4101376776 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faf3c013070 con 0x7faf4c104f10 2024-05-08T05:43:18.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.760+0000 7faf4be00700 1 -- 172.21.15.112:0/4101376776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7faf4c02e620 con 0x7faf4c104f10 2024-05-08T05:43:18.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.765+0000 7faf50a00700 1 -- 172.21.15.112:0/4101376776 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7faf3c0310c0 con 0x7faf4c104f10 2024-05-08T05:43:18.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.768+0000 7faf4be00700 1 -- 172.21.15.112:0/4101376776 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7faf3003e510 msgr2=0x7faf300409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:18.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.768+0000 7faf4be00700 1 --2- 172.21.15.112:0/4101376776 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7faf3003e510 0x7faf300409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faf38006fd0 tx=0x7faf38006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:18.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.768+0000 7faf4be00700 1 -- 172.21.15.112:0/4101376776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf4c104f10 msgr2=0x7faf4c19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:18.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.768+0000 7faf4be00700 1 --2- 172.21.15.112:0/4101376776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf4c104f10 0x7faf4c19a5d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf3c005800 tx=0x7faf3c009f60 comp rx=0 tx=0).stop 2024-05-08T05:43:18.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.768+0000 7faf4be00700 1 -- 172.21.15.112:0/4101376776 shutdown_connections 2024-05-08T05:43:18.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.768+0000 7faf4be00700 1 --2- 172.21.15.112:0/4101376776 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7faf3003e510 0x7faf300409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:18.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.768+0000 7faf4be00700 1 --2- 172.21.15.112:0/4101376776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf4c104f10 0x7faf4c19a5d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:18.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.768+0000 7faf4be00700 1 -- 172.21.15.112:0/4101376776 >> 172.21.15.112:0/4101376776 conn(0x7faf4c100500 msgr2=0x7faf4c101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:18.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.769+0000 7faf4be00700 1 -- 172.21.15.112:0/4101376776 shutdown_connections 2024-05-08T05:43:18.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:18.769+0000 7faf4be00700 1 -- 172.21.15.112:0/4101376776 wait complete. 2024-05-08T05:43:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:43:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:19.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:19.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:18 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/4101376776' entity='client.admin' 2024-05-08T05:43:19.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:19.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:19.512 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-08T05:43:19.512 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-08T05:43:19.949 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:43:22.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.017+0000 7f565b000700 1 -- 172.21.15.112:0/2477862737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56541096b0 msgr2=0x7f5654109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:22.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.017+0000 7f565b000700 1 --2- 172.21.15.112:0/2477862737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56541096b0 0x7f5654109ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5644009910 tx=0x7f5644009c20 comp rx=0 tx=0).stop 2024-05-08T05:43:22.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.017+0000 7f565b000700 1 -- 172.21.15.112:0/2477862737 shutdown_connections 2024-05-08T05:43:22.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.017+0000 7f565b000700 1 --2- 172.21.15.112:0/2477862737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56541096b0 0x7f5654109ad0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:22.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.017+0000 7f565b000700 1 -- 172.21.15.112:0/2477862737 >> 172.21.15.112:0/2477862737 conn(0x7f5654078790 msgr2=0x7f565407abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:22.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.017+0000 7f565b000700 1 -- 172.21.15.112:0/2477862737 shutdown_connections 2024-05-08T05:43:22.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.017+0000 7f565b000700 1 -- 172.21.15.112:0/2477862737 wait complete. 2024-05-08T05:43:22.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.019+0000 7f565b000700 1 Processor -- start 2024-05-08T05:43:22.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.019+0000 7f565b000700 1 -- start start 2024-05-08T05:43:22.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.019+0000 7f565b000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56541096b0 0x7f565419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:22.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.019+0000 7f565b000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f565410f3d0 con 0x7f56541096b0 2024-05-08T05:43:22.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.020+0000 7f5653e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56541096b0 0x7f565419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.020+0000 7f5653e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56541096b0 0x7f565419eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57998/0 (socket says 172.21.15.112:57998) 2024-05-08T05:43:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.020+0000 7f5653e00700 1 -- 172.21.15.112:0/3128270839 learned_addr learned my addr 172.21.15.112:0/3128270839 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.020+0000 7f5653e00700 1 -- 172.21.15.112:0/3128270839 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565419f070 con 0x7f56541096b0 2024-05-08T05:43:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.020+0000 7f5653e00700 1 --2- 172.21.15.112:0/3128270839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56541096b0 0x7f565419eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5644005190 tx=0x7f5644009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.020+0000 7f5651600700 1 -- 172.21.15.112:0/3128270839 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5644027070 con 0x7f56541096b0 2024-05-08T05:43:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.021+0000 7f5651600700 1 -- 172.21.15.112:0/3128270839 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5644022470 con 0x7f56541096b0 2024-05-08T05:43:22.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.021+0000 7f565b000700 1 -- 172.21.15.112:0/3128270839 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565419f300 con 0x7f56541096b0 2024-05-08T05:43:22.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.021+0000 7f5651600700 1 -- 172.21.15.112:0/3128270839 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f564400f5f0 con 0x7f56541096b0 2024-05-08T05:43:22.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.021+0000 7f565b000700 1 -- 172.21.15.112:0/3128270839 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56541a1ff0 con 0x7f56541096b0 2024-05-08T05:43:22.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.022+0000 7f5651600700 1 -- 172.21.15.112:0/3128270839 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f564402a070 con 0x7f56541096b0 2024-05-08T05:43:22.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.022+0000 7f565b000700 1 -- 172.21.15.112:0/3128270839 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f565410fb50 con 0x7f56541096b0 2024-05-08T05:43:22.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.022+0000 7f5651600700 1 --2- 172.21.15.112:0/3128270839 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f563803e5c0 0x7f5638040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:22.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.022+0000 7f5651600700 1 -- 172.21.15.112:0/3128270839 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f564401e030 con 0x7f56541096b0 2024-05-08T05:43:22.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.023+0000 7f564be00700 1 --2- 172.21.15.112:0/3128270839 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f563803e5c0 0x7f5638040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:22.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.023+0000 7f564be00700 1 --2- 172.21.15.112:0/3128270839 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f563803e5c0 0x7f5638040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f563c006fd0 tx=0x7f563c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:22.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.028+0000 7f5651600700 1 -- 172.21.15.112:0/3128270839 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5644013070 con 0x7f56541096b0 2024-05-08T05:43:22.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.261+0000 7f565b000700 1 -- 172.21.15.112:0/3128270839 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f565402e6b0 con 0x7f563803e5c0 2024-05-08T05:43:22.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.272+0000 7f5651600700 1 -- 172.21.15.112:0/3128270839 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f565402e6b0 con 0x7f563803e5c0 2024-05-08T05:43:22.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.275+0000 7f565b000700 1 -- 172.21.15.112:0/3128270839 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f563803e5c0 msgr2=0x7f5638040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:22.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.275+0000 7f565b000700 1 --2- 172.21.15.112:0/3128270839 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f563803e5c0 0x7f5638040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f563c006fd0 tx=0x7f563c006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:22.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.275+0000 7f565b000700 1 -- 172.21.15.112:0/3128270839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56541096b0 msgr2=0x7f565419eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:22.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.275+0000 7f565b000700 1 --2- 172.21.15.112:0/3128270839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56541096b0 0x7f565419eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5644005190 tx=0x7f5644009ea0 comp rx=0 tx=0).stop 2024-05-08T05:43:22.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.276+0000 7f565b000700 1 -- 172.21.15.112:0/3128270839 shutdown_connections 2024-05-08T05:43:22.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.276+0000 7f565b000700 1 --2- 172.21.15.112:0/3128270839 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f563803e5c0 0x7f5638040a80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:22.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.276+0000 7f565b000700 1 --2- 172.21.15.112:0/3128270839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56541096b0 0x7f565419eb30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:22.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.276+0000 7f565b000700 1 -- 172.21.15.112:0/3128270839 >> 172.21.15.112:0/3128270839 conn(0x7f5654078790 msgr2=0x7f56540792a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:22.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.276+0000 7f565b000700 1 -- 172.21.15.112:0/3128270839 shutdown_connections 2024-05-08T05:43:22.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:22.276+0000 7f565b000700 1 -- 172.21.15.112:0/3128270839 wait complete. 2024-05-08T05:43:23.063 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi150 2024-05-08T05:43:23.063 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-08T05:43:23.063 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.conf 2024-05-08T05:43:23.082 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-08T05:43:23.082 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:23.139 INFO:tasks.cephadm:Adding host smithi150 to orchestrator... 2024-05-08T05:43:23.140 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch host add smithi150 2024-05-08T05:43:23.535 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:43:23.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:23 smithi112 ceph-mon[30003]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:23.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:23 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:23.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:23 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:23.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:23 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:23.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:23 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:23.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:23 smithi112 ceph-mon[30003]: Updating smithi112:/etc/ceph/ceph.conf 2024-05-08T05:43:23.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:23 smithi112 ceph-mon[30003]: Updating smithi112:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:24.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:24 smithi112 ceph-mon[30003]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:24.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:24 smithi112 ceph-mon[30003]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:24.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:24 smithi112 ceph-mon[30003]: Updating smithi112:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:43:25.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.531+0000 7efec1c00700 1 -- 172.21.15.112:0/2012273005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc1052f0 msgr2=0x7efebc1056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:25.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.531+0000 7efec1c00700 1 --2- 172.21.15.112:0/2012273005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc1052f0 0x7efebc1056d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efeac009910 tx=0x7efeac009c20 comp rx=0 tx=0).stop 2024-05-08T05:43:25.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.531+0000 7efec1c00700 1 -- 172.21.15.112:0/2012273005 shutdown_connections 2024-05-08T05:43:25.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.531+0000 7efec1c00700 1 --2- 172.21.15.112:0/2012273005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc1052f0 0x7efebc1056d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:25.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.531+0000 7efec1c00700 1 -- 172.21.15.112:0/2012273005 >> 172.21.15.112:0/2012273005 conn(0x7efebc100bc0 msgr2=0x7efebc102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:25.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.531+0000 7efec1c00700 1 -- 172.21.15.112:0/2012273005 shutdown_connections 2024-05-08T05:43:25.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.531+0000 7efec1c00700 1 -- 172.21.15.112:0/2012273005 wait complete. 2024-05-08T05:43:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.533+0000 7efec1c00700 1 Processor -- start 2024-05-08T05:43:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.533+0000 7efec1c00700 1 -- start start 2024-05-08T05:43:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.533+0000 7efec1c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc1052f0 0x7efebc1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.533+0000 7efec1c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efebc10f570 con 0x7efebc1052f0 2024-05-08T05:43:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.533+0000 7efebaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc1052f0 0x7efebc1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:25.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.533+0000 7efebaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc1052f0 0x7efebc1a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41278/0 (socket says 172.21.15.112:41278) 2024-05-08T05:43:25.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.533+0000 7efebaa00700 1 -- 172.21.15.112:0/3766172596 learned_addr learned my addr 172.21.15.112:0/3766172596 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:25.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.534+0000 7efebaa00700 1 -- 172.21.15.112:0/3766172596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc1a0c80 con 0x7efebc1052f0 2024-05-08T05:43:25.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.534+0000 7efebaa00700 1 --2- 172.21.15.112:0/3766172596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc1052f0 0x7efebc1a0740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efeac006010 tx=0x7efeac005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:25.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.534+0000 7efeb3400700 1 -- 172.21.15.112:0/3766172596 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeac026070 con 0x7efebc1052f0 2024-05-08T05:43:25.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.534+0000 7efeb3400700 1 -- 172.21.15.112:0/3766172596 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeac00bba0 con 0x7efebc1052f0 2024-05-08T05:43:25.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.534+0000 7efeb3400700 1 -- 172.21.15.112:0/3766172596 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeac003910 con 0x7efebc1052f0 2024-05-08T05:43:25.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.534+0000 7efec1c00700 1 -- 172.21.15.112:0/3766172596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efebc1a0f70 con 0x7efebc1052f0 2024-05-08T05:43:25.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.535+0000 7efec1c00700 1 -- 172.21.15.112:0/3766172596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efebc19a900 con 0x7efebc1052f0 2024-05-08T05:43:25.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.536+0000 7efeb3400700 1 -- 172.21.15.112:0/3766172596 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7efeac029070 con 0x7efebc1052f0 2024-05-08T05:43:25.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.536+0000 7efec1c00700 1 -- 172.21.15.112:0/3766172596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efebc1067d0 con 0x7efebc1052f0 2024-05-08T05:43:25.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.536+0000 7efeb3400700 1 --2- 172.21.15.112:0/3766172596 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efea003e510 0x7efea00409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:25.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.536+0000 7efeb3400700 1 -- 172.21.15.112:0/3766172596 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efeac01e070 con 0x7efebc1052f0 2024-05-08T05:43:25.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.537+0000 7efeba000700 1 --2- 172.21.15.112:0/3766172596 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efea003e510 0x7efea00409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:25.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.537+0000 7efeba000700 1 --2- 172.21.15.112:0/3766172596 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efea003e510 0x7efea00409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efea8006fd0 tx=0x7efea8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:25.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.542+0000 7efeb3400700 1 -- 172.21.15.112:0/3766172596 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efeac013070 con 0x7efebc1052f0 2024-05-08T05:43:25.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:25.779+0000 7efec1c00700 1 -- 172.21.15.112:0/3766172596 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}) -- 0x7efebc19bea0 con 0x7efea003e510 2024-05-08T05:43:26.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:25 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:26.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:25 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:26.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:25 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:27.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:26 smithi112 ceph-mon[30003]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:27.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:26 smithi112 ceph-mon[30003]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:28.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:27 smithi112 ceph-mon[30003]: Deploying cephadm binary to smithi150 2024-05-08T05:43:29.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:28 smithi112 ceph-mon[30003]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:30.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:30.457+0000 7efeb3400700 1 -- 172.21.15.112:0/3766172596 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7efebc19bea0 con 0x7efea003e510 2024-05-08T05:43:30.458 INFO:teuthology.orchestra.run.smithi112.stdout:Added host 'smithi150' with addr '172.21.15.150' 2024-05-08T05:43:30.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:30.460+0000 7efec1c00700 1 -- 172.21.15.112:0/3766172596 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efea003e510 msgr2=0x7efea00409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:30.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:30.460+0000 7efec1c00700 1 --2- 172.21.15.112:0/3766172596 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efea003e510 0x7efea00409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efea8006fd0 tx=0x7efea8006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:30.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:30.461+0000 7efec1c00700 1 -- 172.21.15.112:0/3766172596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc1052f0 msgr2=0x7efebc1a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:30.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:30.461+0000 7efec1c00700 1 --2- 172.21.15.112:0/3766172596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc1052f0 0x7efebc1a0740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efeac006010 tx=0x7efeac005780 comp rx=0 tx=0).stop 2024-05-08T05:43:30.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:30.461+0000 7efec1c00700 1 -- 172.21.15.112:0/3766172596 shutdown_connections 2024-05-08T05:43:30.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:30.461+0000 7efec1c00700 1 --2- 172.21.15.112:0/3766172596 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efea003e510 0x7efea00409d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:30.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:30.461+0000 7efec1c00700 1 --2- 172.21.15.112:0/3766172596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efebc1052f0 0x7efebc1a0740 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:30.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:30.461+0000 7efec1c00700 1 -- 172.21.15.112:0/3766172596 >> 172.21.15.112:0/3766172596 conn(0x7efebc100bc0 msgr2=0x7efebc106cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:30.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:30.461+0000 7efec1c00700 1 -- 172.21.15.112:0/3766172596 shutdown_connections 2024-05-08T05:43:30.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:30.461+0000 7efec1c00700 1 -- 172.21.15.112:0/3766172596 wait complete. 2024-05-08T05:43:31.131 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:30 smithi112 ceph-mon[30003]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:31.131 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:31.131 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:31.190 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T05:43:31.625 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:43:32.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:32 smithi112 ceph-mon[30003]: Added host smithi150 2024-05-08T05:43:32.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:32.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:33.229 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:33 smithi112 ceph-mon[30003]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:33.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.692+0000 7f6f47200700 1 -- 172.21.15.112:0/2637168383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48105130 msgr2=0x7f6f48105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:33.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.692+0000 7f6f47200700 1 --2- 172.21.15.112:0/2637168383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48105130 0x7f6f48105510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009910 tx=0x7f6f38009c20 comp rx=0 tx=0).stop 2024-05-08T05:43:33.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.692+0000 7f6f47200700 1 -- 172.21.15.112:0/2637168383 shutdown_connections 2024-05-08T05:43:33.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.692+0000 7f6f47200700 1 --2- 172.21.15.112:0/2637168383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48105130 0x7f6f48105510 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:33.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.692+0000 7f6f47200700 1 -- 172.21.15.112:0/2637168383 >> 172.21.15.112:0/2637168383 conn(0x7f6f481009e0 msgr2=0x7f6f48102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:33.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.692+0000 7f6f47200700 1 -- 172.21.15.112:0/2637168383 shutdown_connections 2024-05-08T05:43:33.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.692+0000 7f6f47200700 1 -- 172.21.15.112:0/2637168383 wait complete. 2024-05-08T05:43:33.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.693+0000 7f6f47200700 1 Processor -- start 2024-05-08T05:43:33.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.694+0000 7f6f47200700 1 -- start start 2024-05-08T05:43:33.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.694+0000 7f6f47200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48105130 0x7f6f481a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:33.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.694+0000 7f6f47200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4810f3b0 con 0x7f6f48105130 2024-05-08T05:43:33.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.694+0000 7f6f45e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48105130 0x7f6f481a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:33.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.694+0000 7f6f45e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48105130 0x7f6f481a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41292/0 (socket says 172.21.15.112:41292) 2024-05-08T05:43:33.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.694+0000 7f6f45e00700 1 -- 172.21.15.112:0/350504906 learned_addr learned my addr 172.21.15.112:0/350504906 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:33.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.695+0000 7f6f45e00700 1 -- 172.21.15.112:0/350504906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f481a0be0 con 0x7f6f48105130 2024-05-08T05:43:33.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.695+0000 7f6f45e00700 1 --2- 172.21.15.112:0/350504906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48105130 0x7f6f481a06a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f38006010 tx=0x7f6f38005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:33.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.695+0000 7f6f3ea00700 1 -- 172.21.15.112:0/350504906 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f38026070 con 0x7f6f48105130 2024-05-08T05:43:33.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.695+0000 7f6f3ea00700 1 -- 172.21.15.112:0/350504906 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f3800bba0 con 0x7f6f48105130 2024-05-08T05:43:33.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.695+0000 7f6f3ea00700 1 -- 172.21.15.112:0/350504906 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f38003910 con 0x7f6f48105130 2024-05-08T05:43:33.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.695+0000 7f6f47200700 1 -- 172.21.15.112:0/350504906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f481a0ed0 con 0x7f6f48105130 2024-05-08T05:43:33.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.695+0000 7f6f47200700 1 -- 172.21.15.112:0/350504906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4819a940 con 0x7f6f48105130 2024-05-08T05:43:33.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.696+0000 7f6f47200700 1 -- 172.21.15.112:0/350504906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f48106610 con 0x7f6f48105130 2024-05-08T05:43:33.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.697+0000 7f6f3ea00700 1 -- 172.21.15.112:0/350504906 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f6f38029070 con 0x7f6f48105130 2024-05-08T05:43:33.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.697+0000 7f6f3ea00700 1 --2- 172.21.15.112:0/350504906 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6f2c03e570 0x7f6f2c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:33.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.697+0000 7f6f3ea00700 1 -- 172.21.15.112:0/350504906 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6f3801e070 con 0x7f6f48105130 2024-05-08T05:43:33.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.698+0000 7f6f45400700 1 --2- 172.21.15.112:0/350504906 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6f2c03e570 0x7f6f2c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:33.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.698+0000 7f6f45400700 1 --2- 172.21.15.112:0/350504906 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6f2c03e570 0x7f6f2c040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6f34006fd0 tx=0x7f6f34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:33.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.703+0000 7f6f3ea00700 1 -- 172.21.15.112:0/350504906 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6f38013070 con 0x7f6f48105130 2024-05-08T05:43:33.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.940+0000 7f6f47200700 1 -- 172.21.15.112:0/350504906 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f4819bca0 con 0x7f6f2c03e570 2024-05-08T05:43:33.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.943+0000 7f6f3ea00700 1 -- 172.21.15.112:0/350504906 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f6f4819bca0 con 0x7f6f2c03e570 2024-05-08T05:43:33.944 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:43:33.944 INFO:teuthology.orchestra.run.smithi112.stdout:[{"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}] 2024-05-08T05:43:33.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.946+0000 7f6f47200700 1 -- 172.21.15.112:0/350504906 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6f2c03e570 msgr2=0x7f6f2c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:33.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.946+0000 7f6f47200700 1 --2- 172.21.15.112:0/350504906 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6f2c03e570 0x7f6f2c040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6f34006fd0 tx=0x7f6f34006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:33.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.946+0000 7f6f47200700 1 -- 172.21.15.112:0/350504906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48105130 msgr2=0x7f6f481a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:33.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.946+0000 7f6f47200700 1 --2- 172.21.15.112:0/350504906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48105130 0x7f6f481a06a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f38006010 tx=0x7f6f38005780 comp rx=0 tx=0).stop 2024-05-08T05:43:33.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.946+0000 7f6f47200700 1 -- 172.21.15.112:0/350504906 shutdown_connections 2024-05-08T05:43:33.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.946+0000 7f6f47200700 1 --2- 172.21.15.112:0/350504906 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6f2c03e570 0x7f6f2c040a30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:33.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.946+0000 7f6f47200700 1 --2- 172.21.15.112:0/350504906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48105130 0x7f6f481a06a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:33.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.946+0000 7f6f47200700 1 -- 172.21.15.112:0/350504906 >> 172.21.15.112:0/350504906 conn(0x7f6f481009e0 msgr2=0x7f6f48106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:33.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.947+0000 7f6f47200700 1 -- 172.21.15.112:0/350504906 shutdown_connections 2024-05-08T05:43:33.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:33.947+0000 7f6f47200700 1 -- 172.21.15.112:0/350504906 wait complete. 2024-05-08T05:43:34.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:34.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:34 smithi112 ceph-mon[30003]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:34.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:34 smithi112 ceph-mon[30003]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:43:34.694 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi162 2024-05-08T05:43:34.694 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-08T05:43:34.694 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/etc/ceph/ceph.conf 2024-05-08T05:43:34.713 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-08T05:43:34.713 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:34.770 INFO:tasks.cephadm:Adding host smithi162 to orchestrator... 2024-05-08T05:43:34.770 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch host add smithi162 2024-05-08T05:43:35.183 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:43:36.766 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:36 smithi112 ceph-mon[30003]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:37.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.312+0000 7fd1d1e00700 1 -- 172.21.15.112:0/49386353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc102900 msgr2=0x7fd1cc102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:37.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.312+0000 7fd1d1e00700 1 --2- 172.21.15.112:0/49386353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc102900 0x7fd1cc102ce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009910 tx=0x7fd1bc009c20 comp rx=0 tx=0).stop 2024-05-08T05:43:37.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.312+0000 7fd1d1e00700 1 -- 172.21.15.112:0/49386353 shutdown_connections 2024-05-08T05:43:37.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.312+0000 7fd1d1e00700 1 --2- 172.21.15.112:0/49386353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc102900 0x7fd1cc102ce0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:37.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.312+0000 7fd1d1e00700 1 -- 172.21.15.112:0/49386353 >> 172.21.15.112:0/49386353 conn(0x7fd1cc079400 msgr2=0x7fd1cc079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:37.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.313+0000 7fd1d1e00700 1 -- 172.21.15.112:0/49386353 shutdown_connections 2024-05-08T05:43:37.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.313+0000 7fd1d1e00700 1 -- 172.21.15.112:0/49386353 wait complete. 2024-05-08T05:43:37.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.314+0000 7fd1d1e00700 1 Processor -- start 2024-05-08T05:43:37.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.314+0000 7fd1d1e00700 1 -- start start 2024-05-08T05:43:37.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.314+0000 7fd1d1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc102900 0x7fd1cc19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:37.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.314+0000 7fd1d1e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc10c330 con 0x7fd1cc102900 2024-05-08T05:43:37.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.315+0000 7fd1caa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc102900 0x7fd1cc19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:37.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.315+0000 7fd1caa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc102900 0x7fd1cc19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56594/0 (socket says 172.21.15.112:56594) 2024-05-08T05:43:37.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.315+0000 7fd1caa00700 1 -- 172.21.15.112:0/1416033545 learned_addr learned my addr 172.21.15.112:0/1416033545 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:37.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.315+0000 7fd1caa00700 1 -- 172.21.15.112:0/1416033545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc19eb40 con 0x7fd1cc102900 2024-05-08T05:43:37.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.315+0000 7fd1caa00700 1 --2- 172.21.15.112:0/1416033545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc102900 0x7fd1cc19e600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc006010 tx=0x7fd1bc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:37.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.316+0000 7fd1d0a00700 1 -- 172.21.15.112:0/1416033545 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1bc026070 con 0x7fd1cc102900 2024-05-08T05:43:37.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.316+0000 7fd1d0a00700 1 -- 172.21.15.112:0/1416033545 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1bc00bba0 con 0x7fd1cc102900 2024-05-08T05:43:37.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.316+0000 7fd1d0a00700 1 -- 172.21.15.112:0/1416033545 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1bc003910 con 0x7fd1cc102900 2024-05-08T05:43:37.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.316+0000 7fd1d1e00700 1 -- 172.21.15.112:0/1416033545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1cc19ee30 con 0x7fd1cc102900 2024-05-08T05:43:37.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.316+0000 7fd1d1e00700 1 -- 172.21.15.112:0/1416033545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1cc1988a0 con 0x7fd1cc102900 2024-05-08T05:43:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.317+0000 7fd1d1e00700 1 -- 172.21.15.112:0/1416033545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1cc1036b0 con 0x7fd1cc102900 2024-05-08T05:43:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.317+0000 7fd1d0a00700 1 -- 172.21.15.112:0/1416033545 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fd1bc029070 con 0x7fd1cc102900 2024-05-08T05:43:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.318+0000 7fd1d0a00700 1 --2- 172.21.15.112:0/1416033545 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd1b003e570 0x7fd1b0040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:37.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.318+0000 7fd1d0a00700 1 -- 172.21.15.112:0/1416033545 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd1bc01e070 con 0x7fd1cc102900 2024-05-08T05:43:37.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.318+0000 7fd1ca000700 1 --2- 172.21.15.112:0/1416033545 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd1b003e570 0x7fd1b0040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:37.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.319+0000 7fd1ca000700 1 --2- 172.21.15.112:0/1416033545 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd1b003e570 0x7fd1b0040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8006fd0 tx=0x7fd1b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.323+0000 7fd1d0a00700 1 -- 172.21.15.112:0/1416033545 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd1bc013070 con 0x7fd1cc102900 2024-05-08T05:43:37.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:37.550+0000 7fd1d1e00700 1 -- 172.21.15.112:0/1416033545 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi162", "target": ["mon-mgr", ""]}) -- 0x7fd1cc199c30 con 0x7fd1b003e570 2024-05-08T05:43:37.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:37 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:37.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:37 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:37.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:37 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:37.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:37 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:37.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:37 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:43:37.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:37 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:37.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:37 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:37.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:37 smithi112 ceph-mon[30003]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-08T05:43:37.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:37 smithi112 ceph-mon[30003]: Updating smithi150:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:38 smithi112 ceph-mon[30003]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:38 smithi112 ceph-mon[30003]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:38 smithi112 ceph-mon[30003]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi162", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:38 smithi112 ceph-mon[30003]: Updating smithi150:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:43:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:38 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:38 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:38 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:40.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:39 smithi112 ceph-mon[30003]: Deploying cephadm binary to smithi162 2024-05-08T05:43:41.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:40 smithi112 ceph-mon[30003]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:42.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:42.303+0000 7fd1d0a00700 1 -- 172.21.15.112:0/1416033545 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd1cc199c30 con 0x7fd1b003e570 2024-05-08T05:43:42.304 INFO:teuthology.orchestra.run.smithi112.stdout:Added host 'smithi162' with addr '172.21.15.162' 2024-05-08T05:43:42.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:42.308+0000 7fd1d1e00700 1 -- 172.21.15.112:0/1416033545 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd1b003e570 msgr2=0x7fd1b0040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:42.308+0000 7fd1d1e00700 1 --2- 172.21.15.112:0/1416033545 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd1b003e570 0x7fd1b0040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8006fd0 tx=0x7fd1b8006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:42.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:42.308+0000 7fd1d1e00700 1 -- 172.21.15.112:0/1416033545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc102900 msgr2=0x7fd1cc19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:42.308+0000 7fd1d1e00700 1 --2- 172.21.15.112:0/1416033545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc102900 0x7fd1cc19e600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc006010 tx=0x7fd1bc005780 comp rx=0 tx=0).stop 2024-05-08T05:43:42.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:42.308+0000 7fd1d1e00700 1 -- 172.21.15.112:0/1416033545 shutdown_connections 2024-05-08T05:43:42.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:42.308+0000 7fd1d1e00700 1 --2- 172.21.15.112:0/1416033545 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd1b003e570 0x7fd1b0040a30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:42.308+0000 7fd1d1e00700 1 --2- 172.21.15.112:0/1416033545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc102900 0x7fd1cc19e600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:42.308+0000 7fd1d1e00700 1 -- 172.21.15.112:0/1416033545 >> 172.21.15.112:0/1416033545 conn(0x7fd1cc079400 msgr2=0x7fd1cc105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:42.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:42.308+0000 7fd1d1e00700 1 -- 172.21.15.112:0/1416033545 shutdown_connections 2024-05-08T05:43:42.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:42.308+0000 7fd1d1e00700 1 -- 172.21.15.112:0/1416033545 wait complete. 2024-05-08T05:43:43.055 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T05:43:43.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:42 smithi112 ceph-mon[30003]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:43.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:43.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:43.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:43.481 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:43:44.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:43 smithi112 ceph-mon[30003]: Added host smithi162 2024-05-08T05:43:44.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:43 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:45.275 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:44 smithi112 ceph-mon[30003]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.353+0000 7f13b2800700 1 -- 172.21.15.112:0/1043209751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13ac1030e0 msgr2=0x7f13ac1034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.353+0000 7f13b2800700 1 --2- 172.21.15.112:0/1043209751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13ac1030e0 0x7f13ac1034c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f139c00b0a0 tx=0x7f139c00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.353+0000 7f13b2800700 1 -- 172.21.15.112:0/1043209751 shutdown_connections 2024-05-08T05:43:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.353+0000 7f13b2800700 1 --2- 172.21.15.112:0/1043209751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13ac1030e0 0x7f13ac1034c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.353+0000 7f13b2800700 1 -- 172.21.15.112:0/1043209751 >> 172.21.15.112:0/1043209751 conn(0x7f13ac0fe990 msgr2=0x7f13ac100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.353+0000 7f13b2800700 1 -- 172.21.15.112:0/1043209751 shutdown_connections 2024-05-08T05:43:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.353+0000 7f13b2800700 1 -- 172.21.15.112:0/1043209751 wait complete. 2024-05-08T05:43:45.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.354+0000 7f13b2800700 1 Processor -- start 2024-05-08T05:43:45.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.354+0000 7f13b2800700 1 -- start start 2024-05-08T05:43:45.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.355+0000 7f13b2800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13ac1030e0 0x7f13ac19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.355+0000 7f13b2800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f13ac10d360 con 0x7f13ac1030e0 2024-05-08T05:43:45.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.355+0000 7f13ab400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13ac1030e0 0x7f13ac19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.356+0000 7f13ab400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13ac1030e0 0x7f13ac19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35818/0 (socket says 172.21.15.112:35818) 2024-05-08T05:43:45.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.356+0000 7f13ab400700 1 -- 172.21.15.112:0/3243011140 learned_addr learned my addr 172.21.15.112:0/3243011140 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:45.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.356+0000 7f13ab400700 1 -- 172.21.15.112:0/3243011140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ac19eba0 con 0x7f13ac1030e0 2024-05-08T05:43:45.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.356+0000 7f13ab400700 1 --2- 172.21.15.112:0/3243011140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13ac1030e0 0x7f13ac19e660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f139c00d010 tx=0x7f139c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:45.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.356+0000 7f13a3e00700 1 -- 172.21.15.112:0/3243011140 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f139c021020 con 0x7f13ac1030e0 2024-05-08T05:43:45.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.356+0000 7f13a3e00700 1 -- 172.21.15.112:0/3243011140 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f139c0044b0 con 0x7f13ac1030e0 2024-05-08T05:43:45.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.356+0000 7f13b2800700 1 -- 172.21.15.112:0/3243011140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13ac19ee30 con 0x7f13ac1030e0 2024-05-08T05:43:45.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.357+0000 7f13a3e00700 1 -- 172.21.15.112:0/3243011140 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f139c017040 con 0x7f13ac1030e0 2024-05-08T05:43:45.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.357+0000 7f13b2800700 1 -- 172.21.15.112:0/3243011140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13ac198730 con 0x7f13ac1030e0 2024-05-08T05:43:45.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.358+0000 7f13b2800700 1 -- 172.21.15.112:0/3243011140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13ac1045c0 con 0x7f13ac1030e0 2024-05-08T05:43:45.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.358+0000 7f13a3e00700 1 -- 172.21.15.112:0/3243011140 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f139c024020 con 0x7f13ac1030e0 2024-05-08T05:43:45.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.358+0000 7f13a3e00700 1 --2- 172.21.15.112:0/3243011140 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f139003e560 0x7f1390040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.358+0000 7f13a3e00700 1 -- 172.21.15.112:0/3243011140 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f139c00f050 con 0x7f13ac1030e0 2024-05-08T05:43:45.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.359+0000 7f13aaa00700 1 --2- 172.21.15.112:0/3243011140 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f139003e560 0x7f1390040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.359+0000 7f13aaa00700 1 --2- 172.21.15.112:0/3243011140 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f139003e560 0x7f1390040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1398006fd0 tx=0x7f1398006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:45.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.364+0000 7f13a3e00700 1 -- 172.21.15.112:0/3243011140 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f139c02b020 con 0x7f13ac1030e0 2024-05-08T05:43:45.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.601+0000 7f13b2800700 1 -- 172.21.15.112:0/3243011140 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f13ac199c60 con 0x7f139003e560 2024-05-08T05:43:45.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.603+0000 7f13a3e00700 1 -- 172.21.15.112:0/3243011140 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f13ac199c60 con 0x7f139003e560 2024-05-08T05:43:45.603 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:43:45.603 INFO:teuthology.orchestra.run.smithi112.stdout:[{"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}, {"addr": "172.21.15.162", "hostname": "smithi162", "labels": [], "status": ""}] 2024-05-08T05:43:45.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.606+0000 7f13b2800700 1 -- 172.21.15.112:0/3243011140 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f139003e560 msgr2=0x7f1390040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.606+0000 7f13b2800700 1 --2- 172.21.15.112:0/3243011140 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f139003e560 0x7f1390040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1398006fd0 tx=0x7f1398006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:45.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.606+0000 7f13b2800700 1 -- 172.21.15.112:0/3243011140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13ac1030e0 msgr2=0x7f13ac19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.606+0000 7f13b2800700 1 --2- 172.21.15.112:0/3243011140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13ac1030e0 0x7f13ac19e660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f139c00d010 tx=0x7f139c003ce0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.606+0000 7f13b2800700 1 -- 172.21.15.112:0/3243011140 shutdown_connections 2024-05-08T05:43:45.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.606+0000 7f13b2800700 1 --2- 172.21.15.112:0/3243011140 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f139003e560 0x7f1390040a20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.606+0000 7f13b2800700 1 --2- 172.21.15.112:0/3243011140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f13ac1030e0 0x7f13ac19e660 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.606+0000 7f13b2800700 1 -- 172.21.15.112:0/3243011140 >> 172.21.15.112:0/3243011140 conn(0x7f13ac0fe990 msgr2=0x7f13ac105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:45.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.607+0000 7f13b2800700 1 -- 172.21.15.112:0/3243011140 shutdown_connections 2024-05-08T05:43:45.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:45.607+0000 7f13b2800700 1 -- 172.21.15.112:0/3243011140 wait complete. 2024-05-08T05:43:46.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:46 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:46.366 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-08T05:43:46.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-08T05:43:46.790 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:43:47.210 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:47 smithi112 ceph-mon[30003]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:47.211 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:47 smithi112 ceph-mon[30003]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:43:48.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.725+0000 7fddf4c00700 1 -- 172.21.15.112:0/172436951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddf0105130 msgr2=0x7fddf0105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:48.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.725+0000 7fddf4c00700 1 --2- 172.21.15.112:0/172436951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddf0105130 0x7fddf0105510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009910 tx=0x7fdde0009c20 comp rx=0 tx=0).stop 2024-05-08T05:43:48.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.725+0000 7fddf4c00700 1 -- 172.21.15.112:0/172436951 shutdown_connections 2024-05-08T05:43:48.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.725+0000 7fddf4c00700 1 --2- 172.21.15.112:0/172436951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddf0105130 0x7fddf0105510 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:48.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.725+0000 7fddf4c00700 1 -- 172.21.15.112:0/172436951 >> 172.21.15.112:0/172436951 conn(0x7fddf01009e0 msgr2=0x7fddf0102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:48.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.725+0000 7fddf4c00700 1 -- 172.21.15.112:0/172436951 shutdown_connections 2024-05-08T05:43:48.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.725+0000 7fddf4c00700 1 -- 172.21.15.112:0/172436951 wait complete. 2024-05-08T05:43:48.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.726+0000 7fddf4c00700 1 Processor -- start 2024-05-08T05:43:48.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.726+0000 7fddf4c00700 1 -- start start 2024-05-08T05:43:48.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.727+0000 7fddf4c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddf0105130 0x7fddf01a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:48.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.727+0000 7fddf4c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fddf010f3b0 con 0x7fddf0105130 2024-05-08T05:43:48.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.727+0000 7fddef400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddf0105130 0x7fddf01a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:48.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.727+0000 7fddef400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddf0105130 0x7fddf01a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35838/0 (socket says 172.21.15.112:35838) 2024-05-08T05:43:48.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.727+0000 7fddef400700 1 -- 172.21.15.112:0/350209145 learned_addr learned my addr 172.21.15.112:0/350209145 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:43:48.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.727+0000 7fddef400700 1 -- 172.21.15.112:0/350209145 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf01a0b80 con 0x7fddf0105130 2024-05-08T05:43:48.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.728+0000 7fddef400700 1 --2- 172.21.15.112:0/350209145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddf0105130 0x7fddf01a0640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdde0006010 tx=0x7fdde0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:48.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.728+0000 7fdde7e00700 1 -- 172.21.15.112:0/350209145 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdde0026070 con 0x7fddf0105130 2024-05-08T05:43:48.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.728+0000 7fdde7e00700 1 -- 172.21.15.112:0/350209145 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdde000bba0 con 0x7fddf0105130 2024-05-08T05:43:48.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.728+0000 7fddf4c00700 1 -- 172.21.15.112:0/350209145 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf01a0e70 con 0x7fddf0105130 2024-05-08T05:43:48.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.728+0000 7fdde7e00700 1 -- 172.21.15.112:0/350209145 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdde0003910 con 0x7fddf0105130 2024-05-08T05:43:48.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.728+0000 7fddf4c00700 1 -- 172.21.15.112:0/350209145 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf019a8e0 con 0x7fddf0105130 2024-05-08T05:43:48.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.729+0000 7fddf4c00700 1 -- 172.21.15.112:0/350209145 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf0106610 con 0x7fddf0105130 2024-05-08T05:43:48.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.729+0000 7fdde7e00700 1 -- 172.21.15.112:0/350209145 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fdde0029070 con 0x7fddf0105130 2024-05-08T05:43:48.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.730+0000 7fdde7e00700 1 --2- 172.21.15.112:0/350209145 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fddd403e570 0x7fddd4040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:48.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.730+0000 7fdde7e00700 1 -- 172.21.15.112:0/350209145 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdde001e070 con 0x7fddf0105130 2024-05-08T05:43:48.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.730+0000 7fddeea00700 1 --2- 172.21.15.112:0/350209145 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fddd403e570 0x7fddd4040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:48.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.731+0000 7fddeea00700 1 --2- 172.21.15.112:0/350209145 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fddd403e570 0x7fddd4040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdddc006fd0 tx=0x7fdddc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:48.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.736+0000 7fdde7e00700 1 -- 172.21.15.112:0/350209145 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdde0013070 con 0x7fddf0105130 2024-05-08T05:43:48.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:48.977+0000 7fddf4c00700 1 -- 172.21.15.112:0/350209145 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fddf00008d0 con 0x7fddf0105130 2024-05-08T05:43:49.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:49.127+0000 7fdde7e00700 1 -- 172.21.15.112:0/350209145 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fdde0030090 con 0x7fddf0105130 2024-05-08T05:43:49.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:49.131+0000 7fddf4c00700 1 -- 172.21.15.112:0/350209145 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fddd403e570 msgr2=0x7fddd4040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:49.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:49.131+0000 7fddf4c00700 1 --2- 172.21.15.112:0/350209145 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fddd403e570 0x7fddd4040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdddc006fd0 tx=0x7fdddc006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:49.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:49.132+0000 7fddf4c00700 1 -- 172.21.15.112:0/350209145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddf0105130 msgr2=0x7fddf01a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:49.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:49.132+0000 7fddf4c00700 1 --2- 172.21.15.112:0/350209145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddf0105130 0x7fddf01a0640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdde0006010 tx=0x7fdde0005780 comp rx=0 tx=0).stop 2024-05-08T05:43:49.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:49.132+0000 7fddf4c00700 1 -- 172.21.15.112:0/350209145 shutdown_connections 2024-05-08T05:43:49.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:49.132+0000 7fddf4c00700 1 --2- 172.21.15.112:0/350209145 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fddd403e570 0x7fddd4040a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:49.132+0000 7fddf4c00700 1 --2- 172.21.15.112:0/350209145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddf0105130 0x7fddf01a0640 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:49.132+0000 7fddf4c00700 1 -- 172.21.15.112:0/350209145 >> 172.21.15.112:0/350209145 conn(0x7fddf01009e0 msgr2=0x7fddf0106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:49.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:49.132+0000 7fddf4c00700 1 -- 172.21.15.112:0/350209145 shutdown_connections 2024-05-08T05:43:49.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:43:49.132+0000 7fddf4c00700 1 -- 172.21.15.112:0/350209145 wait complete. 2024-05-08T05:43:49.135 INFO:teuthology.orchestra.run.smithi112.stderr:adjusted tunables profile to default 2024-05-08T05:43:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:49 smithi112 ceph-mon[30003]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:49 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:49 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:49 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:49 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:49 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi162", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:43:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:49 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:49 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:49 smithi112 ceph-mon[30003]: Updating smithi162:/etc/ceph/ceph.conf 2024-05-08T05:43:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:49 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/350209145' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T05:43:49.899 INFO:tasks.cephadm:Adding mon.a on smithi112 2024-05-08T05:43:49.899 INFO:tasks.cephadm:Adding mon.b on smithi150 2024-05-08T05:43:49.899 INFO:tasks.cephadm:Adding mon.c on smithi162 2024-05-08T05:43:49.899 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi162:172.21.15.162=c' 2024-05-08T05:43:50.126 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:50.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:50 smithi112 ceph-mon[30003]: Updating smithi162:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:50.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:50 smithi112 ceph-mon[30003]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:50.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:50 smithi112 ceph-mon[30003]: Updating smithi162:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:43:50.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:50 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/350209145' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T05:43:50.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:50 smithi112 ceph-mon[30003]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:43:50.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:50.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:50.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:51.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.510+0000 7fcc5d903700 1 -- 172.21.15.162:0/2033236822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc581030e0 msgr2=0x7fcc581034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:51.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.510+0000 7fcc5d903700 1 --2- 172.21.15.162:0/2033236822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc581030e0 0x7fcc581034c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009910 tx=0x7fcc48009c20 comp rx=0 tx=0).stop 2024-05-08T05:43:51.511 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.510+0000 7fcc5d903700 1 -- 172.21.15.162:0/2033236822 shutdown_connections 2024-05-08T05:43:51.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.510+0000 7fcc5d903700 1 --2- 172.21.15.162:0/2033236822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc581030e0 0x7fcc581034c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:51.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.510+0000 7fcc5d903700 1 -- 172.21.15.162:0/2033236822 >> 172.21.15.162:0/2033236822 conn(0x7fcc580fe990 msgr2=0x7fcc58100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:51.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.510+0000 7fcc5d903700 1 -- 172.21.15.162:0/2033236822 shutdown_connections 2024-05-08T05:43:51.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.510+0000 7fcc5d903700 1 -- 172.21.15.162:0/2033236822 wait complete. 2024-05-08T05:43:51.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.511+0000 7fcc5d903700 1 Processor -- start 2024-05-08T05:43:51.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.511+0000 7fcc5d903700 1 -- start start 2024-05-08T05:43:51.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.511+0000 7fcc5d903700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc581030e0 0x7fcc5819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:51.512 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.511+0000 7fcc5d903700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5810d360 con 0x7fcc581030e0 2024-05-08T05:43:51.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.511+0000 7fcc56ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc581030e0 0x7fcc5819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:51.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.512+0000 7fcc56ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc581030e0 0x7fcc5819e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.162:37424/0 (socket says 172.21.15.162:37424) 2024-05-08T05:43:51.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.512+0000 7fcc56ffd700 1 -- 172.21.15.162:0/875402195 learned_addr learned my addr 172.21.15.162:0/875402195 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-08T05:43:51.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.512+0000 7fcc56ffd700 1 -- 172.21.15.162:0/875402195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5819eb90 con 0x7fcc581030e0 2024-05-08T05:43:51.513 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.513+0000 7fcc56ffd700 1 --2- 172.21.15.162:0/875402195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc581030e0 0x7fcc5819e650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcc48006010 tx=0x7fcc48005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:51.514 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.513+0000 7fcc5c901700 1 -- 172.21.15.162:0/875402195 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc48026070 con 0x7fcc581030e0 2024-05-08T05:43:51.514 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.513+0000 7fcc5d903700 1 -- 172.21.15.162:0/875402195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5819ee80 con 0x7fcc581030e0 2024-05-08T05:43:51.514 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.513+0000 7fcc5c901700 1 -- 172.21.15.162:0/875402195 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc4800bba0 con 0x7fcc581030e0 2024-05-08T05:43:51.514 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.513+0000 7fcc5d903700 1 -- 172.21.15.162:0/875402195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc581988f0 con 0x7fcc581030e0 2024-05-08T05:43:51.514 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.513+0000 7fcc5c901700 1 -- 172.21.15.162:0/875402195 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc48030060 con 0x7fcc581030e0 2024-05-08T05:43:51.514 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.514+0000 7fcc5d903700 1 -- 172.21.15.162:0/875402195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc581045c0 con 0x7fcc581030e0 2024-05-08T05:43:51.515 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.514+0000 7fcc5c901700 1 -- 172.21.15.162:0/875402195 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fcc48029070 con 0x7fcc581030e0 2024-05-08T05:43:51.515 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.515+0000 7fcc5c901700 1 --2- 172.21.15.162:0/875402195 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcc4003e510 0x7fcc400409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:51.515 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.515+0000 7fcc5c901700 1 -- 172.21.15.162:0/875402195 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcc4801e070 con 0x7fcc581030e0 2024-05-08T05:43:51.515 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.515+0000 7fcc567fc700 1 --2- 172.21.15.162:0/875402195 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcc4003e510 0x7fcc400409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:51.516 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.516+0000 7fcc567fc700 1 --2- 172.21.15.162:0/875402195 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcc4003e510 0x7fcc400409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c006fd0 tx=0x7fcc4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:51.530 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.530+0000 7fcc5c901700 1 -- 172.21.15.162:0/875402195 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc48013070 con 0x7fcc581030e0 2024-05-08T05:43:51.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:51 smithi112 ceph-mon[30003]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:51.742 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.741+0000 7fcc5d903700 1 -- 172.21.15.162:0/875402195 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi162:172.21.15.162=c", "target": ["mon-mgr", ""]}) -- 0x7fcc58199d10 con 0x7fcc4003e510 2024-05-08T05:43:51.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.756+0000 7fcc5c901700 1 -- 172.21.15.162:0/875402195 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcc58199d10 con 0x7fcc4003e510 2024-05-08T05:43:51.757 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled mon update... 2024-05-08T05:43:51.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.759+0000 7fcc5d903700 1 -- 172.21.15.162:0/875402195 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcc4003e510 msgr2=0x7fcc400409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:51.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.759+0000 7fcc5d903700 1 --2- 172.21.15.162:0/875402195 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcc4003e510 0x7fcc400409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c006fd0 tx=0x7fcc4c006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:51.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.759+0000 7fcc5d903700 1 -- 172.21.15.162:0/875402195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc581030e0 msgr2=0x7fcc5819e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:51.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.759+0000 7fcc5d903700 1 --2- 172.21.15.162:0/875402195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc581030e0 0x7fcc5819e650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcc48006010 tx=0x7fcc48005780 comp rx=0 tx=0).stop 2024-05-08T05:43:51.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.759+0000 7fcc5d903700 1 -- 172.21.15.162:0/875402195 shutdown_connections 2024-05-08T05:43:51.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.759+0000 7fcc5d903700 1 --2- 172.21.15.162:0/875402195 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcc4003e510 0x7fcc400409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:51.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.759+0000 7fcc5d903700 1 --2- 172.21.15.162:0/875402195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc581030e0 0x7fcc5819e650 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:51.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.759+0000 7fcc5d903700 1 -- 172.21.15.162:0/875402195 >> 172.21.15.162:0/875402195 conn(0x7fcc580fe990 msgr2=0x7fcc58104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:51.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.759+0000 7fcc5d903700 1 -- 172.21.15.162:0/875402195 shutdown_connections 2024-05-08T05:43:51.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:51.759+0000 7fcc5d903700 1 -- 172.21.15.162:0/875402195 wait complete. 2024-05-08T05:43:52.283 DEBUG:teuthology.orchestra.run.smithi150:mon.b> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mon.b.service 2024-05-08T05:43:52.286 DEBUG:teuthology.orchestra.run.smithi162:mon.c> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mon.c.service 2024-05-08T05:43:52.288 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T05:43:52.288 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T05:43:53.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:52 smithi112 ceph-mon[30003]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:53.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:52 smithi112 ceph-mon[30003]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi162:172.21.15.162=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:53.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:52 smithi112 ceph-mon[30003]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi162:172.21.15.162=c;count:3 2024-05-08T05:43:53.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:52 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:53.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:52 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:53.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:52 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:53.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:52 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:53.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:52 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:53.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:52 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:43:53.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:52 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:53.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:52 smithi112 ceph-mon[30003]: Deploying daemon mon.c on smithi162 2024-05-08T05:43:53.431 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:55.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:43:54 smithi112 ceph-mon[30003]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.127+0000 7fd844849700 1 -- 172.21.15.162:0/2797215609 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8300079c0 con 0x7fd840101fe0 2024-05-08T05:43:56.129 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.128+0000 7fd847aaf700 1 -- 172.21.15.162:0/2797215609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd840101fe0 msgr2=0x7fd8401023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:56.129 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.128+0000 7fd847aaf700 1 --2- 172.21.15.162:0/2797215609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd840101fe0 0x7fd8401023c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd83000b0a0 tx=0x7fd83000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:43:56.129 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.128+0000 7fd847aaf700 1 -- 172.21.15.162:0/2797215609 shutdown_connections 2024-05-08T05:43:56.129 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.128+0000 7fd847aaf700 1 --2- 172.21.15.162:0/2797215609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd840101fe0 0x7fd8401023c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:56.129 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.128+0000 7fd847aaf700 1 -- 172.21.15.162:0/2797215609 >> 172.21.15.162:0/2797215609 conn(0x7fd840078a70 msgr2=0x7fd8400ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:56.129 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.128+0000 7fd847aaf700 1 -- 172.21.15.162:0/2797215609 shutdown_connections 2024-05-08T05:43:56.129 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.128+0000 7fd847aaf700 1 -- 172.21.15.162:0/2797215609 wait complete. 2024-05-08T05:43:56.129 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.128+0000 7fd847aaf700 1 Processor -- start 2024-05-08T05:43:56.129 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.128+0000 7fd847aaf700 1 -- start start 2024-05-08T05:43:56.130 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.129+0000 7fd847aaf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd840101fe0 0x7fd8401996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:56.130 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.129+0000 7fd847aaf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd84010c260 con 0x7fd840101fe0 2024-05-08T05:43:56.130 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.129+0000 7fd84584b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd840101fe0 0x7fd8401996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:56.130 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.129+0000 7fd84584b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd840101fe0 0x7fd8401996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.162:37446/0 (socket says 172.21.15.162:37446) 2024-05-08T05:43:56.130 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.129+0000 7fd84584b700 1 -- 172.21.15.162:0/377447184 learned_addr learned my addr 172.21.15.162:0/377447184 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-08T05:43:56.130 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.130+0000 7fd84584b700 1 -- 172.21.15.162:0/377447184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd840199bf0 con 0x7fd840101fe0 2024-05-08T05:43:56.131 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.130+0000 7fd84584b700 1 --2- 172.21.15.162:0/377447184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd840101fe0 0x7fd8401996b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd830006130 tx=0x7fd8300097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:56.131 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.131+0000 7fd83effd700 1 -- 172.21.15.162:0/377447184 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd830021020 con 0x7fd840101fe0 2024-05-08T05:43:56.131 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.131+0000 7fd847aaf700 1 -- 172.21.15.162:0/377447184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd840199e80 con 0x7fd840101fe0 2024-05-08T05:43:56.131 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.131+0000 7fd847aaf700 1 -- 172.21.15.162:0/377447184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84019a340 con 0x7fd840101fe0 2024-05-08T05:43:56.131 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.131+0000 7fd83effd700 1 -- 172.21.15.162:0/377447184 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd830009360 con 0x7fd840101fe0 2024-05-08T05:43:56.132 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.131+0000 7fd83effd700 1 -- 172.21.15.162:0/377447184 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd830017040 con 0x7fd840101fe0 2024-05-08T05:43:56.132 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.132+0000 7fd847aaf700 1 -- 172.21.15.162:0/377447184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd82c005330 con 0x7fd840101fe0 2024-05-08T05:43:56.133 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.133+0000 7fd83effd700 1 -- 172.21.15.162:0/377447184 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fd830024020 con 0x7fd840101fe0 2024-05-08T05:43:56.133 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.133+0000 7fd83effd700 1 --2- 172.21.15.162:0/377447184 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd82803e510 0x7fd8280409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:56.133 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.133+0000 7fd83effd700 1 -- 172.21.15.162:0/377447184 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd83000f050 con 0x7fd840101fe0 2024-05-08T05:43:56.134 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.134+0000 7fd84504a700 1 --2- 172.21.15.162:0/377447184 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd82803e510 0x7fd8280409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:56.135 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.135+0000 7fd84504a700 1 --2- 172.21.15.162:0/377447184 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd82803e510 0x7fd8280409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd834006fd0 tx=0x7fd834006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:56.149 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.149+0000 7fd83effd700 1 -- 172.21.15.162:0/377447184 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd83002b030 con 0x7fd840101fe0 2024-05-08T05:43:56.448 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.447+0000 7fd847aaf700 1 -- 172.21.15.162:0/377447184 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd82c0058f0 con 0x7fd840101fe0 2024-05-08T05:43:56.448 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.448+0000 7fd83effd700 1 -- 172.21.15.162:0/377447184 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fd830025c40 con 0x7fd840101fe0 2024-05-08T05:43:56.448 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:43:56.449 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":1,"fsid":"7a07ff60-0cfd-11ef-bc97-c7b262605968","modified":"2024-05-08T05:41:43.845828Z","created":"2024-05-08T05:41:43.845828Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-08T05:43:56.450 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.450+0000 7fd847aaf700 1 -- 172.21.15.162:0/377447184 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd82803e510 msgr2=0x7fd8280409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:56.451 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.450+0000 7fd847aaf700 1 --2- 172.21.15.162:0/377447184 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd82803e510 0x7fd8280409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd834006fd0 tx=0x7fd834006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:56.451 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.450+0000 7fd847aaf700 1 -- 172.21.15.162:0/377447184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd840101fe0 msgr2=0x7fd8401996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:56.451 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.450+0000 7fd847aaf700 1 --2- 172.21.15.162:0/377447184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd840101fe0 0x7fd8401996b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd830006130 tx=0x7fd8300097d0 comp rx=0 tx=0).stop 2024-05-08T05:43:56.451 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.450+0000 7fd847aaf700 1 -- 172.21.15.162:0/377447184 shutdown_connections 2024-05-08T05:43:56.451 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.450+0000 7fd847aaf700 1 --2- 172.21.15.162:0/377447184 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd82803e510 0x7fd8280409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:56.451 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.450+0000 7fd847aaf700 1 --2- 172.21.15.162:0/377447184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd840101fe0 0x7fd8401996b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:56.451 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.450+0000 7fd847aaf700 1 -- 172.21.15.162:0/377447184 >> 172.21.15.162:0/377447184 conn(0x7fd840078a70 msgr2=0x7fd8401039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:56.451 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.450+0000 7fd847aaf700 1 -- 172.21.15.162:0/377447184 shutdown_connections 2024-05-08T05:43:56.451 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:43:56.450+0000 7fd847aaf700 1 -- 172.21.15.162:0/377447184 wait complete. 2024-05-08T05:43:56.451 INFO:teuthology.orchestra.run.smithi162.stderr:dumped monmap epoch 1 2024-05-08T05:43:56.497 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T05:43:56.497 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T05:43:56.497 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T05:43:56.497 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T05:43:56.497 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T05:43:56.497 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T05:43:56.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T05:43:56.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:43:56.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:43:56.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:43:56.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:56.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: Added host smithi150 2024-05-08T05:43:56.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:43:56.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.819 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:43:56.819 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:56.819 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:56.819 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-08T05:43:56.819 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: Updating smithi150:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:56.819 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:56.819 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.819 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi162", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:56.819 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: Updating smithi150:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:43:56.819 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.820 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.820 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.820 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: Deploying cephadm binary to smithi162 2024-05-08T05:43:56.820 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.820 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.820 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.820 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:56.820 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.820 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: Added host smithi162 2024-05-08T05:43:56.820 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.820 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.820 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.820 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.820 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:43:56.821 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.821 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.821 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.821 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.821 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.821 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi162", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:43:56.821 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:56.821 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:56.821 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: Updating smithi162:/etc/ceph/ceph.conf 2024-05-08T05:43:56.821 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/350209145' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T05:43:56.821 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: Updating smithi162:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:56.821 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:56.821 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: Updating smithi162:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:43:56.821 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/350209145' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T05:43:56.821 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:43:56.822 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.822 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.822 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.822 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.822 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.822 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi162:172.21.15.162=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:56.822 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi162:172.21.15.162=c;count:3 2024-05-08T05:43:56.822 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.822 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:56.822 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:56.822 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:56.822 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:43:56.822 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:43:56.822 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:56.823 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: Deploying daemon mon.c on smithi162 2024-05-08T05:43:56.823 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.823 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:43:56 smithi162 ceph-mon[34800]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T05:43:58.175 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T05:43:58.175 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T05:43:58.420 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:44:02.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:44:02.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: mon.a calling monitor election 2024-05-08T05:44:02.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:02.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:02.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:02.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:02.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: mon.c calling monitor election 2024-05-08T05:44:02.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:02.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:02.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:02.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:02.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T05:44:02.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: monmap epoch 2 2024-05-08T05:44:02.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:44:02.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: last_changed 2024-05-08T05:43:56.504158+0000 2024-05-08T05:44:02.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: created 2024-05-08T05:41:43.845828+0000 2024-05-08T05:44:02.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: min_mon_release 19 (squid) 2024-05-08T05:44:02.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: election_strategy: 1 2024-05-08T05:44:02.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-08T05:44:02.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-05-08T05:44:02.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: fsmap 2024-05-08T05:44:02.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:44:02.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: mgrmap e13: x(active, since 58s) 2024-05-08T05:44:02.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: overall HEALTH_OK 2024-05-08T05:44:02.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:02.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:01 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:02.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:44:02.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: mon.a calling monitor election 2024-05-08T05:44:02.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:02.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:02.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:02.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:02.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: mon.c calling monitor election 2024-05-08T05:44:02.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:02.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:02.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:02.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:02.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T05:44:02.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: monmap epoch 2 2024-05-08T05:44:02.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:44:02.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: last_changed 2024-05-08T05:43:56.504158+0000 2024-05-08T05:44:02.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: created 2024-05-08T05:41:43.845828+0000 2024-05-08T05:44:02.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: min_mon_release 19 (squid) 2024-05-08T05:44:02.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: election_strategy: 1 2024-05-08T05:44:02.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-08T05:44:02.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-05-08T05:44:02.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: fsmap 2024-05-08T05:44:02.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:44:02.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: mgrmap e13: x(active, since 58s) 2024-05-08T05:44:02.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: overall HEALTH_OK 2024-05-08T05:44:02.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:02.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:01 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:02.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.865+0000 7f5541edf700 1 -- 172.21.15.162:0/1977074118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55240035c0 msgr2=0x7f5524005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.865+0000 7f5541edf700 1 --2- 172.21.15.162:0/1977074118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55240035c0 0x7f5524005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f553c102ef0 tx=0x7f552c00feb0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.865+0000 7f5541edf700 1 -- 172.21.15.162:0/1977074118 shutdown_connections 2024-05-08T05:44:02.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.865+0000 7f5541edf700 1 --2- 172.21.15.162:0/1977074118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55240035c0 0x7f5524005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.865+0000 7f5541edf700 1 -- 172.21.15.162:0/1977074118 >> 172.21.15.162:0/1977074118 conn(0x7f553c0feab0 msgr2=0x7f553c100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:02.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.865+0000 7f5541edf700 1 -- 172.21.15.162:0/1977074118 shutdown_connections 2024-05-08T05:44:02.867 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.865+0000 7f5541edf700 1 -- 172.21.15.162:0/1977074118 wait complete. 2024-05-08T05:44:02.867 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:02 smithi162 ceph-mon[34800]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:02.867 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:02 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:02.867 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:02 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:44:02.867 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:02 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:02.867 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:02 smithi162 ceph-mon[34800]: Deploying daemon mon.b on smithi150 2024-05-08T05:44:02.867 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:02 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:02.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.866+0000 7f5541edf700 1 Processor -- start 2024-05-08T05:44:02.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.866+0000 7f5541edf700 1 -- start start 2024-05-08T05:44:02.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.866+0000 7f5541edf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55240035c0 0x7f553c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.866+0000 7f5541edf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f553c075ce0 0x7f553c076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.866+0000 7f5541edf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f553c10af10 con 0x7f553c075ce0 2024-05-08T05:44:02.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.866+0000 7f5541edf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f553c10ad90 con 0x7f55240035c0 2024-05-08T05:44:02.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.867+0000 7f553b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55240035c0 0x7f553c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.867+0000 7f553b7fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55240035c0 0x7f553c0757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58074/0 (socket says 172.21.15.162:58074) 2024-05-08T05:44:02.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.867+0000 7f553b7fe700 1 -- 172.21.15.162:0/3057244695 learned_addr learned my addr 172.21.15.162:0/3057244695 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-08T05:44:02.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.867+0000 7f553b7fe700 1 -- 172.21.15.162:0/3057244695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55240035c0 msgr2=0x7f553c0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-08T05:44:02.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.867+0000 7f553b7fe700 1 -- 172.21.15.162:0/3057244695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55240035c0 msgr2=0x7f553c0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T05:44:02.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.867+0000 7f553b7fe700 1 --2- 172.21.15.162:0/3057244695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55240035c0 0x7f553c0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T05:44:02.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.867+0000 7f553b7fe700 1 --2- 172.21.15.162:0/3057244695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55240035c0 0x7f553c0757a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:44:02.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.867+0000 7f553affd700 1 --2- 172.21.15.162:0/3057244695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f553c075ce0 0x7f553c076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.868+0000 7f553affd700 1 -- 172.21.15.162:0/3057244695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55240035c0 msgr2=0x7f553c0757a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:44:02.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.868+0000 7f553affd700 1 --2- 172.21.15.162:0/3057244695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55240035c0 0x7f553c0757a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.868+0000 7f553affd700 1 -- 172.21.15.162:0/3057244695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c0776c0 con 0x7f553c075ce0 2024-05-08T05:44:02.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.868+0000 7f553affd700 1 --2- 172.21.15.162:0/3057244695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f553c075ce0 0x7f553c076160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f553000b4d0 tx=0x7f553000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:02.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.869+0000 7f5538ff9700 1 -- 172.21.15.162:0/3057244695 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5530011630 con 0x7f553c075ce0 2024-05-08T05:44:02.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.869+0000 7f5541edf700 1 -- 172.21.15.162:0/3057244695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553c1a4d10 con 0x7f553c075ce0 2024-05-08T05:44:02.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.869+0000 7f5541edf700 1 -- 172.21.15.162:0/3057244695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553c1a5250 con 0x7f553c075ce0 2024-05-08T05:44:02.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.869+0000 7f5538ff9700 1 -- 172.21.15.162:0/3057244695 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5530011c80 con 0x7f553c075ce0 2024-05-08T05:44:02.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.869+0000 7f5538ff9700 1 -- 172.21.15.162:0/3057244695 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f553001ad60 con 0x7f553c075ce0 2024-05-08T05:44:02.872 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.870+0000 7f5541edf700 1 -- 172.21.15.162:0/3057244695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553c104700 con 0x7f553c075ce0 2024-05-08T05:44:02.872 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.871+0000 7f5538ff9700 1 -- 172.21.15.162:0/3057244695 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f5530018020 con 0x7f553c075ce0 2024-05-08T05:44:02.872 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.871+0000 7f5538ff9700 1 --2- 172.21.15.162:0/3057244695 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f552803e2b0 0x7f5528040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.872 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.871+0000 7f5538ff9700 1 -- 172.21.15.162:0/3057244695 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5530052260 con 0x7f553c075ce0 2024-05-08T05:44:02.872 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.872+0000 7f553b7fe700 1 --2- 172.21.15.162:0/3057244695 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f552803e2b0 0x7f5528040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.873 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.873+0000 7f553b7fe700 1 --2- 172.21.15.162:0/3057244695 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f552803e2b0 0x7f5528040770 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f552c00fa60 tx=0x7f552c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:02.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:02.889+0000 7f5538ff9700 1 -- 172.21.15.162:0/3057244695 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f553001f020 con 0x7f553c075ce0 2024-05-08T05:44:03.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:02 smithi112 ceph-mon[30003]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:03.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:02 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:03.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:02 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:44:03.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:02 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:03.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:02 smithi112 ceph-mon[30003]: Deploying daemon mon.b on smithi150 2024-05-08T05:44:03.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:02 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:03.178 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:03.178+0000 7f5541edf700 1 -- 172.21.15.162:0/3057244695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f553c1a5530 con 0x7f553c075ce0 2024-05-08T05:44:03.179 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:03.179+0000 7f5538ff9700 1 -- 172.21.15.162:0/3057244695 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f5530022350 con 0x7f553c075ce0 2024-05-08T05:44:03.179 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:44:03.179 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":2,"fsid":"7a07ff60-0cfd-11ef-bc97-c7b262605968","modified":"2024-05-08T05:43:56.504158Z","created":"2024-05-08T05:41:43.845828Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T05:44:03.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:03.182+0000 7f5541edf700 1 -- 172.21.15.162:0/3057244695 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f552803e2b0 msgr2=0x7f5528040770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:03.182+0000 7f5541edf700 1 --2- 172.21.15.162:0/3057244695 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f552803e2b0 0x7f5528040770 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f552c00fa60 tx=0x7f552c005960 comp rx=0 tx=0).stop 2024-05-08T05:44:03.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:03.182+0000 7f5541edf700 1 -- 172.21.15.162:0/3057244695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f553c075ce0 msgr2=0x7f553c076160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:03.182+0000 7f5541edf700 1 --2- 172.21.15.162:0/3057244695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f553c075ce0 0x7f553c076160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f553000b4d0 tx=0x7f553000b890 comp rx=0 tx=0).stop 2024-05-08T05:44:03.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:03.182+0000 7f5541edf700 1 -- 172.21.15.162:0/3057244695 shutdown_connections 2024-05-08T05:44:03.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:03.182+0000 7f5541edf700 1 --2- 172.21.15.162:0/3057244695 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f552803e2b0 0x7f5528040770 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:03.182+0000 7f5541edf700 1 --2- 172.21.15.162:0/3057244695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55240035c0 0x7f553c0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:03.182+0000 7f5541edf700 1 --2- 172.21.15.162:0/3057244695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f553c075ce0 0x7f553c076160 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:03.182+0000 7f5541edf700 1 -- 172.21.15.162:0/3057244695 >> 172.21.15.162:0/3057244695 conn(0x7f553c0feab0 msgr2=0x7f553c0ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:03.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:03.182+0000 7f5541edf700 1 -- 172.21.15.162:0/3057244695 shutdown_connections 2024-05-08T05:44:03.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:03.182+0000 7f5541edf700 1 -- 172.21.15.162:0/3057244695 wait complete. 2024-05-08T05:44:03.183 INFO:teuthology.orchestra.run.smithi162.stderr:dumped monmap epoch 2 2024-05-08T05:44:03.749 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:03 smithi162 ceph-mon[34800]: from='client.? 172.21.15.162:0/3057244695' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:44:03.967 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:03 smithi112 ceph-mon[30003]: from='client.? 172.21.15.162:0/3057244695' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:44:04.777 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T05:44:04.777 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T05:44:04.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:04 smithi112 ceph-mon[30003]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.021 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:44:05.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:04 smithi162 ceph-mon[34800]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.337 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 systemd[1]: Starting Ceph mon.b for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:44:05.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 podman[34289]: 2024-05-08 05:44:05.523442443 +0000 UTC m=+0.110620079 container create 7241b4923745792aed2fbaeb9a4c8734678da1f048de5fd08182a2e5becbf5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-08T05:44:05.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 podman[34289]: 2024-05-08 05:44:05.431737547 +0000 UTC m=+0.018915198 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:44:05.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 podman[34289]: 2024-05-08 05:44:05.734051128 +0000 UTC m=+0.321228765 container init 7241b4923745792aed2fbaeb9a4c8734678da1f048de5fd08182a2e5becbf5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-08T05:44:05.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 podman[34289]: 2024-05-08 05:44:05.737372582 +0000 UTC m=+0.324550220 container start 7241b4923745792aed2fbaeb9a4c8734678da1f048de5fd08182a2e5becbf5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:44:05.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T05:44:05.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev), process ceph-mon, pid 2 2024-05-08T05:44:05.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pidfile_write: ignore empty --pid-file 2024-05-08T05:44:05.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: load: jerasure load: lrc 2024-05-08T05:44:05.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: RocksDB version: 7.9.2 2024-05-08T05:44:05.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Git sha 0 2024-05-08T05:44:05.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Compile date 2024-05-07 11:18:56 2024-05-08T05:44:05.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: DB SUMMARY 2024-05-08T05:44:05.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: DB Session ID: 7J73YZV6289TD3J2MKPV 2024-05-08T05:44:05.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: CURRENT file: CURRENT 2024-05-08T05:44:05.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T05:44:05.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-08T05:44:05.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-08T05:44:05.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-08T05:44:05.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.error_if_exists: 0 2024-05-08T05:44:05.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.create_if_missing: 0 2024-05-08T05:44:05.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.paranoid_checks: 1 2024-05-08T05:44:05.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T05:44:05.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T05:44:05.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T05:44:05.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.env: 0x55e8a95cdcc0 2024-05-08T05:44:05.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T05:44:05.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.info_log: 0x55e8a9f17da0 2024-05-08T05:44:05.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T05:44:05.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.statistics: (nil) 2024-05-08T05:44:05.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.use_fsync: 0 2024-05-08T05:44:05.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_log_file_size: 0 2024-05-08T05:44:05.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T05:44:05.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T05:44:05.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T05:44:05.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T05:44:05.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.allow_fallocate: 1 2024-05-08T05:44:05.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T05:44:05.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T05:44:05.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.use_direct_reads: 0 2024-05-08T05:44:05.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T05:44:05.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T05:44:05.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.db_log_dir: 2024-05-08T05:44:05.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.wal_dir: 2024-05-08T05:44:05.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T05:44:05.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T05:44:05.937 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T05:44:05.937 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T05:44:05.937 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T05:44:05.937 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T05:44:05.937 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T05:44:05.937 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T05:44:05.937 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.write_buffer_manager: 0x55e8a9ee90e0 2024-05-08T05:44:05.937 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T05:44:05.937 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T05:44:05.937 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T05:44:05.937 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T05:44:05.937 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T05:44:05.937 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T05:44:05.937 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T05:44:05.938 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T05:44:05.938 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.unordered_write: 0 2024-05-08T05:44:05.938 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T05:44:05.938 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T05:44:05.938 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T05:44:05.938 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T05:44:05.938 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.row_cache: None 2024-05-08T05:44:05.938 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.wal_filter: None 2024-05-08T05:44:05.939 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T05:44:05.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T05:44:05.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.two_write_queues: 0 2024-05-08T05:44:05.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T05:44:05.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.wal_compression: 0 2024-05-08T05:44:05.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.atomic_flush: 0 2024-05-08T05:44:05.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T05:44:05.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T05:44:05.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T05:44:05.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.log_readahead_size: 0 2024-05-08T05:44:05.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T05:44:05.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T05:44:05.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T05:44:05.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T05:44:05.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_background_jobs: 2 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_background_compactions: -1 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_subcompactions: 1 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_open_files: -1 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T05:44:05.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T05:44:05.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T05:44:05.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_background_flushes: -1 2024-05-08T05:44:05.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Compression algorithms supported: 2024-05-08T05:44:05.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T05:44:05.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: kZSTD supported: 0 2024-05-08T05:44:05.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: kXpressCompression supported: 0 2024-05-08T05:44:05.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T05:44:05.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: kZlibCompression supported: 1 2024-05-08T05:44:05.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: kSnappyCompression supported: 1 2024-05-08T05:44:05.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: kLZ4Compression supported: 1 2024-05-08T05:44:05.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: kBZip2Compression supported: 0 2024-05-08T05:44:05.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T05:44:05.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T05:44:05.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.merge_operator: 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compaction_filter: None 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e8a9f17f60) 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout: cache_index_and_filter_blocks: 1 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout: pin_top_level_index_and_filter: 1 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout: index_type: 0 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout: data_block_index_type: 0 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout: index_shortening: 1 2024-05-08T05:44:05.943 INFO:journalctl@ceph.mon.b.smithi150.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: checksum: 4 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: no_block_cache: 0 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_cache: 0x55e8a9f1b1f0 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_cache_name: BinnedLRUCache 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_cache_options: 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: capacity : 536870912 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: num_shard_bits : 4 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: strict_capacity_limit : 0 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: high_pri_pool_ratio: 0.000 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_cache_compressed: (nil) 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: persistent_cache: (nil) 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_size: 4096 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_size_deviation: 10 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_restart_interval: 16 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: index_block_restart_interval: 1 2024-05-08T05:44:05.944 INFO:journalctl@ceph.mon.b.smithi150.stdout: metadata_block_size: 4096 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout: partition_filters: 0 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout: use_delta_encoding: 1 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout: filter_policy: bloomfilter 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout: whole_key_filtering: 1 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout: verify_compression: 0 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout: read_amp_bytes_per_bit: 0 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout: format_version: 5 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout: enable_index_compression: 1 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout: block_align: 0 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout: max_auto_readahead_size: 262144 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout: prepopulate_block_cache: 0 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout: initial_auto_readahead_size: 8192 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compression: NoCompression 2024-05-08T05:44:05.945 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T05:44:05.946 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T05:44:05.946 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T05:44:05.946 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.num_levels: 7 2024-05-08T05:44:05.946 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T05:44:05.946 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T05:44:05.946 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T05:44:05.946 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T05:44:05.946 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T05:44:05.946 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T05:44:05.946 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T05:44:05.946 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:44:05.946 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T05:44:05.946 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T05:44:05.946 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:44:05.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:44:05.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T05:44:05.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T05:44:05.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T05:44:05.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T05:44:05.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:44:05.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:44:05.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T05:44:05.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T05:44:05.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:44:05.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T05:44:05.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T05:44:05.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T05:44:05.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T05:44:05.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T05:44:05.948 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T05:44:05.948 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T05:44:05.948 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T05:44:05.948 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T05:44:05.948 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T05:44:05.948 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T05:44:05.948 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T05:44:05.948 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T05:44:05.948 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T05:44:05.948 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T05:44:05.948 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T05:44:05.948 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T05:44:05.948 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T05:44:05.948 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T05:44:05.948 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T05:44:05.949 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T05:44:05.949 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T05:44:05.949 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T05:44:05.949 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T05:44:05.949 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T05:44:05.949 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T05:44:05.949 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T05:44:05.949 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T05:44:05.949 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T05:44:05.949 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T05:44:05.949 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T05:44:05.949 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T05:44:05.949 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-08T05:44:05.949 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.inplace_update_support: 0 2024-05-08T05:44:05.949 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T05:44:05.950 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T05:44:05.950 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T05:44:05.950 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T05:44:05.950 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.bloom_locality: 0 2024-05-08T05:44:05.950 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.max_successive_merges: 0 2024-05-08T05:44:05.950 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T05:44:05.950 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T05:44:05.950 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T05:44:05.950 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T05:44:05.950 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.ttl: 2592000 2024-05-08T05:44:05.950 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T05:44:05.950 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T05:44:05.950 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T05:44:05.950 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.enable_blob_files: false 2024-05-08T05:44:05.951 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.min_blob_size: 0 2024-05-08T05:44:05.951 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T05:44:05.951 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T05:44:05.951 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T05:44:05.951 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T05:44:05.951 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T05:44:05.951 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T05:44:05.951 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T05:44:05.951 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T05:44:05.951 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-08T05:44:05.951 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-08T05:44:05.951 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a3aed597-ad27-4ca5-9bc5-ee1e37cd911b 2024-05-08T05:44:05.951 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715147045768146, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-08T05:44:05.951 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-08T05:44:05.952 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715147045768570, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715147045, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a3aed597-ad27-4ca5-9bc5-ee1e37cd911b", "db_session_id": "7J73YZV6289TD3J2MKPV", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-08T05:44:05.952 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715147045768655, "job": 1, "event": "recovery_finished"} 2024-05-08T05:44:05.952 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-08T05:44:05.952 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T05:44:05.952 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e8aa038000 2024-05-08T05:44:05.952 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: DB pointer 0x55e8aa024000 2024-05-08T05:44:05.952 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T05:44:05.952 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T05:44:05.952 INFO:journalctl@ceph.mon.b.smithi150.stdout: ** DB Stats ** 2024-05-08T05:44:05.952 INFO:journalctl@ceph.mon.b.smithi150.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:44:05.952 INFO:journalctl@ceph.mon.b.smithi150.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T05:44:05.952 INFO:journalctl@ceph.mon.b.smithi150.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:44:05.952 INFO:journalctl@ceph.mon.b.smithi150.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:44:05.952 INFO:journalctl@ceph.mon.b.smithi150.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T05:44:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:44:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:44:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-05-08T05:44:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout: ** Compaction Stats [default] ** 2024-05-08T05:44:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:44:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T05:44:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:44:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:44:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:44:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-05-08T05:44:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout: ** Compaction Stats [default] ** 2024-05-08T05:44:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:44:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T05:44:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:44:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-05-08T05:44:05.954 INFO:journalctl@ceph.mon.b.smithi150.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T05:44:05.954 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-05-08T05:44:05.954 INFO:journalctl@ceph.mon.b.smithi150.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:44:05.954 INFO:journalctl@ceph.mon.b.smithi150.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T05:44:05.954 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T05:44:05.954 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T05:44:05.954 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T05:44:05.954 INFO:journalctl@ceph.mon.b.smithi150.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T05:44:05.954 INFO:journalctl@ceph.mon.b.smithi150.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:44:05.954 INFO:journalctl@ceph.mon.b.smithi150.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:44:05.954 INFO:journalctl@ceph.mon.b.smithi150.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T05:44:05.954 INFO:journalctl@ceph.mon.b.smithi150.stdout: Block cache BinnedLRUCache@0x55e8a9f1b1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-08T05:44:05.954 INFO:journalctl@ceph.mon.b.smithi150.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-08T05:44:05.954 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-05-08T05:44:05.955 INFO:journalctl@ceph.mon.b.smithi150.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T05:44:05.955 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-08T05:44:05.955 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: using public_addr v2:172.21.15.150:0/0 -> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] 2024-05-08T05:44:05.955 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: starting mon.b rank -1 at public addrs [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] at bind addrs [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:44:05.955 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.b@-1(???) e0 preinit fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:44:05.955 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 bash[34289]: 7241b4923745792aed2fbaeb9a4c8734678da1f048de5fd08182a2e5becbf5ef 2024-05-08T05:44:05.955 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.b@-1(synchronizing).mds e1 new map 2024-05-08T05:44:05.955 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-08T05:44:05.955 INFO:journalctl@ceph.mon.b.smithi150.stdout: e1 2024-05-08T05:44:05.955 INFO:journalctl@ceph.mon.b.smithi150.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T05:44:05.955 INFO:journalctl@ceph.mon.b.smithi150.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T05:44:05.955 INFO:journalctl@ceph.mon.b.smithi150.stdout: legacy client fscid: -1 2024-05-08T05:44:05.955 INFO:journalctl@ceph.mon.b.smithi150.stdout: 2024-05-08T05:44:05.955 INFO:journalctl@ceph.mon.b.smithi150.stdout: No filesystems configured 2024-05-08T05:44:05.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T05:44:05.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T05:44:05.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T05:44:05.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T05:44:05.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T05:44:05.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T05:44:05.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T05:44:05.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:44:05.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:44:05.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:44:05.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: Added host smithi150 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:05.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-08T05:44:05.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: Updating smithi150:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:44:05.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:44:05.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi162", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:44:05.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: Updating smithi150:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:44:05.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: Deploying cephadm binary to smithi162 2024-05-08T05:44:05.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:05.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: Added host smithi162 2024-05-08T05:44:05.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:44:05.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi162", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:44:05.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:05.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: Updating smithi162:/etc/ceph/ceph.conf 2024-05-08T05:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/350209145' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T05:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: Updating smithi162:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: Updating smithi162:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/350209145' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T05:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi162:172.21.15.162=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi162:172.21.15.162=c;count:3 2024-05-08T05:44:05.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: Deploying daemon mon.c on smithi162 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.a calling monitor election 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.c calling monitor election 2024-05-08T05:44:05.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:05.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:05.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:05.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T05:44:05.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: monmap epoch 2 2024-05-08T05:44:05.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:44:05.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: last_changed 2024-05-08T05:43:56.504158+0000 2024-05-08T05:44:05.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: created 2024-05-08T05:41:43.845828+0000 2024-05-08T05:44:05.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: min_mon_release 19 (squid) 2024-05-08T05:44:05.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: election_strategy: 1 2024-05-08T05:44:05.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-08T05:44:05.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-05-08T05:44:05.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: fsmap 2024-05-08T05:44:05.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:44:05.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mgrmap e13: x(active, since 58s) 2024-05-08T05:44:05.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: overall HEALTH_OK 2024-05-08T05:44:05.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:05.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:44:05.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:05.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: Deploying daemon mon.b on smithi150 2024-05-08T05:44:05.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:05.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: from='client.? 172.21.15.162:0/3057244695' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:44:05.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:05.964 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 ceph-mon[34315]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T05:44:05.964 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:05 smithi150 systemd[1]: Started Ceph mon.b for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:44:10.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.964+0000 7f0ee5df0700 1 -- 172.21.15.162:0/3545991004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ec80035c0 msgr2=0x7f0ec8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:10.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.964+0000 7f0ee5df0700 1 --2- 172.21.15.162:0/3545991004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ec80035c0 0x7f0ec8005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ee006af00 tx=0x7f0ed8009e80 comp rx=0 tx=0).stop 2024-05-08T05:44:10.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.965+0000 7f0ee5df0700 1 -- 172.21.15.162:0/3545991004 shutdown_connections 2024-05-08T05:44:10.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.965+0000 7f0ee5df0700 1 --2- 172.21.15.162:0/3545991004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ec80035c0 0x7f0ec8005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:10.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.965+0000 7f0ee5df0700 1 -- 172.21.15.162:0/3545991004 >> 172.21.15.162:0/3545991004 conn(0x7f0ee00fd980 msgr2=0x7f0ee00ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:10.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.965+0000 7f0ee5df0700 1 -- 172.21.15.162:0/3545991004 shutdown_connections 2024-05-08T05:44:10.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.965+0000 7f0ee5df0700 1 -- 172.21.15.162:0/3545991004 wait complete. 2024-05-08T05:44:10.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.965+0000 7f0ee5df0700 1 Processor -- start 2024-05-08T05:44:10.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.966+0000 7f0ee5df0700 1 -- start start 2024-05-08T05:44:10.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.966+0000 7f0ee5df0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ec80035c0 0x7f0ee019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:10.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.966+0000 7f0ee5df0700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ee019ed40 0x7f0ee01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:10.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.966+0000 7f0ee5df0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee0198f00 0x7f0ee01993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:10.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.966+0000 7f0ee5df0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee010ae90 con 0x7f0ec80035c0 2024-05-08T05:44:10.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.966+0000 7f0ee5df0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee010ad10 con 0x7f0ee0198f00 2024-05-08T05:44:10.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.966+0000 7f0ee5df0700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee010b010 con 0x7f0ee019ed40 2024-05-08T05:44:10.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.966+0000 7f0edeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ee019ed40 0x7f0ee01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:10.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.966+0000 7f0edeffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ee019ed40 0x7f0ee01989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:36646/0 (socket says 172.21.15.162:36646) 2024-05-08T05:44:10.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.966+0000 7f0edeffd700 1 -- 172.21.15.162:0/3924196520 learned_addr learned my addr 172.21.15.162:0/3924196520 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-08T05:44:10.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.966+0000 7f0edffff700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee0198f00 0x7f0ee01993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:10.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.966+0000 7f0edeffd700 1 -- 172.21.15.162:0/3924196520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ee019ed40 msgr2=0x7f0ee01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-08T05:44:10.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.967+0000 7f0edeffd700 1 -- 172.21.15.162:0/3924196520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ee019ed40 msgr2=0x7f0ee01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T05:44:10.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.967+0000 7f0edeffd700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ee019ed40 0x7f0ee01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T05:44:10.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.967+0000 7f0edf7fe700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ec80035c0 0x7f0ee019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:10.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.967+0000 7f0edeffd700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ee019ed40 0x7f0ee01989c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:44:10.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.967+0000 7f0edffff700 1 -- 172.21.15.162:0/3924196520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee0198f00 msgr2=0x7f0ee01993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-08T05:44:10.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.967+0000 7f0edffff700 1 -- 172.21.15.162:0/3924196520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee0198f00 msgr2=0x7f0ee01993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T05:44:10.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.967+0000 7f0edffff700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee0198f00 0x7f0ee01993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T05:44:10.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.967+0000 7f0edffff700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee0198f00 0x7f0ee01993b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:44:10.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.967+0000 7f0edf7fe700 1 -- 172.21.15.162:0/3924196520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ee019ed40 msgr2=0x7f0ee01989c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:44:10.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.967+0000 7f0edf7fe700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ee019ed40 0x7f0ee01989c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:10.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.967+0000 7f0edf7fe700 1 -- 172.21.15.162:0/3924196520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee0198f00 msgr2=0x7f0ee01993b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:44:10.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.967+0000 7f0edf7fe700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee0198f00 0x7f0ee01993b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:10.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.967+0000 7f0edf7fe700 1 -- 172.21.15.162:0/3924196520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee01a56d0 con 0x7f0ec80035c0 2024-05-08T05:44:10.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.967+0000 7f0edf7fe700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ec80035c0 0x7f0ee019e800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8002230 tx=0x7f0ed8004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:10.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.968+0000 7f0edcff9700 1 -- 172.21.15.162:0/3924196520 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8003c00 con 0x7f0ec80035c0 2024-05-08T05:44:10.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.968+0000 7f0ee5df0700 1 -- 172.21.15.162:0/3924196520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee01a58a0 con 0x7f0ec80035c0 2024-05-08T05:44:10.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.968+0000 7f0ee5df0700 1 -- 172.21.15.162:0/3924196520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee01a5e30 con 0x7f0ec80035c0 2024-05-08T05:44:10.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.968+0000 7f0edcff9700 1 -- 172.21.15.162:0/3924196520 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed8026070 con 0x7f0ec80035c0 2024-05-08T05:44:10.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.968+0000 7f0edcff9700 1 -- 172.21.15.162:0/3924196520 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8030060 con 0x7f0ec80035c0 2024-05-08T05:44:10.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.969+0000 7f0ee5df0700 1 -- 172.21.15.162:0/3924196520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec8002b60 con 0x7f0ec80035c0 2024-05-08T05:44:10.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.969+0000 7f0edcff9700 1 -- 172.21.15.162:0/3924196520 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f0ed8015040 con 0x7f0ec80035c0 2024-05-08T05:44:10.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.969+0000 7f0edcff9700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0ecc03e9a0 0x7f0ecc040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:10.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.969+0000 7f0edcff9700 1 -- 172.21.15.162:0/3924196520 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0ed8054180 con 0x7f0ec80035c0 2024-05-08T05:44:10.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.970+0000 7f0edeffd700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0ecc03e9a0 0x7f0ecc040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:10.972 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.970+0000 7f0edeffd700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0ecc03e9a0 0x7f0ecc040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4009180 tx=0x7f0ed4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:10.985 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:10.985+0000 7f0edcff9700 1 -- 172.21.15.162:0/3924196520 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0ed8005050 con 0x7f0ec80035c0 2024-05-08T05:44:11.282 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:11.281+0000 7f0ee5df0700 1 -- 172.21.15.162:0/3924196520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0ec8002e50 con 0x7f0ec80035c0 2024-05-08T05:44:11.282 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:11.282 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: mon.c calling monitor election 2024-05-08T05:44:11.283 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:44:11.283 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: mon.a calling monitor election 2024-05-08T05:44:11.283 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:11.283 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:11.283 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:11.283 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:11.283 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:11.283 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:11.283 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:11.283 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:11.283 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:11.283 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T05:44:11.284 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: monmap epoch 3 2024-05-08T05:44:11.284 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:44:11.284 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: last_changed 2024-05-08T05:44:05.799572+0000 2024-05-08T05:44:11.284 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: created 2024-05-08T05:41:43.845828+0000 2024-05-08T05:44:11.284 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: min_mon_release 19 (squid) 2024-05-08T05:44:11.284 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: election_strategy: 1 2024-05-08T05:44:11.284 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-08T05:44:11.284 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-05-08T05:44:11.284 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-05-08T05:44:11.284 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: fsmap 2024-05-08T05:44:11.284 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:44:11.284 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: mgrmap e13: x(active, since 67s) 2024-05-08T05:44:11.284 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: overall HEALTH_OK 2024-05-08T05:44:11.285 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:11.285 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:11.285 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:10 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:11.285 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:11.282+0000 7f0edcff9700 1 -- 172.21.15.162:0/3924196520 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f0ed8005890 con 0x7f0ec80035c0 2024-05-08T05:44:11.285 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-08T05:44:11.286 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":3,"fsid":"7a07ff60-0cfd-11ef-bc97-c7b262605968","modified":"2024-05-08T05:44:05.799572Z","created":"2024-05-08T05:41:43.845828Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T05:44:11.286 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:11.285+0000 7f0ee5df0700 1 -- 172.21.15.162:0/3924196520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0ecc03e9a0 msgr2=0x7f0ecc040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:11.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:11.285+0000 7f0ee5df0700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0ecc03e9a0 0x7f0ecc040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4009180 tx=0x7f0ed4008040 comp rx=0 tx=0).stop 2024-05-08T05:44:11.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:11.285+0000 7f0ee5df0700 1 -- 172.21.15.162:0/3924196520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ec80035c0 msgr2=0x7f0ee019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:11.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:11.285+0000 7f0ee5df0700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ec80035c0 0x7f0ee019e800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8002230 tx=0x7f0ed8004420 comp rx=0 tx=0).stop 2024-05-08T05:44:11.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:11.285+0000 7f0ee5df0700 1 -- 172.21.15.162:0/3924196520 shutdown_connections 2024-05-08T05:44:11.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:11.285+0000 7f0ee5df0700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0ecc03e9a0 0x7f0ecc040e60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:11.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:11.285+0000 7f0ee5df0700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee0198f00 0x7f0ee01993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:11.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:11.285+0000 7f0ee5df0700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ec80035c0 0x7f0ee019e800 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:11.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:11.285+0000 7f0ee5df0700 1 --2- 172.21.15.162:0/3924196520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ee019ed40 0x7f0ee01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:11.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:11.285+0000 7f0ee5df0700 1 -- 172.21.15.162:0/3924196520 >> 172.21.15.162:0/3924196520 conn(0x7f0ee00fd980 msgr2=0x7f0ee0105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:11.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:11.285+0000 7f0ee5df0700 1 -- 172.21.15.162:0/3924196520 shutdown_connections 2024-05-08T05:44:11.287 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:11.285+0000 7f0ee5df0700 1 -- 172.21.15.162:0/3924196520 wait complete. 2024-05-08T05:44:11.287 INFO:teuthology.orchestra.run.smithi162.stderr:dumped monmap epoch 3 2024-05-08T05:44:11.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:11.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: mon.c calling monitor election 2024-05-08T05:44:11.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:44:11.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: mon.a calling monitor election 2024-05-08T05:44:11.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:11.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:11.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:11.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:11.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:11.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:11.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:11.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:11.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:11.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T05:44:11.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: monmap epoch 3 2024-05-08T05:44:11.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:44:11.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: last_changed 2024-05-08T05:44:05.799572+0000 2024-05-08T05:44:11.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: created 2024-05-08T05:41:43.845828+0000 2024-05-08T05:44:11.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: min_mon_release 19 (squid) 2024-05-08T05:44:11.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: election_strategy: 1 2024-05-08T05:44:11.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-08T05:44:11.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-05-08T05:44:11.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-05-08T05:44:11.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: fsmap 2024-05-08T05:44:11.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:44:11.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: mgrmap e13: x(active, since 67s) 2024-05-08T05:44:11.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: overall HEALTH_OK 2024-05-08T05:44:11.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:11.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:11.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:11.937 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-08T05:44:11.937 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-08T05:44:12.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:11 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:12.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:11 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:12.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:11 smithi112 ceph-mon[30003]: from='client.? 172.21.15.162:0/3924196520' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:44:12.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:11 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:12.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:11 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:12.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:11 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:12.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:11 smithi162 ceph-mon[34800]: from='client.? 172.21.15.162:0/3924196520' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:44:12.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:11 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:12.354 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:44:13.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:13.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: mon.c calling monitor election 2024-05-08T05:44:13.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:44:13.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: mon.a calling monitor election 2024-05-08T05:44:13.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:13.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:44:13.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:13.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:13.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:13.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:13.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:13.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:13.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:13.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T05:44:13.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: monmap epoch 3 2024-05-08T05:44:13.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:44:13.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: last_changed 2024-05-08T05:44:05.799572+0000 2024-05-08T05:44:13.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: created 2024-05-08T05:41:43.845828+0000 2024-05-08T05:44:13.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: min_mon_release 19 (squid) 2024-05-08T05:44:13.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: election_strategy: 1 2024-05-08T05:44:13.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-08T05:44:13.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-05-08T05:44:13.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-05-08T05:44:13.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: fsmap 2024-05-08T05:44:13.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:44:13.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: mgrmap e13: x(active, since 67s) 2024-05-08T05:44:13.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: overall HEALTH_OK 2024-05-08T05:44:13.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:13.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:13.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:13.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:13.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:13.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: from='client.? 172.21.15.162:0/3924196520' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:44:13.186 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:13.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: mon.b calling monitor election 2024-05-08T05:44:13.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: mon.b calling monitor election 2024-05-08T05:44:13.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: mon.c calling monitor election 2024-05-08T05:44:13.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:13.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: mon.a calling monitor election 2024-05-08T05:44:13.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T05:44:13.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:13.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: monmap epoch 3 2024-05-08T05:44:13.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:44:13.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: last_changed 2024-05-08T05:44:05.799572+0000 2024-05-08T05:44:13.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: created 2024-05-08T05:41:43.845828+0000 2024-05-08T05:44:13.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: min_mon_release 19 (squid) 2024-05-08T05:44:13.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: election_strategy: 1 2024-05-08T05:44:13.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-08T05:44:13.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-05-08T05:44:13.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-05-08T05:44:13.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: fsmap 2024-05-08T05:44:13.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:44:13.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: mgrmap e13: x(active, since 69s) 2024-05-08T05:44:13.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: overall HEALTH_OK 2024-05-08T05:44:13.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:13.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: Updating smithi112:/etc/ceph/ceph.conf 2024-05-08T05:44:13.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-08T05:44:13.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: Updating smithi162:/etc/ceph/ceph.conf 2024-05-08T05:44:13.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: Updating smithi150:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:44:13.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: Updating smithi162:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:44:13.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:13.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:13.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:13.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:13.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:13.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:13 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:14.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: mon.b calling monitor election 2024-05-08T05:44:14.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: mon.b calling monitor election 2024-05-08T05:44:14.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: mon.c calling monitor election 2024-05-08T05:44:14.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:14.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: mon.a calling monitor election 2024-05-08T05:44:14.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T05:44:14.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:14.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: monmap epoch 3 2024-05-08T05:44:14.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:44:14.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: last_changed 2024-05-08T05:44:05.799572+0000 2024-05-08T05:44:14.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: created 2024-05-08T05:41:43.845828+0000 2024-05-08T05:44:14.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: min_mon_release 19 (squid) 2024-05-08T05:44:14.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: election_strategy: 1 2024-05-08T05:44:14.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-08T05:44:14.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-05-08T05:44:14.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-05-08T05:44:14.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: fsmap 2024-05-08T05:44:14.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:44:14.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: mgrmap e13: x(active, since 69s) 2024-05-08T05:44:14.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: overall HEALTH_OK 2024-05-08T05:44:14.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:14.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: Updating smithi112:/etc/ceph/ceph.conf 2024-05-08T05:44:14.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-08T05:44:14.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: Updating smithi162:/etc/ceph/ceph.conf 2024-05-08T05:44:14.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: Updating smithi150:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:44:14.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: Updating smithi162:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:44:14.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:14.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:14.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:14.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:14.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:14.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:13 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:14.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: mon.b calling monitor election 2024-05-08T05:44:14.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: mon.b calling monitor election 2024-05-08T05:44:14.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: mon.c calling monitor election 2024-05-08T05:44:14.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:14.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: mon.a calling monitor election 2024-05-08T05:44:14.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T05:44:14.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:14.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: monmap epoch 3 2024-05-08T05:44:14.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:44:14.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: last_changed 2024-05-08T05:44:05.799572+0000 2024-05-08T05:44:14.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: created 2024-05-08T05:41:43.845828+0000 2024-05-08T05:44:14.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: min_mon_release 19 (squid) 2024-05-08T05:44:14.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: election_strategy: 1 2024-05-08T05:44:14.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-05-08T05:44:14.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-05-08T05:44:14.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-05-08T05:44:14.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: fsmap 2024-05-08T05:44:14.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:44:14.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: mgrmap e13: x(active, since 69s) 2024-05-08T05:44:14.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: overall HEALTH_OK 2024-05-08T05:44:14.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:14.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: Updating smithi112:/etc/ceph/ceph.conf 2024-05-08T05:44:14.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-08T05:44:14.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: Updating smithi162:/etc/ceph/ceph.conf 2024-05-08T05:44:14.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: Updating smithi150:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:44:14.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: Updating smithi162:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:44:14.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:14.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:14.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:14.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:14.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:14.186 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:13 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:44:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.446+0000 7fb1f3400700 1 -- 172.21.15.112:0/2751102672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1ec101fe0 msgr2=0x7fb1ec1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.446+0000 7fb1f3400700 1 --2- 172.21.15.112:0/2751102672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1ec101fe0 0x7fb1ec1023c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009910 tx=0x7fb1e0009c20 comp rx=0 tx=0).stop 2024-05-08T05:44:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.446+0000 7fb1f3400700 1 -- 172.21.15.112:0/2751102672 shutdown_connections 2024-05-08T05:44:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.446+0000 7fb1f3400700 1 --2- 172.21.15.112:0/2751102672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1ec101fe0 0x7fb1ec1023c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.446+0000 7fb1f3400700 1 -- 172.21.15.112:0/2751102672 >> 172.21.15.112:0/2751102672 conn(0x7fb1ec078a70 msgr2=0x7fb1ec0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.446+0000 7fb1f3400700 1 -- 172.21.15.112:0/2751102672 shutdown_connections 2024-05-08T05:44:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.446+0000 7fb1f3400700 1 -- 172.21.15.112:0/2751102672 wait complete. 2024-05-08T05:44:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.447+0000 7fb1f3400700 1 Processor -- start 2024-05-08T05:44:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.448+0000 7fb1f3400700 1 -- start start 2024-05-08T05:44:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.448+0000 7fb1f3400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1ec101fe0 0x7fb1ec19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.448+0000 7fb1f3400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1ec19b2d0 0x7fb1ec19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.448+0000 7fb1f3400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1ec19fb50 0x7fb1ec1a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.448+0000 7fb1f3400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec10c260 con 0x7fb1ec19fb50 2024-05-08T05:44:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.448+0000 7fb1f3400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec10c0e0 con 0x7fb1ec101fe0 2024-05-08T05:44:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.448+0000 7fb1f3400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec10c3e0 con 0x7fb1ec19b2d0 2024-05-08T05:44:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.448+0000 7fb1f0a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1ec101fe0 0x7fb1ec19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.448+0000 7fb1f0a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1ec101fe0 0x7fb1ec19ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:50164/0 (socket says 172.21.15.112:50164) 2024-05-08T05:44:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.448+0000 7fb1f0a00700 1 -- 172.21.15.112:0/3294448188 learned_addr learned my addr 172.21.15.112:0/3294448188 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:44:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.448+0000 7fb1f1400700 1 --2- 172.21.15.112:0/3294448188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1ec19fb50 0x7fb1ec1a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.448+0000 7fb1ebe00700 1 --2- 172.21.15.112:0/3294448188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1ec19b2d0 0x7fb1ec19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.449+0000 7fb1f0a00700 1 -- 172.21.15.112:0/3294448188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1ec19b2d0 msgr2=0x7fb1ec19f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.449+0000 7fb1f0a00700 1 --2- 172.21.15.112:0/3294448188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1ec19b2d0 0x7fb1ec19f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.449+0000 7fb1f0a00700 1 -- 172.21.15.112:0/3294448188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1ec19fb50 msgr2=0x7fb1ec1a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.449+0000 7fb1f0a00700 1 --2- 172.21.15.112:0/3294448188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1ec19fb50 0x7fb1ec1a0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.449+0000 7fb1f0a00700 1 -- 172.21.15.112:0/3294448188 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ec1a78c0 con 0x7fb1ec101fe0 2024-05-08T05:44:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.449+0000 7fb1f0a00700 1 --2- 172.21.15.112:0/3294448188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1ec101fe0 0x7fb1ec19ad90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0004f80 tx=0x7fb1e0004fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.449+0000 7fb1e9600700 1 -- 172.21.15.112:0/3294448188 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0026070 con 0x7fb1ec101fe0 2024-05-08T05:44:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.449+0000 7fb1e9600700 1 -- 172.21.15.112:0/3294448188 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e000b6f0 con 0x7fb1ec101fe0 2024-05-08T05:44:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.449+0000 7fb1f3400700 1 -- 172.21.15.112:0/3294448188 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ec1a7af0 con 0x7fb1ec101fe0 2024-05-08T05:44:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.449+0000 7fb1e9600700 1 -- 172.21.15.112:0/3294448188 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0009e60 con 0x7fb1ec101fe0 2024-05-08T05:44:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.450+0000 7fb1f3400700 1 -- 172.21.15.112:0/3294448188 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ec1a8080 con 0x7fb1ec101fe0 2024-05-08T05:44:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.450+0000 7fb1e9600700 1 -- 172.21.15.112:0/3294448188 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fb1e0029070 con 0x7fb1ec101fe0 2024-05-08T05:44:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.451+0000 7fb1e9600700 1 --2- 172.21.15.112:0/3294448188 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb1d403e9a0 0x7fb1d4040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.451+0000 7fb1f3400700 1 -- 172.21.15.112:0/3294448188 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ec1034c0 con 0x7fb1ec101fe0 2024-05-08T05:44:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.451+0000 7fb1e9600700 1 -- 172.21.15.112:0/3294448188 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb1e0004260 con 0x7fb1ec101fe0 2024-05-08T05:44:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.451+0000 7fb1ebe00700 1 --2- 172.21.15.112:0/3294448188 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb1d403e9a0 0x7fb1d4040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:14.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.452+0000 7fb1ebe00700 1 --2- 172.21.15.112:0/3294448188 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb1d403e9a0 0x7fb1d4040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009ad0 tx=0x7fb1d8009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:14.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.457+0000 7fb1e9600700 1 -- 172.21.15.112:0/3294448188 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb1e0013070 con 0x7fb1ec101fe0 2024-05-08T05:44:14.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.687+0000 7fb1f3400700 1 -- 172.21.15.112:0/3294448188 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb1ec1a8250 con 0x7fb1ec101fe0 2024-05-08T05:44:14.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.688+0000 7fb1e9600700 1 -- 172.21.15.112:0/3294448188 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7fb1e000ee70 con 0x7fb1ec101fe0 2024-05-08T05:44:14.689 INFO:teuthology.orchestra.run.smithi112.stdout:# minimal ceph.conf for 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:44:14.689 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-05-08T05:44:14.689 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:44:14.689 INFO:teuthology.orchestra.run.smithi112.stdout: mon_host = [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-05-08T05:44:14.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.691+0000 7fb1f3400700 1 -- 172.21.15.112:0/3294448188 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb1d403e9a0 msgr2=0x7fb1d4040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:14.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.691+0000 7fb1f3400700 1 --2- 172.21.15.112:0/3294448188 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb1d403e9a0 0x7fb1d4040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009ad0 tx=0x7fb1d8009210 comp rx=0 tx=0).stop 2024-05-08T05:44:14.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.691+0000 7fb1f3400700 1 -- 172.21.15.112:0/3294448188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1ec101fe0 msgr2=0x7fb1ec19ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:14.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.691+0000 7fb1f3400700 1 --2- 172.21.15.112:0/3294448188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1ec101fe0 0x7fb1ec19ad90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0004f80 tx=0x7fb1e0004fb0 comp rx=0 tx=0).stop 2024-05-08T05:44:14.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.691+0000 7fb1f3400700 1 -- 172.21.15.112:0/3294448188 shutdown_connections 2024-05-08T05:44:14.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.691+0000 7fb1f3400700 1 --2- 172.21.15.112:0/3294448188 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb1d403e9a0 0x7fb1d4040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:14.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.691+0000 7fb1f3400700 1 --2- 172.21.15.112:0/3294448188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1ec19fb50 0x7fb1ec1a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:14.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.691+0000 7fb1f3400700 1 --2- 172.21.15.112:0/3294448188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1ec101fe0 0x7fb1ec19ad90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:14.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.691+0000 7fb1f3400700 1 --2- 172.21.15.112:0/3294448188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1ec19b2d0 0x7fb1ec19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:14.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.691+0000 7fb1f3400700 1 -- 172.21.15.112:0/3294448188 >> 172.21.15.112:0/3294448188 conn(0x7fb1ec078a70 msgr2=0x7fb1ec1059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:14.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.691+0000 7fb1f3400700 1 -- 172.21.15.112:0/3294448188 shutdown_connections 2024-05-08T05:44:14.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:14.691+0000 7fb1f3400700 1 -- 172.21.15.112:0/3294448188 wait complete. 2024-05-08T05:44:15.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:15 smithi150 ceph-mon[34315]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:15.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:15 smithi150 ceph-mon[34315]: Updating smithi112:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:44:15.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:15 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:15 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:15 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:15 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:15 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:15 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:15 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:15 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:44:15.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:15 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:44:15.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:15 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:15.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:15 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3294448188' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:15.449 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:15 smithi112 ceph-mon[30003]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:15.449 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:15 smithi112 ceph-mon[30003]: Updating smithi112:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:44:15.449 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:15 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.449 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:15 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.449 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:15 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.449 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:15 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.449 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:15 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.449 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:15 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.449 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:15 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.449 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:15 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:44:15.449 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:15 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:44:15.449 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:15 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:15.450 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:15 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3294448188' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:15.510 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-08T05:44:15.511 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:44:15.511 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T05:44:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:15 smithi162 ceph-mon[34800]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:15 smithi162 ceph-mon[34800]: Updating smithi112:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:44:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:15 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:15 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:15 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:15 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:15 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:15 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:15 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:15 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:44:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:15 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:44:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:15 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:15.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:15 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3294448188' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:15.589 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:44:15.589 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:44:15.720 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-08T05:44:15.720 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T05:44:15.752 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-08T05:44:15.752 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:44:15.819 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-08T05:44:15.819 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T05:44:15.846 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-08T05:44:15.846 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:44:15.910 INFO:tasks.cephadm:Adding mgr.x on smithi112 2024-05-08T05:44:15.910 INFO:tasks.cephadm:Adding mgr.y on smithi150 2024-05-08T05:44:15.910 INFO:tasks.cephadm:Adding mgr.z on smithi162 2024-05-08T05:44:15.910 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi112=x;smithi150=y;smithi162=z' 2024-05-08T05:44:16.198 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:44:16.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:16 smithi112 ceph-mon[30003]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T05:44:16.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:16 smithi112 ceph-mon[30003]: Reconfiguring daemon mon.a on smithi112 2024-05-08T05:44:16.387 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:16 smithi150 ceph-mon[34315]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T05:44:16.387 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:16 smithi150 ceph-mon[34315]: Reconfiguring daemon mon.a on smithi112 2024-05-08T05:44:16.438 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:16 smithi162 ceph-mon[34800]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T05:44:16.438 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:16 smithi162 ceph-mon[34800]: Reconfiguring daemon mon.a on smithi112 2024-05-08T05:44:17.378 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:17 smithi162 ceph-mon[34800]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:17.378 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:17 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:17.378 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:17 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:17.378 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:17 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:44:17.378 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:17 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:44:17.378 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:17 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:17 smithi150 ceph-mon[34315]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:17 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:17 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:17 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:44:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:17 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:44:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:17 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:17.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:17 smithi112 ceph-mon[30003]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:17.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:17 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:17.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:17 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:17.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:17 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:44:17.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:17 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:44:17.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:17 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:17.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.587+0000 7efe4840c700 1 -- 172.21.15.162:0/4143880382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe40101f00 msgr2=0x7efe40102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:17.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.587+0000 7efe4840c700 1 --2- 172.21.15.162:0/4143880382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe40101f00 0x7efe40102340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009910 tx=0x7efe3c009c20 comp rx=0 tx=0).stop 2024-05-08T05:44:17.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.587+0000 7efe4840c700 1 -- 172.21.15.162:0/4143880382 shutdown_connections 2024-05-08T05:44:17.589 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.587+0000 7efe4840c700 1 --2- 172.21.15.162:0/4143880382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe40101f00 0x7efe40102340 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:17.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.587+0000 7efe4840c700 1 -- 172.21.15.162:0/4143880382 >> 172.21.15.162:0/4143880382 conn(0x7efe400fd510 msgr2=0x7efe400ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:17.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.587+0000 7efe4840c700 1 -- 172.21.15.162:0/4143880382 shutdown_connections 2024-05-08T05:44:17.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.587+0000 7efe4840c700 1 -- 172.21.15.162:0/4143880382 wait complete. 2024-05-08T05:44:17.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe4840c700 1 Processor -- start 2024-05-08T05:44:17.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe4840c700 1 -- start start 2024-05-08T05:44:17.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe4840c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe40101f00 0x7efe4006c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:17.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe4840c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe4019b540 0x7efe4019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:17.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe4840c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe4019dec0 0x7efe401a0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:17.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe4840c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe40107df0 con 0x7efe4019b540 2024-05-08T05:44:17.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe4840c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efe40107c70 con 0x7efe4019dec0 2024-05-08T05:44:17.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe4840c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7efe40107f70 con 0x7efe40101f00 2024-05-08T05:44:17.590 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe461a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe40101f00 0x7efe4006c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:17.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe461a8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe40101f00 0x7efe4006c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:36648/0 (socket says 172.21.15.162:36648) 2024-05-08T05:44:17.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe461a8700 1 -- 172.21.15.162:0/1862144979 learned_addr learned my addr 172.21.15.162:0/1862144979 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-08T05:44:17.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe469a9700 1 --2- 172.21.15.162:0/1862144979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe4019dec0 0x7efe401a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:17.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe461a8700 1 -- 172.21.15.162:0/1862144979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe4019dec0 msgr2=0x7efe401a0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:17.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe461a8700 1 --2- 172.21.15.162:0/1862144979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe4019dec0 0x7efe401a0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:17.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe461a8700 1 -- 172.21.15.162:0/1862144979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe4019b540 msgr2=0x7efe4019d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:17.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe461a8700 1 --2- 172.21.15.162:0/1862144979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe4019b540 0x7efe4019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:17.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe461a8700 1 -- 172.21.15.162:0/1862144979 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe401a0c20 con 0x7efe40101f00 2024-05-08T05:44:17.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.588+0000 7efe461a8700 1 --2- 172.21.15.162:0/1862144979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe40101f00 0x7efe4006c210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0059d0 tx=0x7efe3c0050d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:17.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.589+0000 7efe377fe700 1 -- 172.21.15.162:0/1862144979 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c027070 con 0x7efe40101f00 2024-05-08T05:44:17.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.589+0000 7efe4840c700 1 -- 172.21.15.162:0/1862144979 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe401a0eb0 con 0x7efe40101f00 2024-05-08T05:44:17.591 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.589+0000 7efe377fe700 1 -- 172.21.15.162:0/1862144979 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe3c00be10 con 0x7efe40101f00 2024-05-08T05:44:17.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.589+0000 7efe377fe700 1 -- 172.21.15.162:0/1862144979 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c022e20 con 0x7efe40101f00 2024-05-08T05:44:17.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.589+0000 7efe4840c700 1 -- 172.21.15.162:0/1862144979 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe401a13e0 con 0x7efe40101f00 2024-05-08T05:44:17.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.589+0000 7efe377fe700 1 -- 172.21.15.162:0/1862144979 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7efe3c02a070 con 0x7efe40101f00 2024-05-08T05:44:17.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.589+0000 7efe4840c700 1 -- 172.21.15.162:0/1862144979 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe40064b00 con 0x7efe40101f00 2024-05-08T05:44:17.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.590+0000 7efe377fe700 1 --2- 172.21.15.162:0/1862144979 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efe2c03e9a0 0x7efe2c040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:17.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.590+0000 7efe377fe700 1 -- 172.21.15.162:0/1862144979 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efe3c053a30 con 0x7efe40101f00 2024-05-08T05:44:17.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.590+0000 7efe459a7700 1 --2- 172.21.15.162:0/1862144979 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efe2c03e9a0 0x7efe2c040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:17.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.591+0000 7efe459a7700 1 --2- 172.21.15.162:0/1862144979 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efe2c03e9a0 0x7efe2c040e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efe4006c790 tx=0x7efe30006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:17.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.595+0000 7efe377fe700 1 -- 172.21.15.162:0/1862144979 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe3c013070 con 0x7efe40101f00 2024-05-08T05:44:17.809 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.809+0000 7efe4840c700 1 -- 172.21.15.162:0/1862144979 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi150=y;smithi162=z", "target": ["mon-mgr", ""]}) -- 0x7efe400ff430 con 0x7efe2c03e9a0 2024-05-08T05:44:17.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.823+0000 7efe377fe700 1 -- 172.21.15.162:0/1862144979 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7efe400ff430 con 0x7efe2c03e9a0 2024-05-08T05:44:17.824 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled mgr update... 2024-05-08T05:44:17.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.826+0000 7efe4840c700 1 -- 172.21.15.162:0/1862144979 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efe2c03e9a0 msgr2=0x7efe2c040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:17.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.826+0000 7efe4840c700 1 --2- 172.21.15.162:0/1862144979 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efe2c03e9a0 0x7efe2c040e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efe4006c790 tx=0x7efe30006dd0 comp rx=0 tx=0).stop 2024-05-08T05:44:17.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.826+0000 7efe4840c700 1 -- 172.21.15.162:0/1862144979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe40101f00 msgr2=0x7efe4006c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:17.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.826+0000 7efe4840c700 1 --2- 172.21.15.162:0/1862144979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe40101f00 0x7efe4006c210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0059d0 tx=0x7efe3c0050d0 comp rx=0 tx=0).stop 2024-05-08T05:44:17.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.826+0000 7efe4840c700 1 -- 172.21.15.162:0/1862144979 shutdown_connections 2024-05-08T05:44:17.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.826+0000 7efe4840c700 1 --2- 172.21.15.162:0/1862144979 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efe2c03e9a0 0x7efe2c040e60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:17.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.826+0000 7efe4840c700 1 --2- 172.21.15.162:0/1862144979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe4019dec0 0x7efe401a0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:17.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.826+0000 7efe4840c700 1 --2- 172.21.15.162:0/1862144979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe40101f00 0x7efe4006c210 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:17.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.826+0000 7efe4840c700 1 --2- 172.21.15.162:0/1862144979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe4019b540 0x7efe4019d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:17.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.826+0000 7efe4840c700 1 -- 172.21.15.162:0/1862144979 >> 172.21.15.162:0/1862144979 conn(0x7efe400fd510 msgr2=0x7efe400fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:17.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.826+0000 7efe4840c700 1 -- 172.21.15.162:0/1862144979 shutdown_connections 2024-05-08T05:44:17.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:44:17.826+0000 7efe4840c700 1 -- 172.21.15.162:0/1862144979 wait complete. 2024-05-08T05:44:18.394 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:18 smithi162 ceph-mon[34800]: Reconfiguring mon.b (monmap changed)... 2024-05-08T05:44:18.395 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:18 smithi162 ceph-mon[34800]: Reconfiguring daemon mon.b on smithi150 2024-05-08T05:44:18.395 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:18 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:18.422 DEBUG:teuthology.orchestra.run.smithi150:mgr.y> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.y.service 2024-05-08T05:44:18.425 DEBUG:teuthology.orchestra.run.smithi162:mgr.z> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.z.service 2024-05-08T05:44:18.427 INFO:tasks.cephadm:Deploying OSDs... 2024-05-08T05:44:18.428 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:44:18.428 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T05:44:18.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:18 smithi150 ceph-mon[34315]: Reconfiguring mon.b (monmap changed)... 2024-05-08T05:44:18.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:18 smithi150 ceph-mon[34315]: Reconfiguring daemon mon.b on smithi150 2024-05-08T05:44:18.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:18 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:18.484 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T05:44:18.484 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-05-08T05:44:18.534 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T05:44:18.534 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:44:18.534 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-05-08T05:44:18.534 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:44:18.534 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:44:18.534 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-05-08 05:43:16.536791245 +0000 2024-05-08T05:44:18.534 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-05-08 05:36:46.804017586 +0000 2024-05-08T05:44:18.534 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-05-08 05:36:46.804017586 +0000 2024-05-08T05:44:18.535 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-05-08 05:36:46.804017586 +0000 2024-05-08T05:44:18.535 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T05:44:18.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:18 smithi112 ceph-mon[30003]: Reconfiguring mon.b (monmap changed)... 2024-05-08T05:44:18.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:18 smithi112 ceph-mon[30003]: Reconfiguring daemon mon.b on smithi150 2024-05-08T05:44:18.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:18.610 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-05-08T05:44:18.610 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-05-08T05:44:18.610 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000391278 s, 1.3 MB/s 2024-05-08T05:44:18.613 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T05:44:18.711 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-05-08T05:44:18.759 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T05:44:18.759 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:44:18.759 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-05-08T05:44:18.759 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:44:18.759 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:44:18.759 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-05-08 05:43:16.603789524 +0000 2024-05-08T05:44:18.759 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-05-08 05:36:47.391002439 +0000 2024-05-08T05:44:18.759 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-05-08 05:36:47.391002439 +0000 2024-05-08T05:44:18.759 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-05-08 05:36:47.391002439 +0000 2024-05-08T05:44:18.759 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T05:44:18.836 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-05-08T05:44:18.836 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-05-08T05:44:18.836 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000368912 s, 1.4 MB/s 2024-05-08T05:44:18.840 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T05:44:18.895 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-05-08T05:44:18.946 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T05:44:18.946 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:44:18.946 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-05-08T05:44:18.946 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:44:18.946 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:44:18.946 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-05-08 05:43:16.673787727 +0000 2024-05-08T05:44:18.946 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-05-08 05:36:48.027986003 +0000 2024-05-08T05:44:18.946 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-05-08 05:36:48.027986003 +0000 2024-05-08T05:44:18.946 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-05-08 05:36:48.027986003 +0000 2024-05-08T05:44:18.946 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T05:44:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-05-08T05:44:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-05-08T05:44:19.023 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000546511 s, 937 kB/s 2024-05-08T05:44:19.027 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T05:44:19.081 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-05-08T05:44:19.131 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T05:44:19.132 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:44:19.132 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-05-08T05:44:19.132 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:44:19.132 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:44:19.132 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-05-08 05:43:16.756785595 +0000 2024-05-08T05:44:19.132 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-05-08 05:36:48.604971115 +0000 2024-05-08T05:44:19.132 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-05-08 05:36:48.604971115 +0000 2024-05-08T05:44:19.132 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-05-08 05:36:48.604971115 +0000 2024-05-08T05:44:19.132 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T05:44:19.199 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:19 smithi162 ceph-mon[34800]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:19.199 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:19 smithi162 ceph-mon[34800]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi150=y;smithi162=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:44:19.199 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:19 smithi162 ceph-mon[34800]: Saving service mgr spec with placement smithi112=x;smithi150=y;smithi162=z;count:3 2024-05-08T05:44:19.199 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:19 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:19.199 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:19 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:19.199 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:19 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:44:19.199 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:19 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:44:19.199 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:19 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:19.208 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-05-08T05:44:19.209 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-05-08T05:44:19.209 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000419156 s, 1.2 MB/s 2024-05-08T05:44:19.212 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T05:44:19.266 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-08T05:44:19.266 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T05:44:19.283 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T05:44:19.283 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-05-08T05:44:19.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:19 smithi112 ceph-mon[30003]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:19.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:19 smithi112 ceph-mon[30003]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi150=y;smithi162=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:44:19.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:19 smithi112 ceph-mon[30003]: Saving service mgr spec with placement smithi112=x;smithi150=y;smithi162=z;count:3 2024-05-08T05:44:19.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:19 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:19.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:19 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:19.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:19 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:44:19.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:19 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:44:19.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:19 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:19.339 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T05:44:19.339 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:44:19.339 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-08T05:44:19.339 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:44:19.339 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:44:19.339 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-08 05:44:05.217465381 +0000 2024-05-08T05:44:19.339 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-08 05:35:40.230070479 +0000 2024-05-08T05:44:19.339 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-08 05:35:40.230070479 +0000 2024-05-08T05:44:19.339 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-05-08 05:35:40.230070479 +0000 2024-05-08T05:44:19.339 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T05:44:19.400 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:19 smithi150 ceph-mon[34315]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:19.401 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:19 smithi150 ceph-mon[34315]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi150=y;smithi162=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:44:19.401 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:19 smithi150 ceph-mon[34315]: Saving service mgr spec with placement smithi112=x;smithi150=y;smithi162=z;count:3 2024-05-08T05:44:19.401 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:19 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:19.401 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:19 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:19.401 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:19 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:44:19.401 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:19 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:44:19.401 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:19 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:19.403 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-08T05:44:19.403 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-08T05:44:19.403 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000211332 s, 2.4 MB/s 2024-05-08T05:44:19.404 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T05:44:19.461 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-05-08T05:44:19.516 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T05:44:19.516 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:44:19.516 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-05-08T05:44:19.516 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:44:19.516 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:44:19.516 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-08 05:44:05.217465381 +0000 2024-05-08T05:44:19.516 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-08 05:35:40.586063438 +0000 2024-05-08T05:44:19.516 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-08 05:35:40.586063438 +0000 2024-05-08T05:44:19.516 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-05-08 05:35:40.586063438 +0000 2024-05-08T05:44:19.516 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T05:44:19.579 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-08T05:44:19.579 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-08T05:44:19.579 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000178716 s, 2.9 MB/s 2024-05-08T05:44:19.580 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T05:44:19.635 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-05-08T05:44:19.692 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T05:44:19.692 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:44:19.692 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-05-08T05:44:19.692 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:44:19.692 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:44:19.692 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-08 05:44:05.218465361 +0000 2024-05-08T05:44:19.692 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-08 05:35:40.928056675 +0000 2024-05-08T05:44:19.692 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-08 05:35:40.928056675 +0000 2024-05-08T05:44:19.692 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-05-08 05:35:40.928056675 +0000 2024-05-08T05:44:19.692 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T05:44:19.754 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-08T05:44:19.754 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-08T05:44:19.754 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000214451 s, 2.4 MB/s 2024-05-08T05:44:19.755 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T05:44:19.811 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-05-08T05:44:19.866 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T05:44:19.866 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:44:19.866 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-05-08T05:44:19.866 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:44:19.866 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:44:19.866 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-08 05:44:05.218465361 +0000 2024-05-08T05:44:19.866 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-08 05:35:41.260050108 +0000 2024-05-08T05:44:19.866 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-08 05:35:41.260050108 +0000 2024-05-08T05:44:19.866 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-05-08 05:35:41.260050108 +0000 2024-05-08T05:44:19.867 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T05:44:19.928 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-08T05:44:19.928 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-08T05:44:19.928 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000171415 s, 3.0 MB/s 2024-05-08T05:44:19.929 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T05:44:19.984 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-08T05:44:19.984 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T05:44:20.008 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T05:44:20.008 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_1 2024-05-08T05:44:20.063 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T05:44:20.064 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:44:20.064 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-08T05:44:20.064 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:44:20.064 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:44:20.064 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-05-08 05:43:55.611828380 +0000 2024-05-08T05:44:20.064 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-05-08 05:35:41.205104391 +0000 2024-05-08T05:44:20.064 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-05-08 05:35:41.205104391 +0000 2024-05-08T05:44:20.064 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-05-08 05:35:41.205104391 +0000 2024-05-08T05:44:20.064 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T05:44:20.129 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-05-08T05:44:20.129 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-05-08T05:44:20.129 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000277531 s, 1.8 MB/s 2024-05-08T05:44:20.130 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T05:44:20.186 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_2 2024-05-08T05:44:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T05:44:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:44:20.242 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-08T05:44:20.242 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:44:20.242 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:44:20.242 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-05-08 05:43:55.612828359 +0000 2024-05-08T05:44:20.242 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-05-08 05:35:41.569096772 +0000 2024-05-08T05:44:20.242 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-05-08 05:35:41.569096772 +0000 2024-05-08T05:44:20.242 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-05-08 05:35:41.569096772 +0000 2024-05-08T05:44:20.243 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T05:44:20.304 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-05-08T05:44:20.304 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-05-08T05:44:20.304 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000203634 s, 2.5 MB/s 2024-05-08T05:44:20.305 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T05:44:20.361 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_3 2024-05-08T05:44:20.415 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T05:44:20.415 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:44:20.415 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-08T05:44:20.416 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:44:20.416 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:44:20.416 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-05-08 05:43:55.612828359 +0000 2024-05-08T05:44:20.416 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-05-08 05:35:41.908089677 +0000 2024-05-08T05:44:20.416 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-05-08 05:35:41.908089677 +0000 2024-05-08T05:44:20.416 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-05-08 05:35:41.908089677 +0000 2024-05-08T05:44:20.416 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T05:44:20.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:20 smithi150 ceph-mon[34315]: Reconfiguring mon.c (monmap changed)... 2024-05-08T05:44:20.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:20 smithi150 ceph-mon[34315]: Reconfiguring daemon mon.c on smithi162 2024-05-08T05:44:20.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:20 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:20.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:20 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:20.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:20 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:20.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:20 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:20.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:20 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:20.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:20 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:20.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:20 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:44:20.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:20 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:44:20.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:20 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:44:20.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:20 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:20.477 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:20 smithi162 ceph-mon[34800]: Reconfiguring mon.c (monmap changed)... 2024-05-08T05:44:20.477 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:20 smithi162 ceph-mon[34800]: Reconfiguring daemon mon.c on smithi162 2024-05-08T05:44:20.478 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:20 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:20.478 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:20 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:20.478 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:20 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:20.478 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:20 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:20.478 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:20 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:20.478 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:20 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:20.478 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:20 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:44:20.478 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:20 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:44:20.478 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:20 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:44:20.478 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:20 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:20.480 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-05-08T05:44:20.480 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-05-08T05:44:20.480 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000178121 s, 2.9 MB/s 2024-05-08T05:44:20.481 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T05:44:20.536 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_4 2024-05-08T05:44:20.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:20 smithi112 ceph-mon[30003]: Reconfiguring mon.c (monmap changed)... 2024-05-08T05:44:20.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:20 smithi112 ceph-mon[30003]: Reconfiguring daemon mon.c on smithi162 2024-05-08T05:44:20.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:20 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:20.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:20 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:20.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:20 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:20.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:20 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:20.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:20 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:20.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:20 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:20.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:20 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:44:20.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:20 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:44:20.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:20 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:44:20.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:20 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:20.592 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T05:44:20.592 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:44:20.592 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-08T05:44:20.592 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:44:20.592 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:44:20.593 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-05-08 05:43:55.612828359 +0000 2024-05-08T05:44:20.593 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-05-08 05:35:42.265082204 +0000 2024-05-08T05:44:20.593 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-05-08 05:35:42.265082204 +0000 2024-05-08T05:44:20.593 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-05-08 05:35:42.265082204 +0000 2024-05-08T05:44:20.593 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T05:44:20.655 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-05-08T05:44:20.655 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-05-08T05:44:20.655 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000176555 s, 2.9 MB/s 2024-05-08T05:44:20.656 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T05:44:20.712 INFO:tasks.cephadm:Deploying osd.0 on smithi112 with /dev/vg_nvme/lv_4... 2024-05-08T05:44:20.712 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T05:44:21.146 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:44:21.406 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:21 smithi112 ceph-mon[30003]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:21.406 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:21 smithi112 ceph-mon[30003]: Deploying daemon mgr.z on smithi162 2024-05-08T05:44:21.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:21 smithi150 ceph-mon[34315]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:21.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:21 smithi150 ceph-mon[34315]: Deploying daemon mgr.z on smithi162 2024-05-08T05:44:21.456 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:21 smithi162 ceph-mon[34800]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:21.456 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:21 smithi162 ceph-mon[34800]: Deploying daemon mgr.z on smithi162 2024-05-08T05:44:22.410 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:22 smithi162 systemd[1]: Started Ceph mgr.z for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:44:22.699 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:22 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:22.409+0000 7f2e87fe8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:44:22.699 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:22 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:22.597+0000 7f2e87fe8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:44:22.699 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:22 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:22.699+0000 7f2e87fe8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:44:23.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:23 smithi112 ceph-mon[30003]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:23.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:23 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:23.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:23 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:23.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:23 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:23.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:23 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:44:23.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:23 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:44:23.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:23 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:44:23.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:23 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:23.317 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:23 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:23.010+0000 7f2e87fe8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:44:23.317 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:23 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:23.095+0000 7f2e87fe8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:44:23.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:23 smithi162 ceph-mon[34800]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:23.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:23 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:23.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:23 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:23.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:23 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:23.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:23 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:44:23.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:23 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:44:23.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:23 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:44:23.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:23 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:23.370 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:23 smithi150 ceph-mon[34315]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:23.370 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:23 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:23.370 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:23 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:23.370 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:23 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:23.370 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:23 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:44:23.370 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:23 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:44:23.370 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:23 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:44:23.370 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:23 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:23.975 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T05:44:23.976 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T05:44:23.976 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-05-08T05:44:23.976 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-05-08T05:44:23.976 INFO:teuthology.orchestra.run.smithi112.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0367934 s, 285 MB/s 2024-05-08T05:44:23.976 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-05-08T05:44:24.238 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:24 smithi150 ceph-mon[34315]: Deploying daemon mgr.y on smithi150 2024-05-08T05:44:24.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:24 smithi112 ceph-mon[30003]: Deploying daemon mgr.y on smithi150 2024-05-08T05:44:24.470 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:24 smithi162 ceph-mon[34800]: Deploying daemon mgr.y on smithi150 2024-05-08T05:44:24.471 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:24 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:24.299+0000 7f2e87fe8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:44:24.492 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:24 smithi150 systemd[1]: Starting Ceph mgr.y for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:44:24.754 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:24 smithi150 podman[35689]: 2024-05-08 05:44:24.535304411 +0000 UTC m=+0.017922388 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:44:24.755 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:24 smithi150 podman[35689]: 2024-05-08 05:44:24.642513492 +0000 UTC m=+0.125131458 container create fb487a1da7df7572aa9d385258fef8ff94ea9b2fd94a783c9f659ffeccfbeeae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-08T05:44:24.772 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_4 2024-05-08T05:44:24.816 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:24 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:24.470+0000 7f2e87fe8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:44:24.816 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:24 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:24.642+0000 7f2e87fe8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:44:25.166 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:24 smithi150 podman[35689]: 2024-05-08 05:44:24.886549962 +0000 UTC m=+0.369167926 container init fb487a1da7df7572aa9d385258fef8ff94ea9b2fd94a783c9f659ffeccfbeeae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-08T05:44:25.166 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:24 smithi150 podman[35689]: 2024-05-08 05:44:24.889123272 +0000 UTC m=+0.371741237 container start fb487a1da7df7572aa9d385258fef8ff94ea9b2fd94a783c9f659ffeccfbeeae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-08T05:44:25.166 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:24 smithi150 ceph-mgr[35709]: -- 172.21.15.150:0/1727692665 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e61bdb45a0 con 0x55e61bd89000 2024-05-08T05:44:25.167 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:24 smithi150 bash[35689]: fb487a1da7df7572aa9d385258fef8ff94ea9b2fd94a783c9f659ffeccfbeeae 2024-05-08T05:44:25.167 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:24 smithi150 systemd[1]: Started Ceph mgr.y for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:44:25.167 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:25 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:25.045+0000 7fba4faeb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:44:25.211 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:44:25.225 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:25 smithi162 ceph-mon[34800]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:25.225 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:25 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:25.226 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:25 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:25.226 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:25 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:25.226 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:25 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:25.226 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:24 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:24.928+0000 7f2e87fe8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:44:25.226 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:25 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:25.019+0000 7f2e87fe8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:44:25.226 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:25 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:25.225+0000 7f2e87fe8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:44:25.433 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:25 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:25.237+0000 7fba4faeb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:44:25.433 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:25 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:25.340+0000 7fba4faeb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:44:25.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:25 smithi150 ceph-mon[34315]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:25.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:25 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:25.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:25 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:25.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:25 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:25.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:25 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:25.533 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:25 smithi112 ceph-mon[30003]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:25.533 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:25 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:25.533 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:25 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:25.533 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:25 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:25.533 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:25 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:25.566 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:25 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:25.423+0000 7f2e87fe8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:44:25.933 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:25 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:25.667+0000 7fba4faeb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:44:25.933 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:25 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:25.756+0000 7fba4faeb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:44:26.338 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:26.338 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:26 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:26.071+0000 7f2e87fe8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:44:26.339 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:26 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:26.298+0000 7f2e87fe8200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:44:26.430 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:26.480 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:27.066 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:26 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:26.743+0000 7f2e87fe8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:44:27.067 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:26 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:26.839+0000 7f2e87fe8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:44:27.356 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:27 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:27.015+0000 7fba4faeb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:44:27.356 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:27 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:27.185+0000 7fba4faeb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:44:27.356 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:27 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:27.356+0000 7fba4faeb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:44:27.357 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:27 smithi150 ceph-mon[34315]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:27.357 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:27 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:27.357 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:27 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:27.357 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:27 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:27.357 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:27 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:27.357 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:27 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:27.357 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:27 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:27.357 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:27 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:27.357 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:27 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:44:27.357 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:27 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:44:27.357 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:27 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:27.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.435+0000 7f5eda600700 1 -- 172.21.15.112:0/420084498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed4106920 msgr2=0x7f5ed4108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:27.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.435+0000 7f5eda600700 1 --2- 172.21.15.112:0/420084498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed4106920 0x7f5ed4108d10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00098f0 tx=0x7f5ec0009c00 comp rx=0 tx=0).stop 2024-05-08T05:44:27.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.435+0000 7f5eda600700 1 -- 172.21.15.112:0/420084498 shutdown_connections 2024-05-08T05:44:27.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.435+0000 7f5eda600700 1 --2- 172.21.15.112:0/420084498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed4109250 0x7f5ed410b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:27.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.435+0000 7f5eda600700 1 --2- 172.21.15.112:0/420084498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4103ff0 0x7f5ed41063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:27.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.435+0000 7f5eda600700 1 --2- 172.21.15.112:0/420084498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed4106920 0x7f5ed4108d10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:27.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.435+0000 7f5eda600700 1 -- 172.21.15.112:0/420084498 >> 172.21.15.112:0/420084498 conn(0x7f5ed40fd9a0 msgr2=0x7f5ed40ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:27.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.435+0000 7f5eda600700 1 -- 172.21.15.112:0/420084498 shutdown_connections 2024-05-08T05:44:27.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.435+0000 7f5eda600700 1 -- 172.21.15.112:0/420084498 wait complete. 2024-05-08T05:44:27.437 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:27 smithi112 ceph-mon[30003]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:27.437 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:27 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:27.437 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:27 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:27.438 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:27 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:27.438 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:27 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:27.438 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:27 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:27.438 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:27 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:27.438 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:27 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:27.438 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:27 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:44:27.438 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:27 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:44:27.438 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:27 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:27.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.436+0000 7f5eda600700 1 Processor -- start 2024-05-08T05:44:27.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.437+0000 7f5eda600700 1 -- start start 2024-05-08T05:44:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.437+0000 7f5eda600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4103ff0 0x7f5ed419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.437+0000 7f5eda600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed4106920 0x7f5ed419bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.437+0000 7f5eda600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed4109250 0x7f5ed41a2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.437+0000 7f5eda600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed410efd0 con 0x7f5ed4103ff0 2024-05-08T05:44:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.437+0000 7f5eda600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed410ee50 con 0x7f5ed4106920 2024-05-08T05:44:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.437+0000 7f5eda600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed410f150 con 0x7f5ed4109250 2024-05-08T05:44:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.437+0000 7f5ed3e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed4109250 0x7f5ed41a2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.437+0000 7f5ed2a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed4106920 0x7f5ed419bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.437+0000 7f5ed3e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed4109250 0x7f5ed41a2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:42706/0 (socket says 172.21.15.112:42706) 2024-05-08T05:44:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.437+0000 7f5ed3400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4103ff0 0x7f5ed419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.437+0000 7f5ed3e00700 1 -- 172.21.15.112:0/4137764748 learned_addr learned my addr 172.21.15.112:0/4137764748 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:44:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.438+0000 7f5ed2a00700 1 -- 172.21.15.112:0/4137764748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed4109250 msgr2=0x7f5ed41a2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.438+0000 7f5ed2a00700 1 --2- 172.21.15.112:0/4137764748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed4109250 0x7f5ed41a2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:27.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.438+0000 7f5ed2a00700 1 -- 172.21.15.112:0/4137764748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4103ff0 msgr2=0x7f5ed419b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:27.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.438+0000 7f5ed2a00700 1 --2- 172.21.15.112:0/4137764748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4103ff0 0x7f5ed419b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:27.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.438+0000 7f5ed2a00700 1 -- 172.21.15.112:0/4137764748 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed41a35e0 con 0x7f5ed4106920 2024-05-08T05:44:27.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.438+0000 7f5ed3400700 1 --2- 172.21.15.112:0/4137764748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4103ff0 0x7f5ed419b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:27.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.438+0000 7f5ed2a00700 1 --2- 172.21.15.112:0/4137764748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed4106920 0x7f5ed419bec0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000fc20 tx=0x7f5ec0005320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:27.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.438+0000 7f5ecbe00700 1 -- 172.21.15.112:0/4137764748 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000f950 con 0x7f5ed4106920 2024-05-08T05:44:27.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.438+0000 7f5ecbe00700 1 -- 172.21.15.112:0/4137764748 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec0005540 con 0x7f5ed4106920 2024-05-08T05:44:27.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.439+0000 7f5eda600700 1 -- 172.21.15.112:0/4137764748 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed41a3870 con 0x7f5ed4106920 2024-05-08T05:44:27.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.439+0000 7f5ecbe00700 1 -- 172.21.15.112:0/4137764748 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0003a90 con 0x7f5ed4106920 2024-05-08T05:44:27.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.439+0000 7f5eda600700 1 -- 172.21.15.112:0/4137764748 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed41a3cc0 con 0x7f5ed4106920 2024-05-08T05:44:27.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.440+0000 7f5ecbe00700 1 -- 172.21.15.112:0/4137764748 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f5ec0029060 con 0x7f5ed4106920 2024-05-08T05:44:27.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.440+0000 7f5eda600700 1 -- 172.21.15.112:0/4137764748 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed406a190 con 0x7f5ed4106920 2024-05-08T05:44:27.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.441+0000 7f5ecbe00700 1 --2- 172.21.15.112:0/4137764748 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5eb803e750 0x7f5eb8040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:27.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.441+0000 7f5ecbe00700 1 -- 172.21.15.112:0/4137764748 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5ec0053110 con 0x7f5ed4106920 2024-05-08T05:44:27.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.441+0000 7f5ed3400700 1 --2- 172.21.15.112:0/4137764748 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5eb803e750 0x7f5eb8040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:27.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.442+0000 7f5ed3400700 1 --2- 172.21.15.112:0/4137764748 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5eb803e750 0x7f5eb8040c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009cb0 tx=0x7f5ec4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:27.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.446+0000 7f5ecbe00700 1 -- 172.21.15.112:0/4137764748 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ec0013030 con 0x7f5ed4106920 2024-05-08T05:44:27.565 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:27 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:27.213+0000 7f2e87fe8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:44:27.565 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:27 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:27.565+0000 7f2e87fe8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:44:27.565 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:27 smithi162 ceph-mon[34800]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:27 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:27 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:27 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:27 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:27 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:27 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:27 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:27 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:44:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:27 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:44:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:27 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:27.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:27.693+0000 7f5eda600700 1 -- 172.21.15.112:0/4137764748 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5ed40638e0 con 0x7f5eb803e750 2024-05-08T05:44:27.816 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:27 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:27.657+0000 7f2e87fe8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:44:27.933 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:27 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:27.644+0000 7fba4faeb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:44:27.933 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:27 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:27.736+0000 7fba4faeb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:44:28.188 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:27 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:27.895+0000 7f2e87fe8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:44:28.189 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:28 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:28.028+0000 7f2e87fe8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:44:28.189 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:28 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:28.123+0000 7f2e87fe8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:44:28.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:28 smithi112 ceph-mon[30003]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T05:44:28.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:28 smithi112 ceph-mon[30003]: Reconfiguring daemon mgr.x on smithi112 2024-05-08T05:44:28.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:28 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:44:28.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:28 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:44:28.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:28 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:28.433 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:27 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:27.941+0000 7fba4faeb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:44:28.433 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:28 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:28.130+0000 7fba4faeb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:44:28.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:28 smithi150 ceph-mon[34315]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T05:44:28.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:28 smithi150 ceph-mon[34315]: Reconfiguring daemon mgr.x on smithi112 2024-05-08T05:44:28.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:28 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:44:28.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:28 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:44:28.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:28 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:28.453 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:28 smithi162 ceph-mon[34800]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T05:44:28.453 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:28 smithi162 ceph-mon[34800]: Reconfiguring daemon mgr.x on smithi112 2024-05-08T05:44:28.453 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:28 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:44:28.453 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:28 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:44:28.453 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:28 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:28.816 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:28 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:28.452+0000 7f2e87fe8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:44:28.817 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:28 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:28.544+0000 7f2e87fe8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:44:29.183 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:28 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:28.731+0000 7fba4faeb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:44:29.183 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:28 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:28.951+0000 7fba4faeb200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:44:29.463 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:29 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:29.369+0000 7fba4faeb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:44:29.463 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:29 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:29.462+0000 7fba4faeb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:44:29.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:29 smithi150 ceph-mon[34315]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:29.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:29 smithi150 ceph-mon[34315]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:44:29.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:29.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:29.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:29.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:29.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:29.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:29.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:29.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:29 smithi112 ceph-mon[30003]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:29.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:29 smithi112 ceph-mon[30003]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:44:29.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:29.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:29.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:29.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:29.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:29.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:29.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:29.566 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:29 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:29.142+0000 7f2e87fe8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:44:29.567 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 05:44:29 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z[36683]: 2024-05-08T05:44:29.358+0000 7f2e87fe8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:44:29.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:29 smithi162 ceph-mon[34800]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:29.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:29 smithi162 ceph-mon[34800]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:44:29.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:29.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:29.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:29.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:29.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:29.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:29.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:30.180 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:29 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:29.829+0000 7fba4faeb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:44:30.180 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:30 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:30.179+0000 7fba4faeb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:44:30.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:30.207+0000 7f5ecbe00700 1 -- 172.21.15.112:0/4137764748 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 14) ==== 99938+0+0 (secure 0 0 0) 0x7f5ec000e5c0 con 0x7f5ed4106920 2024-05-08T05:44:30.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:30 smithi112 ceph-mon[30003]: from='mgr.? 172.21.15.162:0/4236606207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:44:30.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:30 smithi112 ceph-mon[30003]: Standby manager daemon z started 2024-05-08T05:44:30.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:30 smithi112 ceph-mon[30003]: from='mgr.? 172.21.15.162:0/4236606207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:44:30.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:30 smithi112 ceph-mon[30003]: from='mgr.? 172.21.15.162:0/4236606207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:44:30.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:30 smithi112 ceph-mon[30003]: from='mgr.? 172.21.15.162:0/4236606207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:44:30.433 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:30 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:30.271+0000 7fba4faeb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:44:30.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:30 smithi150 ceph-mon[34315]: from='mgr.? 172.21.15.162:0/4236606207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:44:30.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:30 smithi150 ceph-mon[34315]: Standby manager daemon z started 2024-05-08T05:44:30.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:30 smithi150 ceph-mon[34315]: from='mgr.? 172.21.15.162:0/4236606207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:44:30.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:30 smithi150 ceph-mon[34315]: from='mgr.? 172.21.15.162:0/4236606207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:44:30.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:30 smithi150 ceph-mon[34315]: from='mgr.? 172.21.15.162:0/4236606207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:44:30.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:30 smithi162 ceph-mon[34800]: from='mgr.? 172.21.15.162:0/4236606207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:44:30.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:30 smithi162 ceph-mon[34800]: Standby manager daemon z started 2024-05-08T05:44:30.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:30 smithi162 ceph-mon[34800]: from='mgr.? 172.21.15.162:0/4236606207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:44:30.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:30 smithi162 ceph-mon[34800]: from='mgr.? 172.21.15.162:0/4236606207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:44:30.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:30 smithi162 ceph-mon[34800]: from='mgr.? 172.21.15.162:0/4236606207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:44:30.933 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:30 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:30.510+0000 7fba4faeb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:44:30.933 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:30 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:30.641+0000 7fba4faeb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:44:30.933 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:30 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:30.735+0000 7fba4faeb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:44:31.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:31 smithi112 ceph-mon[30003]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:31.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:31 smithi112 ceph-mon[30003]: mgrmap e14: x(active, since 86s), standbys: z 2024-05-08T05:44:31.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:44:31.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:31 smithi150 ceph-mon[34315]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:31.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:31 smithi150 ceph-mon[34315]: mgrmap e14: x(active, since 86s), standbys: z 2024-05-08T05:44:31.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:44:31.433 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:31 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:31.061+0000 7fba4faeb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:44:31.433 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:31 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:31.152+0000 7fba4faeb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:44:31.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:31 smithi162 ceph-mon[34800]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:31.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:31 smithi162 ceph-mon[34800]: mgrmap e14: x(active, since 86s), standbys: z 2024-05-08T05:44:31.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:44:32.183 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:31 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:31.745+0000 7fba4faeb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:44:32.183 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 05:44:31 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y[35705]: 2024-05-08T05:44:31.959+0000 7fba4faeb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:44:32.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:32.229+0000 7f5ecbe00700 1 -- 172.21.15.112:0/4137764748 <== mon.2 v2:172.21.15.150:3300/0 8 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f5ec0020320 con 0x7f5ed4106920 2024-05-08T05:44:32.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:32 smithi112 ceph-mon[30003]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:32.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:32 smithi112 ceph-mon[30003]: Standby manager daemon y started 2024-05-08T05:44:32.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:32 smithi112 ceph-mon[30003]: from='mgr.? 172.21.15.150:0/4275366587' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:44:32.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:32 smithi112 ceph-mon[30003]: from='mgr.? 172.21.15.150:0/4275366587' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:44:32.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:32 smithi112 ceph-mon[30003]: from='mgr.? 172.21.15.150:0/4275366587' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:44:32.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:32 smithi112 ceph-mon[30003]: from='mgr.? 172.21.15.150:0/4275366587' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:44:32.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:32 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3330629169' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b378542-1bbe-4614-8805-8a6b875ada55"}]: dispatch 2024-05-08T05:44:32.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:32 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b378542-1bbe-4614-8805-8a6b875ada55"}]: dispatch 2024-05-08T05:44:32.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:32 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b378542-1bbe-4614-8805-8a6b875ada55"}]': finished 2024-05-08T05:44:32.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:32 smithi112 ceph-mon[30003]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T05:44:32.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:32 smithi162 ceph-mon[34800]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:32.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:32 smithi162 ceph-mon[34800]: Standby manager daemon y started 2024-05-08T05:44:32.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:32 smithi162 ceph-mon[34800]: from='mgr.? 172.21.15.150:0/4275366587' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:44:32.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:32 smithi162 ceph-mon[34800]: from='mgr.? 172.21.15.150:0/4275366587' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:44:32.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:32 smithi162 ceph-mon[34800]: from='mgr.? 172.21.15.150:0/4275366587' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:44:32.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:32 smithi162 ceph-mon[34800]: from='mgr.? 172.21.15.150:0/4275366587' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:44:32.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:32 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3330629169' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b378542-1bbe-4614-8805-8a6b875ada55"}]: dispatch 2024-05-08T05:44:32.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:32 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b378542-1bbe-4614-8805-8a6b875ada55"}]: dispatch 2024-05-08T05:44:32.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:32 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b378542-1bbe-4614-8805-8a6b875ada55"}]': finished 2024-05-08T05:44:32.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:32 smithi162 ceph-mon[34800]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T05:44:32.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:32 smithi150 ceph-mon[34315]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:32.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:32 smithi150 ceph-mon[34315]: Standby manager daemon y started 2024-05-08T05:44:32.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:32 smithi150 ceph-mon[34315]: from='mgr.? 172.21.15.150:0/4275366587' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:44:32.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:32 smithi150 ceph-mon[34315]: from='mgr.? 172.21.15.150:0/4275366587' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:44:32.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:32 smithi150 ceph-mon[34315]: from='mgr.? 172.21.15.150:0/4275366587' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:44:32.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:32 smithi150 ceph-mon[34315]: from='mgr.? 172.21.15.150:0/4275366587' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:44:32.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:32 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3330629169' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b378542-1bbe-4614-8805-8a6b875ada55"}]: dispatch 2024-05-08T05:44:32.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:32 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b378542-1bbe-4614-8805-8a6b875ada55"}]: dispatch 2024-05-08T05:44:32.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:32 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b378542-1bbe-4614-8805-8a6b875ada55"}]': finished 2024-05-08T05:44:32.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:32 smithi150 ceph-mon[34315]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T05:44:33.418 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:33 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:33.418 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:33 smithi112 ceph-mon[30003]: mgrmap e15: x(active, since 88s), standbys: z, y 2024-05-08T05:44:33.418 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:33 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:44:33.418 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:33 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2774692862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:44:33.491 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:33 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:33.491 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:33 smithi162 ceph-mon[34800]: mgrmap e15: x(active, since 88s), standbys: z, y 2024-05-08T05:44:33.492 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:33 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:44:33.492 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:33 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/2774692862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:44:33.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:33 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:33.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:33 smithi150 ceph-mon[34315]: mgrmap e15: x(active, since 88s), standbys: z, y 2024-05-08T05:44:33.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:33 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:44:33.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:33 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/2774692862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:44:34.562 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:34 smithi150 ceph-mon[34315]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:34.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:34 smithi112 ceph-mon[30003]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:34.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:34 smithi162 ceph-mon[34800]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:36.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:36 smithi150 ceph-mon[34315]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:36.696 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:36 smithi112 ceph-mon[30003]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:36.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:36 smithi162 ceph-mon[34800]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:38.665 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:38 smithi112 ceph-mon[30003]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:38.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:38 smithi150 ceph-mon[34315]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:38.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:38 smithi162 ceph-mon[34800]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:40.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:40 smithi112 ceph-mon[30003]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:40.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:40 smithi150 ceph-mon[34315]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:40.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:40 smithi162 ceph-mon[34800]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:42.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:42 smithi150 ceph-mon[34315]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:42.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:42 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T05:44:42.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:42 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:42.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:42 smithi150 ceph-mon[34315]: Deploying daemon osd.0 on smithi112 2024-05-08T05:44:42.709 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:42 smithi112 ceph-mon[30003]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:42.709 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T05:44:42.710 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:42.710 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:42 smithi112 ceph-mon[30003]: Deploying daemon osd.0 on smithi112 2024-05-08T05:44:42.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:42 smithi162 ceph-mon[34800]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:42.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:42 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T05:44:42.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:42 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:42.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:42 smithi162 ceph-mon[34800]: Deploying daemon osd.0 on smithi112 2024-05-08T05:44:44.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:44 smithi150 ceph-mon[34315]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:44.717 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:44 smithi112 ceph-mon[30003]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:44.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:44 smithi162 ceph-mon[34800]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:46.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:46 smithi112 ceph-mon[30003]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:46.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:46 smithi150 ceph-mon[34315]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:46.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:46 smithi162 ceph-mon[34800]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:48.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:48 smithi150 ceph-mon[34315]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:48.752 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:48 smithi112 ceph-mon[30003]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:48.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:48 smithi162 ceph-mon[34800]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:50.767 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:50 smithi112 ceph-mon[30003]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:50.767 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:50.768 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:50.768 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:50.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:50 smithi150 ceph-mon[34315]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:50.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:50.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:50.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:51.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:50 smithi162 ceph-mon[34800]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:51.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:51.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:51.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:52.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:52 smithi112 ceph-mon[30003]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:52.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:52 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:52.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:52 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:52.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:52 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:52.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:52 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:52.818 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:52 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:52.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:52 smithi150 ceph-mon[34315]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:52.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:52 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:52.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:52 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:52.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:52 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:52.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:52 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:52.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:52 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:53.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:52 smithi162 ceph-mon[34800]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:53.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:52 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:53.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:52 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:53.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:52 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:53.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:52 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:53.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:52 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:53.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:53 smithi112 ceph-mon[30003]: from='osd.0 [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:44:53.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:53 smithi112 ceph-mon[30003]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:44:53.834 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:53 smithi150 ceph-mon[34315]: from='osd.0 [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:44:53.834 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:53 smithi150 ceph-mon[34315]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:44:54.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:53 smithi162 ceph-mon[34800]: from='osd.0 [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:44:54.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:53 smithi162 ceph-mon[34800]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:44:54.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:54.392+0000 7f5ecbe00700 1 -- 172.21.15.112:0/4137764748 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5ed40638e0 con 0x7f5eb803e750 2024-05-08T05:44:54.394 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 0 on host 'smithi112' 2024-05-08T05:44:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:54.397+0000 7f5eda600700 1 -- 172.21.15.112:0/4137764748 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5eb803e750 msgr2=0x7f5eb8040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:54.397+0000 7f5eda600700 1 --2- 172.21.15.112:0/4137764748 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5eb803e750 0x7f5eb8040c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009cb0 tx=0x7f5ec4009260 comp rx=0 tx=0).stop 2024-05-08T05:44:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:54.397+0000 7f5eda600700 1 -- 172.21.15.112:0/4137764748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed4106920 msgr2=0x7f5ed419bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:54.397+0000 7f5eda600700 1 --2- 172.21.15.112:0/4137764748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed4106920 0x7f5ed419bec0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000fc20 tx=0x7f5ec0005320 comp rx=0 tx=0).stop 2024-05-08T05:44:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:54.397+0000 7f5eda600700 1 -- 172.21.15.112:0/4137764748 shutdown_connections 2024-05-08T05:44:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:54.397+0000 7f5eda600700 1 --2- 172.21.15.112:0/4137764748 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5eb803e750 0x7f5eb8040c10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:54.397+0000 7f5eda600700 1 --2- 172.21.15.112:0/4137764748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ed4109250 0x7f5ed41a2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:54.397+0000 7f5eda600700 1 --2- 172.21.15.112:0/4137764748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4103ff0 0x7f5ed419b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:54.397+0000 7f5eda600700 1 --2- 172.21.15.112:0/4137764748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed4106920 0x7f5ed419bec0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:54.397+0000 7f5eda600700 1 -- 172.21.15.112:0/4137764748 >> 172.21.15.112:0/4137764748 conn(0x7f5ed40fd9a0 msgr2=0x7f5ed41037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:54.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:54.397+0000 7f5eda600700 1 -- 172.21.15.112:0/4137764748 shutdown_connections 2024-05-08T05:44:54.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:44:54.397+0000 7f5eda600700 1 -- 172.21.15.112:0/4137764748 wait complete. 2024-05-08T05:44:54.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:54 smithi150 ceph-mon[34315]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:54.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:54 smithi150 ceph-mon[34315]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T05:44:54.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:54 smithi150 ceph-mon[34315]: from='osd.0 [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:44:54.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:54 smithi150 ceph-mon[34315]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T05:44:54.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:54 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:54.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:54 smithi150 ceph-mon[34315]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:44:54.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:54 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:54.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:54 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:54.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:54 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:55.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:54 smithi112 ceph-mon[30003]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:55.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:54 smithi112 ceph-mon[30003]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T05:44:55.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:54 smithi112 ceph-mon[30003]: from='osd.0 [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:44:55.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:54 smithi112 ceph-mon[30003]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T05:44:55.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:54 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:55.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:54 smithi112 ceph-mon[30003]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:44:55.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:54 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:55.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:54 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:55.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:54 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:55.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:54 smithi162 ceph-mon[34800]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:55.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:54 smithi162 ceph-mon[34800]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T05:44:55.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:54 smithi162 ceph-mon[34800]: from='osd.0 [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:44:55.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:54 smithi162 ceph-mon[34800]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T05:44:55.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:54 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:55.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:54 smithi162 ceph-mon[34800]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:44:55.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:54 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:55.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:54 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:44:55.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:54 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:55.177 DEBUG:teuthology.orchestra.run.smithi112:osd.0> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.0.service 2024-05-08T05:44:55.182 INFO:tasks.cephadm:Deploying osd.1 on smithi150 with /dev/vg_nvme/lv_4... 2024-05-08T05:44:55.182 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T05:44:55.414 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:44:55.741 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:55 smithi112 ceph-mon[30003]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-08T05:44:55.741 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:55 smithi112 ceph-mon[30003]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T05:44:55.741 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:55.741 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:55.790 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:55 smithi150 ceph-mon[34315]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-08T05:44:55.790 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:55 smithi150 ceph-mon[34315]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T05:44:55.790 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:55.790 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:56.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:55 smithi162 ceph-mon[34800]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-08T05:44:56.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:55 smithi162 ceph-mon[34800]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T05:44:56.070 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:56.071 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:56.663 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:56 smithi150 ceph-mon[34315]: purged_snaps scrub starts 2024-05-08T05:44:56.774 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:56 smithi150 ceph-mon[34315]: purged_snaps scrub ok 2024-05-08T05:44:56.774 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:56 smithi150 ceph-mon[34315]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:56.774 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:56 smithi150 ceph-mon[34315]: osd.0 [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820] boot 2024-05-08T05:44:56.774 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:56 smithi150 ceph-mon[34315]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T05:44:56.774 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:56 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:56.775 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:56 smithi112 ceph-mon[30003]: purged_snaps scrub starts 2024-05-08T05:44:56.775 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:56 smithi112 ceph-mon[30003]: purged_snaps scrub ok 2024-05-08T05:44:56.775 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:56 smithi112 ceph-mon[30003]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:56.775 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:56 smithi112 ceph-mon[30003]: osd.0 [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820] boot 2024-05-08T05:44:56.775 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:56 smithi112 ceph-mon[30003]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T05:44:56.776 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:56 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:57.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:56 smithi162 ceph-mon[34800]: purged_snaps scrub starts 2024-05-08T05:44:57.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:56 smithi162 ceph-mon[34800]: purged_snaps scrub ok 2024-05-08T05:44:57.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:56 smithi162 ceph-mon[34800]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:57.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:56 smithi162 ceph-mon[34800]: osd.0 [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820] boot 2024-05-08T05:44:57.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:56 smithi162 ceph-mon[34800]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T05:44:57.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:57.213 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T05:44:57.213 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T05:44:57.213 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-08T05:44:57.213 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-08T05:44:57.214 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112398 s, 933 MB/s 2024-05-08T05:44:57.214 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-08T05:44:57.793 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_4 2024-05-08T05:44:57.874 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:57 smithi162 ceph-mon[34800]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T05:44:57.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:57 smithi150 ceph-mon[34315]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T05:44:58.026 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:44:58.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:57 smithi112 ceph-mon[30003]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T05:44:58.821 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:44:58 smithi150 ceph-mon[34315]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:59.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:44:58 smithi112 ceph-mon[30003]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:59.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:44:58 smithi162 ceph-mon[34800]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:59.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.509+0000 7fe790b48700 1 -- 172.21.15.150:0/2695541470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe78c1050a0 msgr2=0x7fe78c10b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:59.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.509+0000 7fe790b48700 1 --2- 172.21.15.150:0/2695541470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe78c1050a0 0x7fe78c10b950 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe78400b0d0 tx=0x7fe78400b3e0 comp rx=0 tx=0).stop 2024-05-08T05:44:59.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.509+0000 7fe790b48700 1 -- 172.21.15.150:0/2695541470 shutdown_connections 2024-05-08T05:44:59.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.509+0000 7fe790b48700 1 --2- 172.21.15.150:0/2695541470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe78c1050a0 0x7fe78c10b950 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:59.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.509+0000 7fe790b48700 1 --2- 172.21.15.150:0/2695541470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe78c103500 0x7fe78c103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:59.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.509+0000 7fe790b48700 1 --2- 172.21.15.150:0/2695541470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe78c104700 0x7fe78c104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:59.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.509+0000 7fe790b48700 1 -- 172.21.15.150:0/2695541470 >> 172.21.15.150:0/2695541470 conn(0x7fe78c0feaa0 msgr2=0x7fe78c100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:59.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.510+0000 7fe790b48700 1 -- 172.21.15.150:0/2695541470 shutdown_connections 2024-05-08T05:44:59.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.510+0000 7fe790b48700 1 -- 172.21.15.150:0/2695541470 wait complete. 2024-05-08T05:44:59.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.510+0000 7fe790b48700 1 Processor -- start 2024-05-08T05:44:59.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.510+0000 7fe790b48700 1 -- start start 2024-05-08T05:44:59.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.510+0000 7fe790b48700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe78c103500 0x7fe78c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:59.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.510+0000 7fe790b48700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe78c104700 0x7fe78c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:59.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.510+0000 7fe790b48700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe78c1050a0 0x7fe78c1a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:59.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.510+0000 7fe790b48700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c10f0e0 con 0x7fe78c103500 2024-05-08T05:44:59.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.510+0000 7fe790b48700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c10ef60 con 0x7fe78c104700 2024-05-08T05:44:59.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.510+0000 7fe790b48700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c10f260 con 0x7fe78c1050a0 2024-05-08T05:44:59.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe789d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe78c104700 0x7fe78c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:59.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe789d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe78c104700 0x7fe78c199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58444/0 (socket says 172.21.15.150:58444) 2024-05-08T05:44:59.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe789d9b700 1 -- 172.21.15.150:0/1278490775 learned_addr learned my addr 172.21.15.150:0/1278490775 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-08T05:44:59.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe78ad9d700 1 --2- 172.21.15.150:0/1278490775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe78c1050a0 0x7fe78c1a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:59.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe789d9b700 1 -- 172.21.15.150:0/1278490775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe78c1050a0 msgr2=0x7fe78c1a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:59.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe789d9b700 1 --2- 172.21.15.150:0/1278490775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe78c1050a0 0x7fe78c1a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:59.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe789d9b700 1 -- 172.21.15.150:0/1278490775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe78c103500 msgr2=0x7fe78c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:59.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe789d9b700 1 --2- 172.21.15.150:0/1278490775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe78c103500 0x7fe78c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:59.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe789d9b700 1 -- 172.21.15.150:0/1278490775 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78c1a1410 con 0x7fe78c104700 2024-05-08T05:44:59.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe78ad9d700 1 --2- 172.21.15.150:0/1278490775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe78c1050a0 0x7fe78c1a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:59.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe789d9b700 1 --2- 172.21.15.150:0/1278490775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe78c104700 0x7fe78c199cf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe78000b530 tx=0x7fe78000b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:59.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe77b7fe700 1 -- 172.21.15.150:0/1278490775 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe780015020 con 0x7fe78c104700 2024-05-08T05:44:59.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe790b48700 1 -- 172.21.15.150:0/1278490775 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78c1a1700 con 0x7fe78c104700 2024-05-08T05:44:59.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe77b7fe700 1 -- 172.21.15.150:0/1278490775 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe780011ce0 con 0x7fe78c104700 2024-05-08T05:44:59.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe77b7fe700 1 -- 172.21.15.150:0/1278490775 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe780010d50 con 0x7fe78c104700 2024-05-08T05:44:59.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.511+0000 7fe790b48700 1 -- 172.21.15.150:0/1278490775 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe78c1a1cc0 con 0x7fe78c104700 2024-05-08T05:44:59.515 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.512+0000 7fe790b48700 1 -- 172.21.15.150:0/1278490775 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe78c06a190 con 0x7fe78c104700 2024-05-08T05:44:59.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.514+0000 7fe77b7fe700 1 -- 172.21.15.150:0/1278490775 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7fe780018020 con 0x7fe78c104700 2024-05-08T05:44:59.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.514+0000 7fe77b7fe700 1 --2- 172.21.15.150:0/1278490775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe7740b27f0 0x7fe7740b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:59.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.514+0000 7fe77b7fe700 1 -- 172.21.15.150:0/1278490775 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fe7800e04f0 con 0x7fe78c104700 2024-05-08T05:44:59.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.515+0000 7fe78a59c700 1 --2- 172.21.15.150:0/1278490775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe7740b27f0 0x7fe7740b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:59.516 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.515+0000 7fe78a59c700 1 --2- 172.21.15.150:0/1278490775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe7740b27f0 0x7fe7740b4cb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe78c104560 tx=0x7fe77c006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:59.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.518+0000 7fe77b7fe700 1 -- 172.21.15.150:0/1278490775 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe78001f020 con 0x7fe78c104700 2024-05-08T05:44:59.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:44:59.728+0000 7fe790b48700 1 -- 172.21.15.150:0/1278490775 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe78c0638e0 con 0x7fe7740b27f0 2024-05-08T05:45:00.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:00 smithi150 ceph-mon[34315]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:00.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:00 smithi150 ceph-mon[34315]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:00.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:00 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:00.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:00 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:00.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:00 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:00.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:00 smithi150 ceph-mon[34315]: Detected new or changed devices on smithi112 2024-05-08T05:45:00.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:00 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:00.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:00 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:00.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:00 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:00.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:00 smithi150 ceph-mon[34315]: Adjusting osd_memory_target on smithi112 to 17089M 2024-05-08T05:45:00.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:00 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:00.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:00 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:00.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:00 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:00.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:00 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:01.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:00 smithi112 ceph-mon[30003]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:01.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:00 smithi112 ceph-mon[30003]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:01.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:00 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:01.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:00 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:01.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:00 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:01.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:00 smithi112 ceph-mon[30003]: Detected new or changed devices on smithi112 2024-05-08T05:45:01.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:00 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:01.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:00 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:01.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:00 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:01.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:00 smithi112 ceph-mon[30003]: Adjusting osd_memory_target on smithi112 to 17089M 2024-05-08T05:45:01.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:00 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:01.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:00 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:01.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:00 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:01.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:00 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:01.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:00 smithi162 ceph-mon[34800]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:01.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:00 smithi162 ceph-mon[34800]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:01.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:00 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:01.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:00 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:01.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:00 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:01.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:00 smithi162 ceph-mon[34800]: Detected new or changed devices on smithi112 2024-05-08T05:45:01.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:00 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:01.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:00 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:01.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:00 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:01.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:00 smithi162 ceph-mon[34800]: Adjusting osd_memory_target on smithi112 to 17089M 2024-05-08T05:45:01.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:00 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:01.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:00 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:01.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:00 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:01.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:00 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:02.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:02 smithi150 ceph-mon[34315]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:02.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:02 smithi150 ceph-mon[34315]: from='client.? 172.21.15.150:0/2728199419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "826ef036-2db7-470e-9956-d9a58b9181e9"}]: dispatch 2024-05-08T05:45:02.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:02 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "826ef036-2db7-470e-9956-d9a58b9181e9"}]: dispatch 2024-05-08T05:45:02.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:02 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "826ef036-2db7-470e-9956-d9a58b9181e9"}]': finished 2024-05-08T05:45:02.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:02 smithi150 ceph-mon[34315]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T05:45:02.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:02 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:03.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:02 smithi112 ceph-mon[30003]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:03.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:02 smithi112 ceph-mon[30003]: from='client.? 172.21.15.150:0/2728199419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "826ef036-2db7-470e-9956-d9a58b9181e9"}]: dispatch 2024-05-08T05:45:03.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:02 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "826ef036-2db7-470e-9956-d9a58b9181e9"}]: dispatch 2024-05-08T05:45:03.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:02 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "826ef036-2db7-470e-9956-d9a58b9181e9"}]': finished 2024-05-08T05:45:03.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:02 smithi112 ceph-mon[30003]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T05:45:03.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:02 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:03.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:02 smithi162 ceph-mon[34800]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:03.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:02 smithi162 ceph-mon[34800]: from='client.? 172.21.15.150:0/2728199419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "826ef036-2db7-470e-9956-d9a58b9181e9"}]: dispatch 2024-05-08T05:45:03.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:02 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "826ef036-2db7-470e-9956-d9a58b9181e9"}]: dispatch 2024-05-08T05:45:03.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:02 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "826ef036-2db7-470e-9956-d9a58b9181e9"}]': finished 2024-05-08T05:45:03.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:02 smithi162 ceph-mon[34800]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T05:45:03.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:02 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:03.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:03 smithi150 ceph-mon[34315]: from='client.? 172.21.15.150:0/603777103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:03.967 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:03 smithi112 ceph-mon[30003]: from='client.? 172.21.15.150:0/603777103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:04.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:03 smithi162 ceph-mon[34800]: from='client.? 172.21.15.150:0/603777103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:04.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:04 smithi150 ceph-mon[34315]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:05.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:04 smithi112 ceph-mon[30003]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:05.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:04 smithi162 ceph-mon[34800]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:06.897 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:06 smithi150 ceph-mon[34315]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:07.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:06 smithi112 ceph-mon[30003]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:07.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:06 smithi162 ceph-mon[34800]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:08.931 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:08 smithi150 ceph-mon[34315]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:09.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:08 smithi112 ceph-mon[30003]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:09.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:08 smithi162 ceph-mon[34800]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:10.729 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:10 smithi150 ceph-mon[34315]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:10.729 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:10 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T05:45:10.729 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:10 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:10.729 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:10 smithi150 ceph-mon[34315]: Deploying daemon osd.1 on smithi150 2024-05-08T05:45:11.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:10 smithi112 ceph-mon[30003]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:11.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T05:45:11.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:11.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:10 smithi112 ceph-mon[30003]: Deploying daemon osd.1 on smithi150 2024-05-08T05:45:11.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:10 smithi162 ceph-mon[34800]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:11.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:10 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T05:45:11.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:10 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:11.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:10 smithi162 ceph-mon[34800]: Deploying daemon osd.1 on smithi150 2024-05-08T05:45:12.930 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:12 smithi150 ceph-mon[34315]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:13.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:12 smithi112 ceph-mon[30003]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:13.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:12 smithi162 ceph-mon[34800]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:14.890 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:14 smithi150 ceph-mon[34315]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:15.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:14 smithi112 ceph-mon[30003]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:15.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:14 smithi162 ceph-mon[34800]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:16.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:16 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:16.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:16 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:16.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:16 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:16.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:16 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:16.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:16 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:16.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:16 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:16.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:16 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:16.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:16 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:16.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:16 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:17.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:17 smithi112 ceph-mon[30003]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:17.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:17 smithi162 ceph-mon[34800]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:17.400 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:17 smithi150 ceph-mon[34315]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:18.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:18 smithi150 ceph-mon[34315]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:18.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:18 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:18.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:18 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:18.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:18 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:18.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:18 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:18.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:18 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:18.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:45:18.765+0000 7fe77b7fe700 1 -- 172.21.15.150:0/1278490775 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe78c0638e0 con 0x7fe7740b27f0 2024-05-08T05:45:18.766 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 1 on host 'smithi150' 2024-05-08T05:45:18.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:45:18.768+0000 7fe790b48700 1 -- 172.21.15.150:0/1278490775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe7740b27f0 msgr2=0x7fe7740b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:18.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:45:18.768+0000 7fe790b48700 1 --2- 172.21.15.150:0/1278490775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe7740b27f0 0x7fe7740b4cb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe78c104560 tx=0x7fe77c006d20 comp rx=0 tx=0).stop 2024-05-08T05:45:18.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:45:18.768+0000 7fe790b48700 1 -- 172.21.15.150:0/1278490775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe78c104700 msgr2=0x7fe78c199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:18.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:45:18.768+0000 7fe790b48700 1 --2- 172.21.15.150:0/1278490775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe78c104700 0x7fe78c199cf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe78000b530 tx=0x7fe78000b8f0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:45:18.768+0000 7fe790b48700 1 -- 172.21.15.150:0/1278490775 shutdown_connections 2024-05-08T05:45:18.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:45:18.768+0000 7fe790b48700 1 --2- 172.21.15.150:0/1278490775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe7740b27f0 0x7fe7740b4cb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:45:18.768+0000 7fe790b48700 1 --2- 172.21.15.150:0/1278490775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe78c1050a0 0x7fe78c1a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:45:18.768+0000 7fe790b48700 1 --2- 172.21.15.150:0/1278490775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe78c103500 0x7fe78c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:45:18.768+0000 7fe790b48700 1 --2- 172.21.15.150:0/1278490775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe78c104700 0x7fe78c199cf0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:45:18.768+0000 7fe790b48700 1 -- 172.21.15.150:0/1278490775 >> 172.21.15.150:0/1278490775 conn(0x7fe78c0feaa0 msgr2=0x7fe78c107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:18.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:45:18.768+0000 7fe790b48700 1 -- 172.21.15.150:0/1278490775 shutdown_connections 2024-05-08T05:45:18.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:45:18.768+0000 7fe790b48700 1 -- 172.21.15.150:0/1278490775 wait complete. 2024-05-08T05:45:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:18 smithi112 ceph-mon[30003]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:19.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:18 smithi162 ceph-mon[34800]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:19.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:18 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:19.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:18 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:19.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:18 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:19.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:18 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:19.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:18 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:19.461 DEBUG:teuthology.orchestra.run.smithi150:osd.1> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.1.service 2024-05-08T05:45:19.463 INFO:tasks.cephadm:Deploying osd.2 on smithi162 with /dev/vg_nvme/lv_4... 2024-05-08T05:45:19.463 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T05:45:19.687 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:19 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:19.687 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:19 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:19.687 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:19 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:19.687 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:19 smithi150 ceph-mon[34315]: from='osd.1 [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:45:19.687 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:19 smithi150 ceph-mon[34315]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:45:19.708 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:45:19.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:19 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:19.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:19 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:19.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:19 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:19.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:19 smithi162 ceph-mon[34800]: from='osd.1 [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:45:19.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:19 smithi162 ceph-mon[34800]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:45:20.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:19 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:20.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:19 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:20.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:19 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:20.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:19 smithi112 ceph-mon[30003]: from='osd.1 [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:45:20.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:19 smithi112 ceph-mon[30003]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:45:20.859 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:20 smithi150 ceph-mon[34315]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:20.859 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:20 smithi150 ceph-mon[34315]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T05:45:20.860 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:20 smithi150 ceph-mon[34315]: from='osd.1 [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:45:20.860 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:20 smithi150 ceph-mon[34315]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T05:45:20.860 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:20 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:20.860 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:20 smithi150 ceph-mon[34315]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:45:20.929 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:20 smithi162 ceph-mon[34800]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:20.930 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:20 smithi162 ceph-mon[34800]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T05:45:20.930 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:20 smithi162 ceph-mon[34800]: from='osd.1 [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:45:20.930 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:20 smithi162 ceph-mon[34800]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T05:45:20.930 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:20 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:20.930 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:20 smithi162 ceph-mon[34800]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:45:21.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:20 smithi112 ceph-mon[30003]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:21.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:20 smithi112 ceph-mon[30003]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T05:45:21.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:20 smithi112 ceph-mon[30003]: from='osd.1 [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:45:21.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:20 smithi112 ceph-mon[30003]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T05:45:21.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:20 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:21.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:20 smithi112 ceph-mon[30003]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:45:21.457 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T05:45:21.457 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T05:45:21.457 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2024-05-08T05:45:21.457 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2024-05-08T05:45:21.457 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114213 s, 918 MB/s 2024-05-08T05:45:21.457 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2024-05-08T05:45:21.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:21 smithi150 ceph-mon[34315]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-08T05:45:21.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:21 smithi150 ceph-mon[34315]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T05:45:21.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:21 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:21.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:21 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:21.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:21 smithi150 ceph-mon[34315]: from='osd.1 ' entity='osd.1' 2024-05-08T05:45:21.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:21 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:21.934 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 05:45:21 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-1[38941]: 2024-05-08T05:45:21.673+0000 7f83ca546700 -1 osd.1 0 waiting for initial osdmap 2024-05-08T05:45:21.934 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 05:45:21 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-1[38941]: 2024-05-08T05:45:21.891+0000 7f83c2331700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:45:22.063 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:21 smithi162 ceph-mon[34800]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-08T05:45:22.063 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:21 smithi162 ceph-mon[34800]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T05:45:22.063 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:21 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:22.063 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:21 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:22.063 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:21 smithi162 ceph-mon[34800]: from='osd.1 ' entity='osd.1' 2024-05-08T05:45:22.063 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:21 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:22.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:21 smithi112 ceph-mon[30003]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-08T05:45:22.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:21 smithi112 ceph-mon[30003]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T05:45:22.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:21 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:22.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:21 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:22.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:21 smithi112 ceph-mon[30003]: from='osd.1 ' entity='osd.1' 2024-05-08T05:45:22.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:21 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:22.091 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_4 2024-05-08T05:45:22.332 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:45:23.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:22 smithi112 ceph-mon[30003]: purged_snaps scrub starts 2024-05-08T05:45:23.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:22 smithi112 ceph-mon[30003]: purged_snaps scrub ok 2024-05-08T05:45:23.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:22 smithi112 ceph-mon[30003]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:23.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:23.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:23.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:23.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:23.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:23.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:23.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:23.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:23.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:22 smithi162 ceph-mon[34800]: purged_snaps scrub starts 2024-05-08T05:45:23.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:22 smithi162 ceph-mon[34800]: purged_snaps scrub ok 2024-05-08T05:45:23.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:22 smithi162 ceph-mon[34800]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:23.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:23.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:23.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:23.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:23.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:23.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:23.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:23.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:23.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:22 smithi150 ceph-mon[34315]: purged_snaps scrub starts 2024-05-08T05:45:23.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:22 smithi150 ceph-mon[34315]: purged_snaps scrub ok 2024-05-08T05:45:23.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:22 smithi150 ceph-mon[34315]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:23.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:23.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:23.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:23.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:23.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:23.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:23.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:23.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:23.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.819+0000 7fa7d37b2700 1 -- 172.21.15.162:0/301504159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7cc103610 msgr2=0x7fa7cc103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:23.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.819+0000 7fa7d37b2700 1 --2- 172.21.15.162:0/301504159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7cc103610 0x7fa7cc103a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009990 tx=0x7fa7bc009ca0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.819+0000 7fa7d37b2700 1 -- 172.21.15.162:0/301504159 shutdown_connections 2024-05-08T05:45:23.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.819+0000 7fa7d37b2700 1 --2- 172.21.15.162:0/301504159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7cc103fb0 0x7fa7cc10a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.821 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.819+0000 7fa7d37b2700 1 --2- 172.21.15.162:0/301504159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7cc102410 0x7fa7cc102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.819+0000 7fa7d37b2700 1 --2- 172.21.15.162:0/301504159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7cc103610 0x7fa7cc103a70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.819+0000 7fa7d37b2700 1 -- 172.21.15.162:0/301504159 >> 172.21.15.162:0/301504159 conn(0x7fa7cc0fd9d0 msgr2=0x7fa7cc0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:23.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.819+0000 7fa7d37b2700 1 -- 172.21.15.162:0/301504159 shutdown_connections 2024-05-08T05:45:23.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.819+0000 7fa7d37b2700 1 -- 172.21.15.162:0/301504159 wait complete. 2024-05-08T05:45:23.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.820+0000 7fa7d37b2700 1 Processor -- start 2024-05-08T05:45:23.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.820+0000 7fa7d37b2700 1 -- start start 2024-05-08T05:45:23.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.820+0000 7fa7d37b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7cc102410 0x7fa7cc1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:23.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.820+0000 7fa7d37b2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7cc103610 0x7fa7cc198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:23.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.820+0000 7fa7d37b2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7cc103fb0 0x7fa7cc19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:23.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.820+0000 7fa7d37b2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc10df20 con 0x7fa7cc102410 2024-05-08T05:45:23.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.820+0000 7fa7d37b2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc10dda0 con 0x7fa7cc103fb0 2024-05-08T05:45:23.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.820+0000 7fa7d37b2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc10e0a0 con 0x7fa7cc103610 2024-05-08T05:45:23.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.820+0000 7fa7d0d4d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7cc103610 0x7fa7cc198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:23.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.820+0000 7fa7d0d4d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7cc103610 0x7fa7cc198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:52718/0 (socket says 172.21.15.162:52718) 2024-05-08T05:45:23.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.820+0000 7fa7d0d4d700 1 -- 172.21.15.162:0/2966680677 learned_addr learned my addr 172.21.15.162:0/2966680677 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-08T05:45:23.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.820+0000 7fa7d1d4f700 1 --2- 172.21.15.162:0/2966680677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7cc103fb0 0x7fa7cc19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:23.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.821+0000 7fa7d0d4d700 1 -- 172.21.15.162:0/2966680677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7cc103fb0 msgr2=0x7fa7cc19fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:23.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.821+0000 7fa7d0d4d700 1 --2- 172.21.15.162:0/2966680677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7cc103fb0 0x7fa7cc19fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.821+0000 7fa7d0d4d700 1 -- 172.21.15.162:0/2966680677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7cc102410 msgr2=0x7fa7cc1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:23.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.821+0000 7fa7d0d4d700 1 --2- 172.21.15.162:0/2966680677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7cc102410 0x7fa7cc1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.821+0000 7fa7d0d4d700 1 -- 172.21.15.162:0/2966680677 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7cc1a0230 con 0x7fa7cc103610 2024-05-08T05:45:23.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.821+0000 7fa7d0d4d700 1 --2- 172.21.15.162:0/2966680677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7cc103610 0x7fa7cc198b10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0059d0 tx=0x7fa7bc0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:23.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.821+0000 7fa7c27fc700 1 -- 172.21.15.162:0/2966680677 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc027070 con 0x7fa7cc103610 2024-05-08T05:45:23.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.821+0000 7fa7c27fc700 1 -- 172.21.15.162:0/2966680677 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7bc004180 con 0x7fa7cc103610 2024-05-08T05:45:23.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.821+0000 7fa7c27fc700 1 -- 172.21.15.162:0/2966680677 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc00f460 con 0x7fa7cc103610 2024-05-08T05:45:23.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.821+0000 7fa7d37b2700 1 -- 172.21.15.162:0/2966680677 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7cc1a04c0 con 0x7fa7cc103610 2024-05-08T05:45:23.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.821+0000 7fa7d37b2700 1 -- 172.21.15.162:0/2966680677 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7cc1a08c0 con 0x7fa7cc103610 2024-05-08T05:45:23.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.822+0000 7fa7d37b2700 1 -- 172.21.15.162:0/2966680677 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7cc06a190 con 0x7fa7cc103610 2024-05-08T05:45:23.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.823+0000 7fa7c27fc700 1 -- 172.21.15.162:0/2966680677 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7fa7bc02a070 con 0x7fa7cc103610 2024-05-08T05:45:23.825 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.824+0000 7fa7c27fc700 1 --2- 172.21.15.162:0/2966680677 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa7b80b27f0 0x7fa7b80b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:23.825 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.824+0000 7fa7c27fc700 1 -- 172.21.15.162:0/2966680677 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fa7bc0e1a90 con 0x7fa7cc103610 2024-05-08T05:45:23.825 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.824+0000 7fa7d154e700 1 --2- 172.21.15.162:0/2966680677 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa7b80b27f0 0x7fa7b80b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:23.825 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.825+0000 7fa7d154e700 1 --2- 172.21.15.162:0/2966680677 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa7b80b27f0 0x7fa7b80b4cb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc103470 tx=0x7fa7c8006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:23.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:23.829+0000 7fa7c27fc700 1 -- 172.21.15.162:0/2966680677 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7bc013070 con 0x7fa7cc103610 2024-05-08T05:45:24.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:24.042+0000 7fa7d37b2700 1 -- 172.21.15.162:0/2966680677 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa7cc0638e0 con 0x7fa7b80b27f0 2024-05-08T05:45:24.043 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:23 smithi162 ceph-mon[34800]: Detected new or changed devices on smithi150 2024-05-08T05:45:24.043 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:23 smithi162 ceph-mon[34800]: Adjusting osd_memory_target on smithi150 to 17088M 2024-05-08T05:45:24.043 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:23 smithi162 ceph-mon[34800]: osd.1 [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797] boot 2024-05-08T05:45:24.043 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:23 smithi162 ceph-mon[34800]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T05:45:24.043 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:23 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:24.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:23 smithi112 ceph-mon[30003]: Detected new or changed devices on smithi150 2024-05-08T05:45:24.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:23 smithi112 ceph-mon[30003]: Adjusting osd_memory_target on smithi150 to 17088M 2024-05-08T05:45:24.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:23 smithi112 ceph-mon[30003]: osd.1 [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797] boot 2024-05-08T05:45:24.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:23 smithi112 ceph-mon[30003]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T05:45:24.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:23 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:24.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:23 smithi150 ceph-mon[34315]: Detected new or changed devices on smithi150 2024-05-08T05:45:24.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:23 smithi150 ceph-mon[34315]: Adjusting osd_memory_target on smithi150 to 17088M 2024-05-08T05:45:24.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:23 smithi150 ceph-mon[34315]: osd.1 [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797] boot 2024-05-08T05:45:24.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:23 smithi150 ceph-mon[34315]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T05:45:24.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:23 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:45:24.992 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:24 smithi162 ceph-mon[34800]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:24.993 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:24 smithi162 ceph-mon[34800]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T05:45:24.993 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:24 smithi162 ceph-mon[34800]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:24.993 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:24.993 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:24.993 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:25.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:24 smithi112 ceph-mon[30003]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:25.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:24 smithi112 ceph-mon[30003]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T05:45:25.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:24 smithi112 ceph-mon[30003]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:25.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:25.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:25.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:25.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:24 smithi150 ceph-mon[34315]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:45:25.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:24 smithi150 ceph-mon[34315]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T05:45:25.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:24 smithi150 ceph-mon[34315]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:25.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:25.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:25.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:27.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:26 smithi112 ceph-mon[30003]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:27.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:26 smithi112 ceph-mon[30003]: from='client.? 172.21.15.162:0/1921473331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32e710e7-2a87-4863-bc7a-906d31aa4ef7"}]: dispatch 2024-05-08T05:45:27.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:26 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32e710e7-2a87-4863-bc7a-906d31aa4ef7"}]: dispatch 2024-05-08T05:45:27.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:26 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32e710e7-2a87-4863-bc7a-906d31aa4ef7"}]': finished 2024-05-08T05:45:27.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:26 smithi112 ceph-mon[30003]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T05:45:27.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:27.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:26 smithi162 ceph-mon[34800]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:27.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:26 smithi162 ceph-mon[34800]: from='client.? 172.21.15.162:0/1921473331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32e710e7-2a87-4863-bc7a-906d31aa4ef7"}]: dispatch 2024-05-08T05:45:27.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:26 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32e710e7-2a87-4863-bc7a-906d31aa4ef7"}]: dispatch 2024-05-08T05:45:27.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:26 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32e710e7-2a87-4863-bc7a-906d31aa4ef7"}]': finished 2024-05-08T05:45:27.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:26 smithi162 ceph-mon[34800]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T05:45:27.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:26 smithi150 ceph-mon[34315]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:26 smithi150 ceph-mon[34315]: from='client.? 172.21.15.162:0/1921473331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32e710e7-2a87-4863-bc7a-906d31aa4ef7"}]: dispatch 2024-05-08T05:45:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:26 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32e710e7-2a87-4863-bc7a-906d31aa4ef7"}]: dispatch 2024-05-08T05:45:27.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:26 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32e710e7-2a87-4863-bc7a-906d31aa4ef7"}]': finished 2024-05-08T05:45:27.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:26 smithi150 ceph-mon[34315]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T05:45:27.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:28.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:27 smithi112 ceph-mon[30003]: from='client.? 172.21.15.162:0/2723982385' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:28.284 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:27 smithi150 ceph-mon[34315]: from='client.? 172.21.15.162:0/2723982385' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:28.285 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:27 smithi162 ceph-mon[34800]: from='client.? 172.21.15.162:0/2723982385' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:29.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:28 smithi112 ceph-mon[30003]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:29.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:28 smithi162 ceph-mon[34800]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:29.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:28 smithi150 ceph-mon[34315]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:30.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:30 smithi162 ceph-mon[34800]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:31.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:30 smithi112 ceph-mon[30003]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:31.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:30 smithi150 ceph-mon[34315]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:32.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:32 smithi162 ceph-mon[34800]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:33.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:32 smithi112 ceph-mon[30003]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:33.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:32 smithi150 ceph-mon[34315]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:35.013 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:34 smithi162 ceph-mon[34800]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:35.013 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T05:45:35.013 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:35.013 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:34 smithi162 ceph-mon[34800]: Deploying daemon osd.2 on smithi162 2024-05-08T05:45:35.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:34 smithi112 ceph-mon[30003]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:35.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T05:45:35.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:35.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:34 smithi112 ceph-mon[30003]: Deploying daemon osd.2 on smithi162 2024-05-08T05:45:35.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:34 smithi150 ceph-mon[34315]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:35.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T05:45:35.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:35.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:34 smithi150 ceph-mon[34315]: Deploying daemon osd.2 on smithi162 2024-05-08T05:45:36.991 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:36 smithi162 ceph-mon[34800]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:37.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:36 smithi112 ceph-mon[30003]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:37.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:36 smithi150 ceph-mon[34315]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:38.932 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:38 smithi162 ceph-mon[34800]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:39.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:38 smithi112 ceph-mon[30003]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:39.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:38 smithi150 ceph-mon[34315]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:40.742 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:40 smithi162 ceph-mon[34800]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:40.742 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:40 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:40.742 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:40 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:40.742 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:40 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:40.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:40 smithi112 ceph-mon[30003]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:40.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:40 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:40.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:40 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:40.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:40 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:40.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:40 smithi150 ceph-mon[34315]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:40.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:40 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:40.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:40 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:40.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:40 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:42.627 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:42 smithi162 ceph-mon[34800]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:42.627 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:42 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:42.627 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:42 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:42.627 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:42 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:42.628 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:42 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:42.628 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:42 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:42.726 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:42.726+0000 7fa7c27fc700 1 -- 172.21.15.162:0/2966680677 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa7cc0638e0 con 0x7fa7b80b27f0 2024-05-08T05:45:42.727 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 2 on host 'smithi162' 2024-05-08T05:45:42.728 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:42.728+0000 7fa7d37b2700 1 -- 172.21.15.162:0/2966680677 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa7b80b27f0 msgr2=0x7fa7b80b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:42.728 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:42.728+0000 7fa7d37b2700 1 --2- 172.21.15.162:0/2966680677 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa7b80b27f0 0x7fa7b80b4cb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc103470 tx=0x7fa7c8006d20 comp rx=0 tx=0).stop 2024-05-08T05:45:42.728 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:42.728+0000 7fa7d37b2700 1 -- 172.21.15.162:0/2966680677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7cc103610 msgr2=0x7fa7cc198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:42.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:42.728+0000 7fa7d37b2700 1 --2- 172.21.15.162:0/2966680677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7cc103610 0x7fa7cc198b10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0059d0 tx=0x7fa7bc0057c0 comp rx=0 tx=0).stop 2024-05-08T05:45:42.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:42.728+0000 7fa7d37b2700 1 -- 172.21.15.162:0/2966680677 shutdown_connections 2024-05-08T05:45:42.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:42.728+0000 7fa7d37b2700 1 --2- 172.21.15.162:0/2966680677 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa7b80b27f0 0x7fa7b80b4cb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:42.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:42.728+0000 7fa7d37b2700 1 --2- 172.21.15.162:0/2966680677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7cc103fb0 0x7fa7cc19fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:42.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:42.728+0000 7fa7d37b2700 1 --2- 172.21.15.162:0/2966680677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7cc102410 0x7fa7cc1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:42.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:42.728+0000 7fa7d37b2700 1 --2- 172.21.15.162:0/2966680677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7cc103610 0x7fa7cc198b10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:42.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:42.728+0000 7fa7d37b2700 1 -- 172.21.15.162:0/2966680677 >> 172.21.15.162:0/2966680677 conn(0x7fa7cc0fd9d0 msgr2=0x7fa7cc106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:42.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:42.728+0000 7fa7d37b2700 1 -- 172.21.15.162:0/2966680677 shutdown_connections 2024-05-08T05:45:42.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:45:42.728+0000 7fa7d37b2700 1 -- 172.21.15.162:0/2966680677 wait complete. 2024-05-08T05:45:42.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:42 smithi112 ceph-mon[30003]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:42.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:42.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:42.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:42.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:42.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:42.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:42 smithi150 ceph-mon[34315]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:42.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:42 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:42.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:42 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:42.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:42 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:42.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:42 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:42.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:42 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:43.331 DEBUG:teuthology.orchestra.run.smithi162:osd.2> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.2.service 2024-05-08T05:45:43.334 INFO:tasks.cephadm:Deploying osd.3 on smithi112 with /dev/vg_nvme/lv_3... 2024-05-08T05:45:43.334 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T05:45:43.567 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 05:45:43 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-2[39958]: 2024-05-08T05:45:43.428+0000 7f53bfac87c0 -1 osd.2 0 log_to_monitors true 2024-05-08T05:45:43.767 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:45:43.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:43 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:43.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:43 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:43.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:43 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:43.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:43 smithi150 ceph-mon[34315]: from='osd.2 [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:45:43.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:43 smithi150 ceph-mon[34315]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:45:44.049 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:43 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:44.049 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:43 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:44.049 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:43 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:44.049 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:43 smithi112 ceph-mon[30003]: from='osd.2 [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:45:44.049 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:43 smithi112 ceph-mon[30003]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:45:44.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:43 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:44.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:43 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:44.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:43 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:44.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:43 smithi162 ceph-mon[34800]: from='osd.2 [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:45:44.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:43 smithi162 ceph-mon[34800]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:45:45.002 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:44 smithi112 ceph-mon[30003]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:45.002 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:44 smithi112 ceph-mon[30003]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T05:45:45.002 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:44 smithi112 ceph-mon[30003]: from='osd.2 [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:45:45.003 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:44 smithi112 ceph-mon[30003]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T05:45:45.003 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:45.003 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:44 smithi112 ceph-mon[30003]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:45:45.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:44 smithi162 ceph-mon[34800]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:45.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:44 smithi162 ceph-mon[34800]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T05:45:45.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:44 smithi162 ceph-mon[34800]: from='osd.2 [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:45:45.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:44 smithi162 ceph-mon[34800]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T05:45:45.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:45.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:44 smithi162 ceph-mon[34800]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:45:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:44 smithi150 ceph-mon[34315]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:44 smithi150 ceph-mon[34315]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T05:45:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:44 smithi150 ceph-mon[34315]: from='osd.2 [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:45:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:44 smithi150 ceph-mon[34315]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T05:45:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:44 smithi150 ceph-mon[34315]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:45:45.816 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 05:45:45 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-2[39958]: 2024-05-08T05:45:45.625+0000 7f53b592d700 -1 osd.2 0 waiting for initial osdmap 2024-05-08T05:45:45.817 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 05:45:45 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-2[39958]: 2024-05-08T05:45:45.708+0000 7f53acf17700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:45:45.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:45 smithi162 ceph-mon[34800]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-05-08T05:45:45.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:45 smithi162 ceph-mon[34800]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T05:45:45.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:45 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:45.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:45 smithi162 ceph-mon[34800]: from='osd.2 ' entity='osd.2' 2024-05-08T05:45:45.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:45 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:45.911 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:45 smithi112 ceph-mon[30003]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-05-08T05:45:45.911 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:45 smithi112 ceph-mon[30003]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T05:45:45.911 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:45 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:45.911 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:45 smithi112 ceph-mon[30003]: from='osd.2 ' entity='osd.2' 2024-05-08T05:45:45.911 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:45 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:46.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:45 smithi150 ceph-mon[34315]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-05-08T05:45:46.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:45 smithi150 ceph-mon[34315]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T05:45:46.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:45 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:46.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:45 smithi150 ceph-mon[34315]: from='osd.2 ' entity='osd.2' 2024-05-08T05:45:46.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:45 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:46.517 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T05:45:46.517 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T05:45:46.517 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-05-08T05:45:46.517 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-05-08T05:45:46.517 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0526641 s, 199 MB/s 2024-05-08T05:45:46.517 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-05-08T05:45:47.203 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_3 2024-05-08T05:45:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:47 smithi150 ceph-mon[34315]: purged_snaps scrub starts 2024-05-08T05:45:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:47 smithi150 ceph-mon[34315]: purged_snaps scrub ok 2024-05-08T05:45:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:47 smithi150 ceph-mon[34315]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:47 smithi150 ceph-mon[34315]: Detected new or changed devices on smithi162 2024-05-08T05:45:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:47 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:47.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:47 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:47.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:47 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:47.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:47 smithi150 ceph-mon[34315]: Adjusting osd_memory_target on smithi162 to 17088M 2024-05-08T05:45:47.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:47 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:47.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:47 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:47.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:47 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:47.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:47 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:47.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:47 smithi150 ceph-mon[34315]: osd.2 [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215] boot 2024-05-08T05:45:47.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:47 smithi150 ceph-mon[34315]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T05:45:47.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:47 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:47 smithi112 ceph-mon[30003]: purged_snaps scrub starts 2024-05-08T05:45:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:47 smithi112 ceph-mon[30003]: purged_snaps scrub ok 2024-05-08T05:45:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:47 smithi112 ceph-mon[30003]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:47 smithi112 ceph-mon[30003]: Detected new or changed devices on smithi162 2024-05-08T05:45:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:47 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:47 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:47 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:47 smithi112 ceph-mon[30003]: Adjusting osd_memory_target on smithi162 to 17088M 2024-05-08T05:45:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:47 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:47 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:47 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:47 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:47.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:47 smithi112 ceph-mon[30003]: osd.2 [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215] boot 2024-05-08T05:45:47.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:47 smithi112 ceph-mon[30003]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T05:45:47.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:47 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:47.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:47 smithi162 ceph-mon[34800]: purged_snaps scrub starts 2024-05-08T05:45:47.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:47 smithi162 ceph-mon[34800]: purged_snaps scrub ok 2024-05-08T05:45:47.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:47 smithi162 ceph-mon[34800]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:47.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:47 smithi162 ceph-mon[34800]: Detected new or changed devices on smithi162 2024-05-08T05:45:47.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:47 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:47.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:47 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:47.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:47 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:47.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:47 smithi162 ceph-mon[34800]: Adjusting osd_memory_target on smithi162 to 17088M 2024-05-08T05:45:47.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:47 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:47.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:47 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:47.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:47 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:47.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:47 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:47.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:47 smithi162 ceph-mon[34800]: osd.2 [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215] boot 2024-05-08T05:45:47.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:47 smithi162 ceph-mon[34800]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T05:45:47.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:47 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:47.634 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:45:48.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:48 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:45:48.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:48 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:45:48.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:48 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:45:49.458 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:49 smithi112 ceph-mon[30003]: pgmap v90: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:49.458 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:49 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T05:45:49.458 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:49 smithi112 ceph-mon[30003]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T05:45:49.458 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:49 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:45:49.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:49 smithi162 ceph-mon[34800]: pgmap v90: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:49.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:49 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T05:45:49.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:49 smithi162 ceph-mon[34800]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T05:45:49.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:49 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:45:49.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:49 smithi150 ceph-mon[34315]: pgmap v90: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:49.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:49 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T05:45:49.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:49 smithi150 ceph-mon[34315]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T05:45:49.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:49 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:45:49.723 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 05:45:49 smithi112 sudo[46941]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:45:49.723 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 05:45:49 smithi112 sudo[46941]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:49.724 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 05:45:49 smithi112 sudo[46941]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:49.724 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 05:45:49 smithi112 sudo[46941]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:49.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.788+0000 7f6c08000700 1 -- 172.21.15.112:0/1686701043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c0010b3e0 msgr2=0x7f6c0010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:49.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.788+0000 7f6c08000700 1 --2- 172.21.15.112:0/1686701043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c0010b3e0 0x7f6c0010d870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009f90 tx=0x7f6bf800b330 comp rx=0 tx=0).stop 2024-05-08T05:45:49.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.788+0000 7f6c08000700 1 -- 172.21.15.112:0/1686701043 shutdown_connections 2024-05-08T05:45:49.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.788+0000 7f6c08000700 1 --2- 172.21.15.112:0/1686701043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c0010b3e0 0x7f6c0010d870 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:49.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.788+0000 7f6c08000700 1 --2- 172.21.15.112:0/1686701043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c00077470 0x7f6c000758d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:49.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.788+0000 7f6c08000700 1 --2- 172.21.15.112:0/1686701043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c00075ea0 0x7f6c00076320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:49.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.788+0000 7f6c08000700 1 -- 172.21.15.112:0/1686701043 >> 172.21.15.112:0/1686701043 conn(0x7f6c000feaa0 msgr2=0x7f6c00100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:49.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.788+0000 7f6c08000700 1 -- 172.21.15.112:0/1686701043 shutdown_connections 2024-05-08T05:45:49.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.788+0000 7f6c08000700 1 -- 172.21.15.112:0/1686701043 wait complete. 2024-05-08T05:45:49.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.789+0000 7f6c08000700 1 Processor -- start 2024-05-08T05:45:49.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.789+0000 7f6c08000700 1 -- start start 2024-05-08T05:45:49.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.789+0000 7f6c08000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c00075ea0 0x7f6c0019b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:49.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.789+0000 7f6c08000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c00077470 0x7f6c0019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:49.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.789+0000 7f6c08000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c0010b3e0 0x7f6c001a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:49.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.789+0000 7f6c08000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001111a0 con 0x7f6c00077470 2024-05-08T05:45:49.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.789+0000 7f6c08000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6c00111020 con 0x7f6c00075ea0 2024-05-08T05:45:49.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.789+0000 7f6c08000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6c00111320 con 0x7f6c0010b3e0 2024-05-08T05:45:49.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.790+0000 7f6c05600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c00075ea0 0x7f6c0019b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:49.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.790+0000 7f6c05600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c00075ea0 0x7f6c0019b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:53848/0 (socket says 172.21.15.112:53848) 2024-05-08T05:45:49.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.790+0000 7f6c05600700 1 -- 172.21.15.112:0/3192578845 learned_addr learned my addr 172.21.15.112:0/3192578845 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:45:49.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.790+0000 7f6c06000700 1 --2- 172.21.15.112:0/3192578845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c0010b3e0 0x7f6c001a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.790+0000 7f6c04c00700 1 --2- 172.21.15.112:0/3192578845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c00077470 0x7f6c0019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.790+0000 7f6c05600700 1 -- 172.21.15.112:0/3192578845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c0010b3e0 msgr2=0x7f6c001a2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.790+0000 7f6c05600700 1 --2- 172.21.15.112:0/3192578845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c0010b3e0 0x7f6c001a2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.790+0000 7f6c05600700 1 -- 172.21.15.112:0/3192578845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c00077470 msgr2=0x7f6c0019bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.790+0000 7f6c05600700 1 --2- 172.21.15.112:0/3192578845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c00077470 0x7f6c0019bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.790+0000 7f6c05600700 1 -- 172.21.15.112:0/3192578845 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c001a33f0 con 0x7f6c00075ea0 2024-05-08T05:45:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.790+0000 7f6c06000700 1 --2- 172.21.15.112:0/3192578845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c0010b3e0 0x7f6c001a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:49.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.790+0000 7f6c05600700 1 --2- 172.21.15.112:0/3192578845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c00075ea0 0x7f6c0019b790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400eb30 tx=0x7f6bf400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:49.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.791+0000 7f6bfe000700 1 -- 172.21.15.112:0/3192578845 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf400cd40 con 0x7f6c00075ea0 2024-05-08T05:45:49.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.791+0000 7f6c08000700 1 -- 172.21.15.112:0/3192578845 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c001a36e0 con 0x7f6c00075ea0 2024-05-08T05:45:49.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.791+0000 7f6bfe000700 1 -- 172.21.15.112:0/3192578845 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bf40044d0 con 0x7f6c00075ea0 2024-05-08T05:45:49.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.791+0000 7f6bfe000700 1 -- 172.21.15.112:0/3192578845 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4010430 con 0x7f6c00075ea0 2024-05-08T05:45:49.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.791+0000 7f6c08000700 1 -- 172.21.15.112:0/3192578845 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c001a3c70 con 0x7f6c00075ea0 2024-05-08T05:45:49.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.792+0000 7f6c08000700 1 -- 172.21.15.112:0/3192578845 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c0006a190 con 0x7f6c00075ea0 2024-05-08T05:45:49.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.794+0000 7f6bfe000700 1 -- 172.21.15.112:0/3192578845 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f6bf4003fe0 con 0x7f6c00075ea0 2024-05-08T05:45:49.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.796+0000 7f6bfe000700 1 --2- 172.21.15.112:0/3192578845 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6be80b2870 0x7f6be80b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:49.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.796+0000 7f6bfe000700 1 -- 172.21.15.112:0/3192578845 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f6bf40e11c0 con 0x7f6c00075ea0 2024-05-08T05:45:49.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.796+0000 7f6c04c00700 1 --2- 172.21.15.112:0/3192578845 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6be80b2870 0x7f6be80b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:49.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.796+0000 7f6c04c00700 1 --2- 172.21.15.112:0/3192578845 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6be80b2870 0x7f6be80b4d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0005fd0 tx=0x7f6bf0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:49.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:49.802+0000 7f6bfe000700 1 -- 172.21.15.112:0/3192578845 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6bf40aa380 con 0x7f6c00075ea0 2024-05-08T05:45:49.992 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:49 smithi112 sudo[46967]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:45:49.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:49 smithi112 sudo[46967]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:49.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:49 smithi112 sudo[46967]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:49.993 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:49 smithi112 sudo[46967]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:49.993 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 05:45:49 smithi112 sudo[46944]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:45:49.993 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 05:45:49 smithi112 sudo[46944]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:49.993 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 05:45:49 smithi112 sudo[46944]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:49.993 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 05:45:49 smithi112 sudo[46944]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:50.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:50.057+0000 7f6c08000700 1 -- 172.21.15.112:0/3192578845 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6c000638e0 con 0x7f6be80b2870 2024-05-08T05:45:50.081 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 05:45:49 smithi150 sudo[42654]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:45:50.081 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 05:45:49 smithi150 sudo[42654]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:50.081 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 05:45:49 smithi150 sudo[42654]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:50.081 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 05:45:49 smithi150 sudo[42654]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:50.081 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 05:45:49 smithi150 sudo[42657]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:45:50.081 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 05:45:49 smithi150 sudo[42657]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:50.081 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 05:45:49 smithi150 sudo[42657]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:50.081 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 05:45:49 smithi150 sudo[42657]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:50.190 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 sudo[43598]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:45:50.190 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 sudo[43598]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:50.190 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 sudo[43598]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:50.190 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 sudo[43598]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:50.190 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 sudo[43601]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:45:50.190 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 sudo[43601]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:50.190 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 sudo[43601]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:50.190 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 sudo[43601]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:50.191 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 05:45:49 smithi162 sudo[43592]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:45:50.191 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 05:45:49 smithi162 sudo[43592]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:50.191 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 05:45:49 smithi162 sudo[43592]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:50.191 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 05:45:49 smithi162 sudo[43592]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:50.191 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 05:45:49 smithi162 sudo[43595]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:45:50.191 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 05:45:49 smithi162 sudo[43595]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:50.191 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 05:45:49 smithi162 sudo[43595]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:50.191 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 05:45:49 smithi162 sudo[43595]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:50.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 sudo[46970]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:45:50.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 sudo[46970]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:50.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 sudo[46970]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:50.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 sudo[46970]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:50.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T05:45:50.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: osdmap e20: 3 total, 3 up, 3 in 2024-05-08T05:45:50.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:50.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:50.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:50.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:50.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:50.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:50.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:50.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:50.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:50.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:50.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:50.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:50.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:50.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:50.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:50.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:50.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:50.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:50.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:50.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:50.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:50 smithi112 ceph-mon[30003]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:50.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 sudo[42660]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:45:50.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 sudo[42660]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:50.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 sudo[42660]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:50.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 sudo[42660]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:50.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 sudo[42663]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:45:50.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 sudo[42663]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:50.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 sudo[42663]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:50.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 sudo[42663]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:50.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T05:45:50.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: osdmap e20: 3 total, 3 up, 3 in 2024-05-08T05:45:50.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:50.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:50.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:50.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:50.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:50.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:50.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:50.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:50.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:50.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:50.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:50.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:50.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:50.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:50.436 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:50.436 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:50.436 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:50.436 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:50.436 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:50.436 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:50.436 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:50 smithi150 ceph-mon[34315]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:50.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T05:45:50.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: osdmap e20: 3 total, 3 up, 3 in 2024-05-08T05:45:50.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:50.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:50.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:45:50.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:50.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:50.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:50 smithi162 ceph-mon[34800]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:51.306 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:51 smithi112 ceph-mon[30003]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:51.306 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:51 smithi112 ceph-mon[30003]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:51.306 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:51 smithi112 ceph-mon[30003]: osdmap e21: 3 total, 3 up, 3 in 2024-05-08T05:45:51.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:51 smithi162 ceph-mon[34800]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:51.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:51 smithi162 ceph-mon[34800]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:51.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:51 smithi162 ceph-mon[34800]: osdmap e21: 3 total, 3 up, 3 in 2024-05-08T05:45:51.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:51 smithi150 ceph-mon[34315]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:51.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:51 smithi150 ceph-mon[34315]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:51.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:51 smithi150 ceph-mon[34315]: osdmap e21: 3 total, 3 up, 3 in 2024-05-08T05:45:52.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:45:52.224+0000 7f6bfe000700 1 -- 172.21.15.112:0/3192578845 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f6bf40a5f10 con 0x7f6c00075ea0 2024-05-08T05:45:53.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:53 smithi150 ceph-mon[34315]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:53.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:53 smithi150 ceph-mon[34315]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T05:45:53.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:53 smithi112 ceph-mon[30003]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:53.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:53 smithi112 ceph-mon[30003]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T05:45:53.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:53 smithi162 ceph-mon[34800]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:53.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:53 smithi162 ceph-mon[34800]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T05:45:54.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:54 smithi112 ceph-mon[30003]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:54.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:54 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/45695086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba1d4a5f-6a95-4b9e-8a48-8524a899b004"}]: dispatch 2024-05-08T05:45:54.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:54 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/45695086' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba1d4a5f-6a95-4b9e-8a48-8524a899b004"}]': finished 2024-05-08T05:45:54.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:54 smithi112 ceph-mon[30003]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T05:45:54.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:54 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:54.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:54 smithi162 ceph-mon[34800]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:54.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:54 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/45695086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba1d4a5f-6a95-4b9e-8a48-8524a899b004"}]: dispatch 2024-05-08T05:45:54.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:54 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/45695086' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba1d4a5f-6a95-4b9e-8a48-8524a899b004"}]': finished 2024-05-08T05:45:54.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:54 smithi162 ceph-mon[34800]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T05:45:54.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:54 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:54.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:54 smithi150 ceph-mon[34315]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:54.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:54 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/45695086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba1d4a5f-6a95-4b9e-8a48-8524a899b004"}]: dispatch 2024-05-08T05:45:54.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:54 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/45695086' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba1d4a5f-6a95-4b9e-8a48-8524a899b004"}]': finished 2024-05-08T05:45:54.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:54 smithi150 ceph-mon[34315]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T05:45:54.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:54 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:55.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:55 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2690521618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:55.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:55 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/2690521618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:55.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:55 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/2690521618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:56.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:56 smithi112 ceph-mon[30003]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:56.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:56 smithi162 ceph-mon[34800]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:56.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:56 smithi150 ceph-mon[34315]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:58.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:45:58 smithi162 ceph-mon[34800]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:58.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:45:58 smithi150 ceph-mon[34315]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:58.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:45:58 smithi112 ceph-mon[30003]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:00.628 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:00 smithi112 ceph-mon[30003]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:00.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:00 smithi150 ceph-mon[34315]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:00.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:00 smithi162 ceph-mon[34800]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:02.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:02 smithi112 ceph-mon[30003]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:02.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:02 smithi162 ceph-mon[34800]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:02.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:02 smithi150 ceph-mon[34315]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:03.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:03 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T05:46:03.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:03 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:03.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:03 smithi112 ceph-mon[30003]: Deploying daemon osd.3 on smithi112 2024-05-08T05:46:03.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:03 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T05:46:03.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:03 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:03.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:03 smithi162 ceph-mon[34800]: Deploying daemon osd.3 on smithi112 2024-05-08T05:46:03.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:03 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T05:46:03.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:03 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:03.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:03 smithi150 ceph-mon[34315]: Deploying daemon osd.3 on smithi112 2024-05-08T05:46:04.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:04 smithi112 ceph-mon[30003]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:04.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:04 smithi162 ceph-mon[34800]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:04.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:04 smithi150 ceph-mon[34315]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:06.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:06 smithi112 ceph-mon[30003]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:06.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:06 smithi162 ceph-mon[34800]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:06.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:06 smithi150 ceph-mon[34315]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:08.679 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:08 smithi112 ceph-mon[30003]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:08.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:08 smithi162 ceph-mon[34800]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:08.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:08 smithi150 ceph-mon[34315]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:10.694 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:10 smithi112 ceph-mon[30003]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:10.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:10 smithi162 ceph-mon[34800]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:10.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:10 smithi150 ceph-mon[34315]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:12.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:12.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:12.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:12.439 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:12 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:12.439 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:12 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:12.439 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:12 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:12.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:12 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:12.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:12 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:12.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:12 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:13.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:13 smithi112 ceph-mon[30003]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:13.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:13 smithi162 ceph-mon[34800]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:13.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:13 smithi150 ceph-mon[34315]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:15.428 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:15 smithi112 ceph-mon[30003]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:15.428 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:15 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:15.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:15 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:15.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:15 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:15.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:15 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:15.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:15 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:15.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:15 smithi112 ceph-mon[30003]: from='osd.3 [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T05:46:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:15 smithi162 ceph-mon[34800]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:15 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:15 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:15 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:15 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:15 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:15 smithi162 ceph-mon[34800]: from='osd.3 [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T05:46:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:15 smithi150 ceph-mon[34315]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:15 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:15 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:15 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:15 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:15 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:15 smithi150 ceph-mon[34315]: from='osd.3 [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T05:46:16.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:46:16.109+0000 7f6bfe000700 1 -- 172.21.15.112:0/3192578845 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6c000638e0 con 0x7f6be80b2870 2024-05-08T05:46:16.110 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 3 on host 'smithi112' 2024-05-08T05:46:16.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:46:16.112+0000 7f6c08000700 1 -- 172.21.15.112:0/3192578845 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6be80b2870 msgr2=0x7f6be80b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:16.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:46:16.112+0000 7f6c08000700 1 --2- 172.21.15.112:0/3192578845 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6be80b2870 0x7f6be80b4d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0005fd0 tx=0x7f6bf0009310 comp rx=0 tx=0).stop 2024-05-08T05:46:16.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:46:16.112+0000 7f6c08000700 1 -- 172.21.15.112:0/3192578845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c00075ea0 msgr2=0x7f6c0019b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:16.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:46:16.113+0000 7f6c08000700 1 --2- 172.21.15.112:0/3192578845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c00075ea0 0x7f6c0019b790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400eb30 tx=0x7f6bf400eef0 comp rx=0 tx=0).stop 2024-05-08T05:46:16.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:46:16.113+0000 7f6c08000700 1 -- 172.21.15.112:0/3192578845 shutdown_connections 2024-05-08T05:46:16.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:46:16.113+0000 7f6c08000700 1 --2- 172.21.15.112:0/3192578845 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6be80b2870 0x7f6be80b4d30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:16.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:46:16.113+0000 7f6c08000700 1 --2- 172.21.15.112:0/3192578845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c0010b3e0 0x7f6c001a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:16.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:46:16.113+0000 7f6c08000700 1 --2- 172.21.15.112:0/3192578845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c00075ea0 0x7f6c0019b790 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:16.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:46:16.113+0000 7f6c08000700 1 --2- 172.21.15.112:0/3192578845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c00077470 0x7f6c0019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:16.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:46:16.113+0000 7f6c08000700 1 -- 172.21.15.112:0/3192578845 >> 172.21.15.112:0/3192578845 conn(0x7f6c000feaa0 msgr2=0x7f6c00107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:16.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:46:16.113+0000 7f6c08000700 1 -- 172.21.15.112:0/3192578845 shutdown_connections 2024-05-08T05:46:16.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:46:16.113+0000 7f6c08000700 1 -- 172.21.15.112:0/3192578845 wait complete. 2024-05-08T05:46:16.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:16 smithi112 ceph-mon[30003]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:16.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:16 smithi112 ceph-mon[30003]: from='osd.3 [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T05:46:16.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:16 smithi112 ceph-mon[30003]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T05:46:16.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:16 smithi112 ceph-mon[30003]: from='osd.3 [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:46:16.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:16 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:46:16.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:16 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:16.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:16 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:16.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:16 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:16.815 DEBUG:teuthology.orchestra.run.smithi112:osd.3> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.3.service 2024-05-08T05:46:16.819 INFO:tasks.cephadm:Deploying osd.4 on smithi150 with /dev/vg_nvme/lv_3... 2024-05-08T05:46:16.820 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T05:46:16.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:16 smithi150 ceph-mon[34315]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:16.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:16 smithi150 ceph-mon[34315]: from='osd.3 [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T05:46:16.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:16 smithi150 ceph-mon[34315]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T05:46:16.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:16 smithi150 ceph-mon[34315]: from='osd.3 [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:46:16.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:16 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:46:16.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:16 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:16.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:16 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:16.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:16 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:17.064 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:46:17.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:16 smithi162 ceph-mon[34800]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:17.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:16 smithi162 ceph-mon[34800]: from='osd.3 [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T05:46:17.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:16 smithi162 ceph-mon[34800]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T05:46:17.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:16 smithi162 ceph-mon[34800]: from='osd.3 [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:46:17.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:16 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:46:17.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:16 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:17.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:16 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:17.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:16 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:17.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:17 smithi150 ceph-mon[34315]: from='osd.3 [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-08T05:46:17.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:17 smithi150 ceph-mon[34315]: osdmap e24: 4 total, 3 up, 4 in 2024-05-08T05:46:17.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:17 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:46:17.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:17 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:46:18.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:17 smithi112 ceph-mon[30003]: from='osd.3 [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-08T05:46:18.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:17 smithi112 ceph-mon[30003]: osdmap e24: 4 total, 3 up, 4 in 2024-05-08T05:46:18.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:17 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:46:18.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:17 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:46:18.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:17 smithi162 ceph-mon[34800]: from='osd.3 [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-08T05:46:18.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:17 smithi162 ceph-mon[34800]: osdmap e24: 4 total, 3 up, 4 in 2024-05-08T05:46:18.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:17 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:46:18.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:17 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:46:18.806 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T05:46:18.807 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T05:46:18.807 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-08T05:46:18.807 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-08T05:46:18.807 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0142001 s, 738 MB/s 2024-05-08T05:46:18.807 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 2024-05-08T05:46:18.807 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-08T05:46:18.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:18 smithi150 ceph-mon[34315]: purged_snaps scrub starts 2024-05-08T05:46:18.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:18 smithi150 ceph-mon[34315]: purged_snaps scrub ok 2024-05-08T05:46:18.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:18 smithi150 ceph-mon[34315]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:18.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:18 smithi150 ceph-mon[34315]: osd.3 [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755] boot 2024-05-08T05:46:18.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:18 smithi150 ceph-mon[34315]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T05:46:18.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:18 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:46:18.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:18 smithi150 ceph-mon[34315]: osdmap e26: 4 total, 4 up, 4 in 2024-05-08T05:46:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:18 smithi112 ceph-mon[30003]: purged_snaps scrub starts 2024-05-08T05:46:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:18 smithi112 ceph-mon[30003]: purged_snaps scrub ok 2024-05-08T05:46:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:18 smithi112 ceph-mon[30003]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:18 smithi112 ceph-mon[30003]: osd.3 [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755] boot 2024-05-08T05:46:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:18 smithi112 ceph-mon[30003]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T05:46:19.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:46:19.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:18 smithi112 ceph-mon[30003]: osdmap e26: 4 total, 4 up, 4 in 2024-05-08T05:46:19.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:18 smithi162 ceph-mon[34800]: purged_snaps scrub starts 2024-05-08T05:46:19.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:18 smithi162 ceph-mon[34800]: purged_snaps scrub ok 2024-05-08T05:46:19.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:18 smithi162 ceph-mon[34800]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:46:19.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:18 smithi162 ceph-mon[34800]: osd.3 [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755] boot 2024-05-08T05:46:19.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:18 smithi162 ceph-mon[34800]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T05:46:19.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:18 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:46:19.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:18 smithi162 ceph-mon[34800]: osdmap e26: 4 total, 4 up, 4 in 2024-05-08T05:46:19.491 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_3 2024-05-08T05:46:19.730 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:46:20.087 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:19 smithi150 ceph-mon[34315]: osdmap e27: 4 total, 4 up, 4 in 2024-05-08T05:46:20.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:19 smithi112 ceph-mon[30003]: osdmap e27: 4 total, 4 up, 4 in 2024-05-08T05:46:20.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:19 smithi162 ceph-mon[34800]: osdmap e27: 4 total, 4 up, 4 in 2024-05-08T05:46:20.989 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:20 smithi150 ceph-mon[34315]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:21.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.188+0000 7f37716fa700 1 -- 172.21.15.150:0/230423354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f376c1050d0 msgr2=0x7f376c10b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:21.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.188+0000 7f37716fa700 1 --2- 172.21.15.150:0/230423354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f376c1050d0 0x7f376c10b980 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f376400b0d0 tx=0x7f376400b3e0 comp rx=0 tx=0).stop 2024-05-08T05:46:21.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.188+0000 7f37716fa700 1 -- 172.21.15.150:0/230423354 shutdown_connections 2024-05-08T05:46:21.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.188+0000 7f37716fa700 1 --2- 172.21.15.150:0/230423354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f376c1050d0 0x7f376c10b980 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:21.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.188+0000 7f37716fa700 1 --2- 172.21.15.150:0/230423354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f376c103530 0x7f376c103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:21.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.188+0000 7f37716fa700 1 --2- 172.21.15.150:0/230423354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f376c104730 0x7f376c104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:21.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.188+0000 7f37716fa700 1 -- 172.21.15.150:0/230423354 >> 172.21.15.150:0/230423354 conn(0x7f376c0feaf0 msgr2=0x7f376c100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:21.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.188+0000 7f37716fa700 1 -- 172.21.15.150:0/230423354 shutdown_connections 2024-05-08T05:46:21.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.188+0000 7f37716fa700 1 -- 172.21.15.150:0/230423354 wait complete. 2024-05-08T05:46:21.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f37716fa700 1 Processor -- start 2024-05-08T05:46:21.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f37716fa700 1 -- start start 2024-05-08T05:46:21.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f37716fa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f376c103530 0x7f376c077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:21.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f37716fa700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f376c104730 0x7f376c075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:21.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f37716fa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f376c1050d0 0x7f376c075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:21.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f37716fa700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f376c10f020 con 0x7f376c103530 2024-05-08T05:46:21.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f37716fa700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f376c10eea0 con 0x7f376c1050d0 2024-05-08T05:46:21.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f37716fa700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f376c10f1a0 con 0x7f376c104730 2024-05-08T05:46:21.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f376b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f376c1050d0 0x7f376c075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:21.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f376b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f376c1050d0 0x7f376c075dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57496/0 (socket says 172.21.15.150:57496) 2024-05-08T05:46:21.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f376b7fe700 1 -- 172.21.15.150:0/2328681835 learned_addr learned my addr 172.21.15.150:0/2328681835 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-08T05:46:21.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f37627fc700 1 --2- 172.21.15.150:0/2328681835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f376c104730 0x7f376c075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:21.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f376b7fe700 1 -- 172.21.15.150:0/2328681835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f376c104730 msgr2=0x7f376c075890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:21.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f376b7fe700 1 --2- 172.21.15.150:0/2328681835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f376c104730 0x7f376c075890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:21.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f376b7fe700 1 -- 172.21.15.150:0/2328681835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f376c103530 msgr2=0x7f376c077240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:21.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f376b7fe700 1 --2- 172.21.15.150:0/2328681835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f376c103530 0x7f376c077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:21.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f376b7fe700 1 -- 172.21.15.150:0/2328681835 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c076680 con 0x7f376c1050d0 2024-05-08T05:46:21.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.189+0000 7f37627fc700 1 --2- 172.21.15.150:0/2328681835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f376c104730 0x7f376c075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:21.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.190+0000 7f376b7fe700 1 --2- 172.21.15.150:0/2328681835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f376c1050d0 0x7f376c075dd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3764009fd0 tx=0x7f3764004db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:21.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.190+0000 7f3768ff9700 1 -- 172.21.15.150:0/2328681835 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376401f030 con 0x7f376c1050d0 2024-05-08T05:46:21.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.190+0000 7f37716fa700 1 -- 172.21.15.150:0/2328681835 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376c1a3770 con 0x7f376c1050d0 2024-05-08T05:46:21.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.190+0000 7f3768ff9700 1 -- 172.21.15.150:0/2328681835 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3764003a00 con 0x7f376c1050d0 2024-05-08T05:46:21.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.190+0000 7f3768ff9700 1 -- 172.21.15.150:0/2328681835 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37640047c0 con 0x7f376c1050d0 2024-05-08T05:46:21.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.190+0000 7f37716fa700 1 -- 172.21.15.150:0/2328681835 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376c1a3bc0 con 0x7f376c1050d0 2024-05-08T05:46:21.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.191+0000 7f37716fa700 1 -- 172.21.15.150:0/2328681835 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f376c06a190 con 0x7f376c1050d0 2024-05-08T05:46:21.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.192+0000 7f3768ff9700 1 -- 172.21.15.150:0/2328681835 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3764003bb0 con 0x7f376c1050d0 2024-05-08T05:46:21.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.192+0000 7f3768ff9700 1 --2- 172.21.15.150:0/2328681835 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f37580b2880 0x7f37580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:21.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.193+0000 7f3768ff9700 1 -- 172.21.15.150:0/2328681835 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f376400f020 con 0x7f376c1050d0 2024-05-08T05:46:21.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.193+0000 7f376affd700 1 --2- 172.21.15.150:0/2328681835 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f37580b2880 0x7f37580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:21.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.194+0000 7f376affd700 1 --2- 172.21.15.150:0/2328681835 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f37580b2880 0x7f37580b4d40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f375c005fd0 tx=0x7f375c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:21.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.197+0000 7f3768ff9700 1 -- 172.21.15.150:0/2328681835 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3764029020 con 0x7f376c1050d0 2024-05-08T05:46:21.208 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:20 smithi112 ceph-mon[30003]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:21.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:20 smithi162 ceph-mon[34800]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:21.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:21.410+0000 7f37716fa700 1 -- 172.21.15.150:0/2328681835 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f376c0639a0 con 0x7f37580b2880 2024-05-08T05:46:22.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:22 smithi150 ceph-mon[34315]: Detected new or changed devices on smithi112 2024-05-08T05:46:22.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:22.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:22.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:22.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:22.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:22 smithi150 ceph-mon[34315]: Adjusting osd_memory_target on smithi112 to 8544M 2024-05-08T05:46:22.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:22.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:22.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:22.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:22.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:22 smithi150 ceph-mon[34315]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:22.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:22 smithi150 ceph-mon[34315]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:46:22.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:46:22.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:46:22.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:22.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:22 smithi112 ceph-mon[30003]: Detected new or changed devices on smithi112 2024-05-08T05:46:22.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:22.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:22.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:22.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:22.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:22 smithi112 ceph-mon[30003]: Adjusting osd_memory_target on smithi112 to 8544M 2024-05-08T05:46:22.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:22.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:22.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:22.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:22.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:22 smithi112 ceph-mon[30003]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:22.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:22 smithi112 ceph-mon[30003]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:46:22.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:46:22.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:46:22.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:22.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:22 smithi162 ceph-mon[34800]: Detected new or changed devices on smithi112 2024-05-08T05:46:22.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:22.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:22.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:22.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:22.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:22 smithi162 ceph-mon[34800]: Adjusting osd_memory_target on smithi112 to 8544M 2024-05-08T05:46:22.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:22.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:22.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:22.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:22.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:22 smithi162 ceph-mon[34800]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:22.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:22 smithi162 ceph-mon[34800]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:46:22.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:46:22.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:46:22.819 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:24.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:24 smithi150 ceph-mon[34315]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:24.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:24 smithi150 ceph-mon[34315]: from='client.? 172.21.15.150:0/1628649317' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb4f1a42-8de8-4b50-b3bc-602de9f4803a"}]: dispatch 2024-05-08T05:46:24.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:24 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb4f1a42-8de8-4b50-b3bc-602de9f4803a"}]: dispatch 2024-05-08T05:46:24.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:24 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb4f1a42-8de8-4b50-b3bc-602de9f4803a"}]': finished 2024-05-08T05:46:24.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:24 smithi150 ceph-mon[34315]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T05:46:24.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:46:24.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:24 smithi112 ceph-mon[30003]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:24.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:24 smithi112 ceph-mon[30003]: from='client.? 172.21.15.150:0/1628649317' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb4f1a42-8de8-4b50-b3bc-602de9f4803a"}]: dispatch 2024-05-08T05:46:24.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:24 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb4f1a42-8de8-4b50-b3bc-602de9f4803a"}]: dispatch 2024-05-08T05:46:24.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:24 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb4f1a42-8de8-4b50-b3bc-602de9f4803a"}]': finished 2024-05-08T05:46:24.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:24 smithi112 ceph-mon[30003]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T05:46:24.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:46:24.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:24 smithi162 ceph-mon[34800]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:24.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:24 smithi162 ceph-mon[34800]: from='client.? 172.21.15.150:0/1628649317' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb4f1a42-8de8-4b50-b3bc-602de9f4803a"}]: dispatch 2024-05-08T05:46:24.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:24 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb4f1a42-8de8-4b50-b3bc-602de9f4803a"}]: dispatch 2024-05-08T05:46:24.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:24 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb4f1a42-8de8-4b50-b3bc-602de9f4803a"}]': finished 2024-05-08T05:46:24.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:24 smithi162 ceph-mon[34800]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T05:46:24.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:46:25.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:25 smithi112 ceph-mon[30003]: from='client.? 172.21.15.150:0/3358274374' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:46:25.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:25 smithi162 ceph-mon[34800]: from='client.? 172.21.15.150:0/3358274374' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:46:25.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:25 smithi150 ceph-mon[34315]: from='client.? 172.21.15.150:0/3358274374' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:46:26.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:26 smithi112 ceph-mon[30003]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-08T05:46:26.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:26 smithi162 ceph-mon[34800]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-08T05:46:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:26 smithi150 ceph-mon[34315]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-08T05:46:28.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:28 smithi112 ceph-mon[30003]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T05:46:28.641 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:28 smithi150 ceph-mon[34315]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T05:46:28.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:28 smithi162 ceph-mon[34800]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T05:46:30.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:30 smithi150 ceph-mon[34315]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:46:30.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:30 smithi112 ceph-mon[30003]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:46:30.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:30 smithi162 ceph-mon[34800]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:46:31.685 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T05:46:31.686 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:31.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T05:46:31.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:31.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T05:46:31.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:32.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:32 smithi150 ceph-mon[34315]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:46:32.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:32 smithi150 ceph-mon[34315]: Deploying daemon osd.4 on smithi150 2024-05-08T05:46:32.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:32 smithi112 ceph-mon[30003]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:46:32.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:32 smithi112 ceph-mon[30003]: Deploying daemon osd.4 on smithi150 2024-05-08T05:46:32.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:32 smithi162 ceph-mon[34800]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:46:32.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:32 smithi162 ceph-mon[34800]: Deploying daemon osd.4 on smithi150 2024-05-08T05:46:34.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:34 smithi150 ceph-mon[34315]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:46:34.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:34 smithi112 ceph-mon[30003]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:46:34.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:34 smithi162 ceph-mon[34800]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:46:36.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:36 smithi150 ceph-mon[34315]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:36.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:36 smithi112 ceph-mon[30003]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:36.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:36 smithi162 ceph-mon[34800]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:38.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:37 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:38.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:37 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:38.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:37 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:38.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:37 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:38.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:37 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:38.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:37 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:38.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:37 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:38.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:37 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:38.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:37 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:39.049 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:38 smithi150 ceph-mon[34315]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:39.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:38 smithi112 ceph-mon[30003]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:39.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:38 smithi162 ceph-mon[34800]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:39.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:39.787+0000 7f3768ff9700 1 -- 172.21.15.150:0/2328681835 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f376c0639a0 con 0x7f37580b2880 2024-05-08T05:46:39.788 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 4 on host 'smithi150' 2024-05-08T05:46:39.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:39.790+0000 7f37716fa700 1 -- 172.21.15.150:0/2328681835 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f37580b2880 msgr2=0x7f37580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:39.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:39.790+0000 7f37716fa700 1 --2- 172.21.15.150:0/2328681835 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f37580b2880 0x7f37580b4d40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f375c005fd0 tx=0x7f375c005d00 comp rx=0 tx=0).stop 2024-05-08T05:46:39.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:39.790+0000 7f37716fa700 1 -- 172.21.15.150:0/2328681835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f376c1050d0 msgr2=0x7f376c075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:39.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:39.790+0000 7f37716fa700 1 --2- 172.21.15.150:0/2328681835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f376c1050d0 0x7f376c075dd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3764009fd0 tx=0x7f3764004db0 comp rx=0 tx=0).stop 2024-05-08T05:46:39.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:39.790+0000 7f37716fa700 1 -- 172.21.15.150:0/2328681835 shutdown_connections 2024-05-08T05:46:39.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:39.790+0000 7f37716fa700 1 --2- 172.21.15.150:0/2328681835 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f37580b2880 0x7f37580b4d40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:39.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:39.790+0000 7f37716fa700 1 --2- 172.21.15.150:0/2328681835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f376c1050d0 0x7f376c075dd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:39.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:39.790+0000 7f37716fa700 1 --2- 172.21.15.150:0/2328681835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f376c103530 0x7f376c077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:39.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:39.790+0000 7f37716fa700 1 --2- 172.21.15.150:0/2328681835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f376c104730 0x7f376c075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:39.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:39.790+0000 7f37716fa700 1 -- 172.21.15.150:0/2328681835 >> 172.21.15.150:0/2328681835 conn(0x7f376c0feaf0 msgr2=0x7f376c1079d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:39.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:39.790+0000 7f37716fa700 1 -- 172.21.15.150:0/2328681835 shutdown_connections 2024-05-08T05:46:39.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:46:39.790+0000 7f37716fa700 1 -- 172.21.15.150:0/2328681835 wait complete. 2024-05-08T05:46:40.530 DEBUG:teuthology.orchestra.run.smithi150:osd.4> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.4.service 2024-05-08T05:46:40.532 INFO:tasks.cephadm:Deploying osd.5 on smithi162 with /dev/vg_nvme/lv_3... 2024-05-08T05:46:40.532 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T05:46:40.784 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:46:40.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:40 smithi162 ceph-mon[34800]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:40.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:40 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:40.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:40 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:40.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:40 smithi162 ceph-mon[34800]: from='osd.4 [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:46:40.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:40 smithi162 ceph-mon[34800]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:46:40.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:40 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:40.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:40 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:40.933 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 05:46:40 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-4[45481]: 2024-05-08T05:46:40.653+0000 7fc4d7b297c0 -1 osd.4 0 log_to_monitors true 2024-05-08T05:46:40.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:40 smithi150 ceph-mon[34315]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:40.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:40 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:40.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:40 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:40.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:40 smithi150 ceph-mon[34315]: from='osd.4 [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:46:40.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:40 smithi150 ceph-mon[34315]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:46:40.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:40 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:40.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:40 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:41.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:40 smithi112 ceph-mon[30003]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:41.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:40 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:41.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:40 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:41.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:40 smithi112 ceph-mon[30003]: from='osd.4 [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:46:41.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:40 smithi112 ceph-mon[30003]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:46:41.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:40 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:41.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:40 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:41.933 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 05:46:41 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-4[45481]: 2024-05-08T05:46:41.789+0000 7fc4cd18d700 -1 osd.4 0 waiting for initial osdmap 2024-05-08T05:46:41.933 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 05:46:41 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-4[45481]: 2024-05-08T05:46:41.795+0000 7fc4c8f80700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:46:41.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:41 smithi150 ceph-mon[34315]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T05:46:41.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:41 smithi150 ceph-mon[34315]: from='osd.4 [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:46:41.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:41 smithi150 ceph-mon[34315]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T05:46:41.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:41 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:46:41.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:41 smithi150 ceph-mon[34315]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:46:42.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:41 smithi112 ceph-mon[30003]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T05:46:42.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:41 smithi112 ceph-mon[30003]: from='osd.4 [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:46:42.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:41 smithi112 ceph-mon[30003]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T05:46:42.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:41 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:46:42.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:41 smithi112 ceph-mon[30003]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:46:42.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:41 smithi162 ceph-mon[34800]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T05:46:42.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:41 smithi162 ceph-mon[34800]: from='osd.4 [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:46:42.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:41 smithi162 ceph-mon[34800]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T05:46:42.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:41 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:46:42.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:41 smithi162 ceph-mon[34800]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:46:42.719 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T05:46:42.719 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T05:46:42.719 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2024-05-08T05:46:42.719 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2024-05-08T05:46:42.719 INFO:teuthology.orchestra.run.smithi162.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0135113 s, 776 MB/s 2024-05-08T05:46:42.719 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 2024-05-08T05:46:42.719 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2024-05-08T05:46:43.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:42 smithi112 ceph-mon[30003]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:43.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:42 smithi112 ceph-mon[30003]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-08T05:46:43.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:42 smithi112 ceph-mon[30003]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T05:46:43.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:46:43.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:46:43.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:42 smithi162 ceph-mon[34800]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:43.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:42 smithi162 ceph-mon[34800]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-08T05:46:43.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:42 smithi162 ceph-mon[34800]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T05:46:43.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:42 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:46:43.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:42 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:46:43.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:42 smithi150 ceph-mon[34315]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:43.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:42 smithi150 ceph-mon[34315]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-08T05:46:43.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:42 smithi150 ceph-mon[34315]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T05:46:43.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:42 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:46:43.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:42 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:46:43.361 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_3 2024-05-08T05:46:43.607 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:46:43.934 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:43 smithi162 ceph-mon[34800]: purged_snaps scrub starts 2024-05-08T05:46:43.934 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:43 smithi162 ceph-mon[34800]: purged_snaps scrub ok 2024-05-08T05:46:43.934 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:43 smithi162 ceph-mon[34800]: osd.4 [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683] boot 2024-05-08T05:46:43.935 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:43 smithi162 ceph-mon[34800]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T05:46:43.935 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:43 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:46:44.020 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:43 smithi150 ceph-mon[34315]: purged_snaps scrub starts 2024-05-08T05:46:44.020 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:43 smithi150 ceph-mon[34315]: purged_snaps scrub ok 2024-05-08T05:46:44.020 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:43 smithi150 ceph-mon[34315]: osd.4 [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683] boot 2024-05-08T05:46:44.020 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:43 smithi150 ceph-mon[34315]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T05:46:44.020 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:43 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:46:44.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:43 smithi112 ceph-mon[30003]: purged_snaps scrub starts 2024-05-08T05:46:44.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:43 smithi112 ceph-mon[30003]: purged_snaps scrub ok 2024-05-08T05:46:44.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:43 smithi112 ceph-mon[30003]: osd.4 [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683] boot 2024-05-08T05:46:44.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:43 smithi112 ceph-mon[30003]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T05:46:44.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:43 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:46:44.940 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:44.941 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T05:46:44.941 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: Detected new or changed devices on smithi150 2024-05-08T05:46:44.941 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:44.941 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:44.941 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:44.941 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:44.941 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: Adjusting osd_memory_target on smithi150 to 8544M 2024-05-08T05:46:44.941 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:44.941 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:44.941 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:44.942 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:44.942 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:44.942 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:44.942 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:44.942 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:45.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.152+0000 7f9b6fc42700 1 -- 172.21.15.162:0/2469190398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b681092b0 msgr2=0x7f9b6810b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:45.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.152+0000 7f9b6fc42700 1 --2- 172.21.15.162:0/2469190398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b681092b0 0x7f9b6810b6a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000b0a0 tx=0x7f9b6000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.152+0000 7f9b6fc42700 1 -- 172.21.15.162:0/2469190398 shutdown_connections 2024-05-08T05:46:45.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.152+0000 7f9b6fc42700 1 --2- 172.21.15.162:0/2469190398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b681092b0 0x7f9b6810b6a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.152+0000 7f9b6fc42700 1 --2- 172.21.15.162:0/2469190398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b68104050 0x7f9b68106440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.152+0000 7f9b6fc42700 1 --2- 172.21.15.162:0/2469190398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b68106980 0x7f9b68108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.152+0000 7f9b6fc42700 1 -- 172.21.15.162:0/2469190398 >> 172.21.15.162:0/2469190398 conn(0x7f9b680fd9a0 msgr2=0x7f9b680ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:45.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.152+0000 7f9b6fc42700 1 -- 172.21.15.162:0/2469190398 shutdown_connections 2024-05-08T05:46:45.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.152+0000 7f9b6fc42700 1 -- 172.21.15.162:0/2469190398 wait complete. 2024-05-08T05:46:45.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.152+0000 7f9b6fc42700 1 Processor -- start 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6fc42700 1 -- start start 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6fc42700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b68104050 0x7f9b68199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6fc42700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b68106980 0x7f9b68199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6fc42700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b681092b0 0x7f9b681a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6fc42700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6810efd0 con 0x7f9b68104050 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6fc42700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6810ee50 con 0x7f9b681092b0 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6fc42700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6810f150 con 0x7f9b68106980 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6d1dd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b68106980 0x7f9b68199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6d1dd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b68106980 0x7f9b68199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:45686/0 (socket says 172.21.15.162:45686) 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6d1dd700 1 -- 172.21.15.162:0/1568654530 learned_addr learned my addr 172.21.15.162:0/1568654530 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6e1df700 1 --2- 172.21.15.162:0/1568654530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b681092b0 0x7f9b681a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6d1dd700 1 -- 172.21.15.162:0/1568654530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b681092b0 msgr2=0x7f9b681a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6d1dd700 1 --2- 172.21.15.162:0/1568654530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b681092b0 0x7f9b681a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6d1dd700 1 -- 172.21.15.162:0/1568654530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b68104050 msgr2=0x7f9b68199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6d1dd700 1 --2- 172.21.15.162:0/1568654530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b68104050 0x7f9b68199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6d1dd700 1 -- 172.21.15.162:0/1568654530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b681a11c0 con 0x7f9b68106980 2024-05-08T05:46:45.155 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b6d1dd700 1 --2- 172.21.15.162:0/1568654530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b68106980 0x7f9b68199aa0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800b560 tx=0x7f9b5800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:45.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.153+0000 7f9b5effd700 1 -- 172.21.15.162:0/1568654530 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58015020 con 0x7f9b68106980 2024-05-08T05:46:45.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.154+0000 7f9b6fc42700 1 -- 172.21.15.162:0/1568654530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b681a14b0 con 0x7f9b68106980 2024-05-08T05:46:45.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.154+0000 7f9b5effd700 1 -- 172.21.15.162:0/1568654530 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b58011ce0 con 0x7f9b68106980 2024-05-08T05:46:45.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.154+0000 7f9b5effd700 1 -- 172.21.15.162:0/1568654530 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58010d20 con 0x7f9b68106980 2024-05-08T05:46:45.156 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.154+0000 7f9b6fc42700 1 -- 172.21.15.162:0/1568654530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b681a1ac0 con 0x7f9b68106980 2024-05-08T05:46:45.157 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.154+0000 7f9b6fc42700 1 -- 172.21.15.162:0/1568654530 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b6806a190 con 0x7f9b68106980 2024-05-08T05:46:45.157 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.156+0000 7f9b5effd700 1 -- 172.21.15.162:0/1568654530 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f9b58018020 con 0x7f9b68106980 2024-05-08T05:46:45.157 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.156+0000 7f9b5effd700 1 --2- 172.21.15.162:0/1568654530 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9b540b2880 0x7f9b540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.157 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.156+0000 7f9b5effd700 1 -- 172.21.15.162:0/1568654530 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f9b580e0ea0 con 0x7f9b68106980 2024-05-08T05:46:45.157 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.157+0000 7f9b6d9de700 1 --2- 172.21.15.162:0/1568654530 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9b540b2880 0x7f9b540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.157 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.158+0000 7f9b6d9de700 1 --2- 172.21.15.162:0/1568654530 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9b540b2880 0x7f9b540b4d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9b64005fd0 tx=0x7f9b64005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:45.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.160+0000 7f9b5effd700 1 -- 172.21.15.162:0/1568654530 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b5801f020 con 0x7f9b68106980 2024-05-08T05:46:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T05:46:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: Detected new or changed devices on smithi150 2024-05-08T05:46:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:45.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: Adjusting osd_memory_target on smithi150 to 8544M 2024-05-08T05:46:45.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:45.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:45.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:45.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:45.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:45.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:45.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:45.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:45.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:46:45.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T05:46:45.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: Detected new or changed devices on smithi150 2024-05-08T05:46:45.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:45.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:45.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:45.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:45.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: Adjusting osd_memory_target on smithi150 to 8544M 2024-05-08T05:46:45.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:45.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:45.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:45.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:45.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:45.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:45.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:45.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:46:45.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:46:45.371+0000 7f9b6fc42700 1 -- 172.21.15.162:0/1568654530 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9b680638e0 con 0x7f9b540b2880 2024-05-08T05:46:46.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:45 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:46:46.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:45 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:46:46.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:45 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:46.107 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:45 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:46:46.107 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:45 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:46:46.107 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:45 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:46.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:45 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:46:46.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:45 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:46:46.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:45 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:47.111 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:46 smithi162 ceph-mon[34800]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:46:47.111 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:46 smithi162 ceph-mon[34800]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:47.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:46 smithi150 ceph-mon[34315]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:46:47.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:46 smithi150 ceph-mon[34315]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:47.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:46 smithi112 ceph-mon[30003]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:46:47.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:46 smithi112 ceph-mon[30003]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:49.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:48 smithi150 ceph-mon[34315]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:49.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:48 smithi150 ceph-mon[34315]: from='client.? 172.21.15.162:0/1093993146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26c03b8-050c-4a2c-8f23-5801fffb7c78"}]: dispatch 2024-05-08T05:46:49.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:48 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26c03b8-050c-4a2c-8f23-5801fffb7c78"}]: dispatch 2024-05-08T05:46:49.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:48 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a26c03b8-050c-4a2c-8f23-5801fffb7c78"}]': finished 2024-05-08T05:46:49.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:48 smithi150 ceph-mon[34315]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T05:46:49.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:48 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:46:49.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:48 smithi150 ceph-mon[34315]: from='client.? 172.21.15.162:0/1957275604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:46:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:48 smithi112 ceph-mon[30003]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:48 smithi112 ceph-mon[30003]: from='client.? 172.21.15.162:0/1093993146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26c03b8-050c-4a2c-8f23-5801fffb7c78"}]: dispatch 2024-05-08T05:46:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:48 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26c03b8-050c-4a2c-8f23-5801fffb7c78"}]: dispatch 2024-05-08T05:46:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:48 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a26c03b8-050c-4a2c-8f23-5801fffb7c78"}]': finished 2024-05-08T05:46:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:48 smithi112 ceph-mon[30003]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T05:46:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:48 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:46:49.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:48 smithi112 ceph-mon[30003]: from='client.? 172.21.15.162:0/1957275604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:46:49.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:48 smithi162 ceph-mon[34800]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:49.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:48 smithi162 ceph-mon[34800]: from='client.? 172.21.15.162:0/1093993146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26c03b8-050c-4a2c-8f23-5801fffb7c78"}]: dispatch 2024-05-08T05:46:49.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:48 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26c03b8-050c-4a2c-8f23-5801fffb7c78"}]: dispatch 2024-05-08T05:46:49.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:48 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a26c03b8-050c-4a2c-8f23-5801fffb7c78"}]': finished 2024-05-08T05:46:49.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:48 smithi162 ceph-mon[34800]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T05:46:49.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:48 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:46:49.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:48 smithi162 ceph-mon[34800]: from='client.? 172.21.15.162:0/1957275604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:46:51.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:50 smithi150 ceph-mon[34315]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:51.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:50 smithi112 ceph-mon[30003]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:51.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:50 smithi162 ceph-mon[34800]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:53.163 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:52 smithi162 ceph-mon[34800]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:53.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:52 smithi150 ceph-mon[34315]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:53.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:52 smithi112 ceph-mon[30003]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:55.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:54 smithi162 ceph-mon[34800]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:55.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:54 smithi150 ceph-mon[34315]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:55.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:54 smithi112 ceph-mon[30003]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:56.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T05:46:56.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:56.219 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T05:46:56.219 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:56.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T05:46:56.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:57.041 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:56 smithi162 ceph-mon[34800]: Deploying daemon osd.5 on smithi162 2024-05-08T05:46:57.041 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:56 smithi162 ceph-mon[34800]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:57.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:56 smithi150 ceph-mon[34315]: Deploying daemon osd.5 on smithi162 2024-05-08T05:46:57.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:56 smithi150 ceph-mon[34315]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:57.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:56 smithi112 ceph-mon[30003]: Deploying daemon osd.5 on smithi162 2024-05-08T05:46:57.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:56 smithi112 ceph-mon[30003]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:58.976 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:46:58 smithi162 ceph-mon[34800]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:59.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:46:58 smithi150 ceph-mon[34315]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:59.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:46:58 smithi112 ceph-mon[30003]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:01.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:00 smithi150 ceph-mon[34315]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:01.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:00 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:01.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:00 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:01.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:00 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:01.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:00 smithi112 ceph-mon[30003]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:01.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:00 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:01.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:00 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:01.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:00 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:01.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:00 smithi162 ceph-mon[34800]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:01.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:00 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:01.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:00 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:01.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:00 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:03.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:02 smithi150 ceph-mon[34315]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:03.194 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:02 smithi162 ceph-mon[34800]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:03.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:02 smithi112 ceph-mon[30003]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:04.206 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:47:04.204+0000 7f9b5effd700 1 -- 172.21.15.162:0/1568654530 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9b680638e0 con 0x7f9b540b2880 2024-05-08T05:47:04.206 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 5 on host 'smithi162' 2024-05-08T05:47:04.208 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:47:04.207+0000 7f9b6fc42700 1 -- 172.21.15.162:0/1568654530 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9b540b2880 msgr2=0x7f9b540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.208 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:47:04.207+0000 7f9b6fc42700 1 --2- 172.21.15.162:0/1568654530 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9b540b2880 0x7f9b540b4d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9b64005fd0 tx=0x7f9b64005d00 comp rx=0 tx=0).stop 2024-05-08T05:47:04.208 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:47:04.207+0000 7f9b6fc42700 1 -- 172.21.15.162:0/1568654530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b68106980 msgr2=0x7f9b68199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.208 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:47:04.207+0000 7f9b6fc42700 1 --2- 172.21.15.162:0/1568654530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b68106980 0x7f9b68199aa0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800b560 tx=0x7f9b5800b870 comp rx=0 tx=0).stop 2024-05-08T05:47:04.208 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:47:04.207+0000 7f9b6fc42700 1 -- 172.21.15.162:0/1568654530 shutdown_connections 2024-05-08T05:47:04.208 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:47:04.207+0000 7f9b6fc42700 1 --2- 172.21.15.162:0/1568654530 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9b540b2880 0x7f9b540b4d40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.209 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:47:04.207+0000 7f9b6fc42700 1 --2- 172.21.15.162:0/1568654530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b681092b0 0x7f9b681a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.209 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:47:04.207+0000 7f9b6fc42700 1 --2- 172.21.15.162:0/1568654530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b68104050 0x7f9b68199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.209 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:47:04.207+0000 7f9b6fc42700 1 --2- 172.21.15.162:0/1568654530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b68106980 0x7f9b68199aa0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.209 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:47:04.207+0000 7f9b6fc42700 1 -- 172.21.15.162:0/1568654530 >> 172.21.15.162:0/1568654530 conn(0x7f9b680fd9a0 msgr2=0x7f9b68104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:04.209 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:47:04.207+0000 7f9b6fc42700 1 -- 172.21.15.162:0/1568654530 shutdown_connections 2024-05-08T05:47:04.209 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:47:04.207+0000 7f9b6fc42700 1 -- 172.21.15.162:0/1568654530 wait complete. 2024-05-08T05:47:04.521 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:04.521 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:04.522 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:04.522 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:04.522 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:04.522 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:04 smithi162 ceph-mon[34800]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:04.522 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:04.522 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:04.522 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:04.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:04.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:04.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:04.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:04.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:04.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:04 smithi112 ceph-mon[30003]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:04.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:04.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:04.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:04 smithi150 ceph-mon[34315]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:04.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:04.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:04.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:04.834 DEBUG:teuthology.orchestra.run.smithi162:osd.5> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.5.service 2024-05-08T05:47:04.836 INFO:tasks.cephadm:Deploying osd.6 on smithi112 with /dev/vg_nvme/lv_2... 2024-05-08T05:47:04.837 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T05:47:05.286 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:47:05.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:05 smithi162 ceph-mon[34800]: from='osd.5 [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:47:05.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:05 smithi162 ceph-mon[34800]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:47:05.662 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:05 smithi112 ceph-mon[30003]: from='osd.5 [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:47:05.662 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:05 smithi112 ceph-mon[30003]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:47:05.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:05 smithi150 ceph-mon[34315]: from='osd.5 [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:47:05.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:05 smithi150 ceph-mon[34315]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:47:06.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:06 smithi162 ceph-mon[34800]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T05:47:06.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:06 smithi162 ceph-mon[34800]: from='osd.5 [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:47:06.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:06 smithi162 ceph-mon[34800]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T05:47:06.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:06 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:47:06.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:06 smithi162 ceph-mon[34800]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:47:06.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:06 smithi162 ceph-mon[34800]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:06.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:06 smithi150 ceph-mon[34315]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T05:47:06.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:06 smithi150 ceph-mon[34315]: from='osd.5 [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:47:06.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:06 smithi150 ceph-mon[34315]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T05:47:06.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:06 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:47:06.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:06 smithi150 ceph-mon[34315]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:47:06.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:06 smithi150 ceph-mon[34315]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:06.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:06 smithi112 ceph-mon[30003]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T05:47:06.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:06 smithi112 ceph-mon[30003]: from='osd.5 [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:47:06.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:06 smithi112 ceph-mon[30003]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T05:47:06.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:06 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:47:06.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:06 smithi112 ceph-mon[30003]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:47:06.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:06 smithi112 ceph-mon[30003]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:07 smithi162 ceph-mon[34800]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-05-08T05:47:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:07 smithi162 ceph-mon[34800]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T05:47:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:47:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:07 smithi162 ceph-mon[34800]: from='osd.5 ' entity='osd.5' 2024-05-08T05:47:07.555 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 05:47:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T05:47:07.327+0000 7f60d8b6c700 -1 osd.5 0 waiting for initial osdmap 2024-05-08T05:47:07.555 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 05:47:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T05:47:07.445+0000 7f60d0156700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:47:07.672 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:07 smithi112 ceph-mon[30003]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-05-08T05:47:07.672 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:07 smithi112 ceph-mon[30003]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T05:47:07.672 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:47:07.672 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:07 smithi112 ceph-mon[30003]: from='osd.5 ' entity='osd.5' 2024-05-08T05:47:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:07 smithi150 ceph-mon[34315]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-05-08T05:47:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:07 smithi150 ceph-mon[34315]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T05:47:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:47:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:07 smithi150 ceph-mon[34315]: from='osd.5 ' entity='osd.5' 2024-05-08T05:47:08.245 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T05:47:08.246 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T05:47:08.246 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-05-08T05:47:08.246 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-05-08T05:47:08.246 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0549355 s, 191 MB/s 2024-05-08T05:47:08.246 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-05-08T05:47:08.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: purged_snaps scrub starts 2024-05-08T05:47:08.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: purged_snaps scrub ok 2024-05-08T05:47:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: osdmap e36: 6 total, 5 up, 6 in 2024-05-08T05:47:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:47:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: Detected new or changed devices on smithi162 2024-05-08T05:47:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: Adjusting osd_memory_target on smithi162 to 8544M 2024-05-08T05:47:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:47:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: purged_snaps scrub starts 2024-05-08T05:47:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: purged_snaps scrub ok 2024-05-08T05:47:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: osdmap e36: 6 total, 5 up, 6 in 2024-05-08T05:47:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:47:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: Detected new or changed devices on smithi162 2024-05-08T05:47:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: Adjusting osd_memory_target on smithi162 to 8544M 2024-05-08T05:47:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:47:08.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: purged_snaps scrub starts 2024-05-08T05:47:08.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: purged_snaps scrub ok 2024-05-08T05:47:08.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:08.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: osdmap e36: 6 total, 5 up, 6 in 2024-05-08T05:47:08.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:47:08.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: Detected new or changed devices on smithi162 2024-05-08T05:47:08.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:08.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:08.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:08.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:08.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: Adjusting osd_memory_target on smithi162 to 8544M 2024-05-08T05:47:08.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:08.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:08.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:08.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:08.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:47:09.200 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_2 2024-05-08T05:47:09.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:09 smithi112 ceph-mon[30003]: osd.5 [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750] boot 2024-05-08T05:47:09.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:09 smithi112 ceph-mon[30003]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T05:47:09.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:09 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:47:09.636 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:47:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:09 smithi150 ceph-mon[34315]: osd.5 [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750] boot 2024-05-08T05:47:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:09 smithi150 ceph-mon[34315]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T05:47:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:09 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:47:09.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:09 smithi162 ceph-mon[34800]: osd.5 [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750] boot 2024-05-08T05:47:09.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:09 smithi162 ceph-mon[34800]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T05:47:09.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:09 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:47:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:10 smithi150 ceph-mon[34315]: pgmap v150: 1 pgs: 1 remapped+peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:10 smithi150 ceph-mon[34315]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T05:47:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:10 smithi150 ceph-mon[34315]: osdmap e39: 6 total, 6 up, 6 in 2024-05-08T05:47:10.799 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:10 smithi112 ceph-mon[30003]: pgmap v150: 1 pgs: 1 remapped+peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:10.799 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:10 smithi112 ceph-mon[30003]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T05:47:10.800 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:10 smithi112 ceph-mon[30003]: osdmap e39: 6 total, 6 up, 6 in 2024-05-08T05:47:10.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:10 smithi162 ceph-mon[34800]: pgmap v150: 1 pgs: 1 remapped+peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:47:10.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:10 smithi162 ceph-mon[34800]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T05:47:10.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:10 smithi162 ceph-mon[34800]: osdmap e39: 6 total, 6 up, 6 in 2024-05-08T05:47:11.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.775+0000 7ff50be00700 1 -- 172.21.15.112:0/1594579831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c105640 msgr2=0x7ff50c105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:11.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.775+0000 7ff50be00700 1 --2- 172.21.15.112:0/1594579831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c105640 0x7ff50c105a60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0099c0 tx=0x7ff4fc009cd0 comp rx=0 tx=0).stop 2024-05-08T05:47:11.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.775+0000 7ff50be00700 1 -- 172.21.15.112:0/1594579831 shutdown_connections 2024-05-08T05:47:11.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.775+0000 7ff50be00700 1 --2- 172.21.15.112:0/1594579831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff50c1071e0 0x7ff50c10da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:11.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.775+0000 7ff50be00700 1 --2- 172.21.15.112:0/1594579831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c105640 0x7ff50c105a60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:11.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.775+0000 7ff50be00700 1 --2- 172.21.15.112:0/1594579831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff50c106840 0x7ff50c106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:11.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.775+0000 7ff50be00700 1 -- 172.21.15.112:0/1594579831 >> 172.21.15.112:0/1594579831 conn(0x7ff50c100be0 msgr2=0x7ff50c103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:11.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.775+0000 7ff50be00700 1 -- 172.21.15.112:0/1594579831 shutdown_connections 2024-05-08T05:47:11.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.775+0000 7ff50be00700 1 -- 172.21.15.112:0/1594579831 wait complete. 2024-05-08T05:47:11.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.776+0000 7ff50be00700 1 Processor -- start 2024-05-08T05:47:11.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.777+0000 7ff50be00700 1 -- start start 2024-05-08T05:47:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.777+0000 7ff50be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c105640 0x7ff50c19b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.777+0000 7ff50be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff50c106840 0x7ff50c19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.777+0000 7ff50be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff50c1071e0 0x7ff50c1a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.777+0000 7ff50be00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c111130 con 0x7ff50c105640 2024-05-08T05:47:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.777+0000 7ff50be00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c110fb0 con 0x7ff50c106840 2024-05-08T05:47:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.777+0000 7ff50be00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1112b0 con 0x7ff50c1071e0 2024-05-08T05:47:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.777+0000 7ff50aa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c105640 0x7ff50c19b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.777+0000 7ff50aa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c105640 0x7ff50c19b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36246/0 (socket says 172.21.15.112:36246) 2024-05-08T05:47:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.777+0000 7ff50aa00700 1 -- 172.21.15.112:0/2984963886 learned_addr learned my addr 172.21.15.112:0/2984963886 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:47:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.778+0000 7ff50a000700 1 --2- 172.21.15.112:0/2984963886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff50c106840 0x7ff50c19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.778+0000 7ff50b400700 1 --2- 172.21.15.112:0/2984963886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff50c1071e0 0x7ff50c1a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.778+0000 7ff50aa00700 1 -- 172.21.15.112:0/2984963886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff50c1071e0 msgr2=0x7ff50c1a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.778+0000 7ff50aa00700 1 --2- 172.21.15.112:0/2984963886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff50c1071e0 0x7ff50c1a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.778+0000 7ff50aa00700 1 -- 172.21.15.112:0/2984963886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff50c106840 msgr2=0x7ff50c19bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.778+0000 7ff50aa00700 1 --2- 172.21.15.112:0/2984963886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff50c106840 0x7ff50c19bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.778+0000 7ff50aa00700 1 -- 172.21.15.112:0/2984963886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c1a3440 con 0x7ff50c105640 2024-05-08T05:47:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.778+0000 7ff50aa00700 1 --2- 172.21.15.112:0/2984963886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c105640 0x7ff50c19b7e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00b580 tx=0x7ff4fc003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.779+0000 7ff503400700 1 -- 172.21.15.112:0/2984963886 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc009e60 con 0x7ff50c105640 2024-05-08T05:47:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.779+0000 7ff503400700 1 -- 172.21.15.112:0/2984963886 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4fc005570 con 0x7ff50c105640 2024-05-08T05:47:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.779+0000 7ff50be00700 1 -- 172.21.15.112:0/2984963886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c1a36d0 con 0x7ff50c105640 2024-05-08T05:47:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.779+0000 7ff503400700 1 -- 172.21.15.112:0/2984963886 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00fe00 con 0x7ff50c105640 2024-05-08T05:47:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.779+0000 7ff50be00700 1 -- 172.21.15.112:0/2984963886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c1a3b20 con 0x7ff50c105640 2024-05-08T05:47:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.781+0000 7ff50be00700 1 -- 172.21.15.112:0/2984963886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff50c06a190 con 0x7ff50c105640 2024-05-08T05:47:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.782+0000 7ff503400700 1 -- 172.21.15.112:0/2984963886 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ff4fc00f4f0 con 0x7ff50c105640 2024-05-08T05:47:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.785+0000 7ff503400700 1 --2- 172.21.15.112:0/2984963886 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff4f00b2880 0x7ff4f00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.785+0000 7ff503400700 1 -- 172.21.15.112:0/2984963886 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4533+0+0 (secure 0 0 0) 0x7ff4fc0e21a0 con 0x7ff50c105640 2024-05-08T05:47:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.785+0000 7ff50a000700 1 --2- 172.21.15.112:0/2984963886 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff4f00b2880 0x7ff4f00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:11.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.786+0000 7ff50a000700 1 --2- 172.21.15.112:0/2984963886 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff4f00b2880 0x7ff4f00b4d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009c60 tx=0x7ff4f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:11.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:11.790+0000 7ff503400700 1 -- 172.21.15.112:0/2984963886 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff4fc013070 con 0x7ff50c105640 2024-05-08T05:47:12.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:12.028+0000 7ff50be00700 1 -- 172.21.15.112:0/2984963886 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff50c0638e0 con 0x7ff4f00b2880 2024-05-08T05:47:12.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:12 smithi150 ceph-mon[34315]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:12.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:12 smithi150 ceph-mon[34315]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:47:12.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:47:12.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:47:12.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:12.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:12 smithi112 ceph-mon[30003]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:12.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:12 smithi112 ceph-mon[30003]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:47:12.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:12 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:47:12.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:12 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:47:12.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:12 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:12.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:12 smithi162 ceph-mon[34800]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:12.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:12 smithi162 ceph-mon[34800]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:47:12.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:12 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:47:12.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:12 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:47:12.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:12 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:14.524 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:14 smithi112 ceph-mon[30003]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:14.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:14 smithi150 ceph-mon[34315]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:14.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:14 smithi162 ceph-mon[34800]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:16.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:16 smithi150 ceph-mon[34315]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-08T05:47:16.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:16 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3373534015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0"}]: dispatch 2024-05-08T05:47:16.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:16 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3373534015' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0"}]': finished 2024-05-08T05:47:16.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:16 smithi150 ceph-mon[34315]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T05:47:16.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:16 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:47:16.762 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:16 smithi112 ceph-mon[30003]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-08T05:47:16.762 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:16 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3373534015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0"}]: dispatch 2024-05-08T05:47:16.762 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:16 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3373534015' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0"}]': finished 2024-05-08T05:47:16.762 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:16 smithi112 ceph-mon[30003]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T05:47:16.762 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:16 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:47:16.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:16 smithi162 ceph-mon[34800]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-08T05:47:16.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:16 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3373534015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0"}]: dispatch 2024-05-08T05:47:16.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:16 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3373534015' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0"}]': finished 2024-05-08T05:47:16.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:16 smithi162 ceph-mon[34800]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T05:47:16.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:16 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:47:17.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:17 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1920469279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:47:17.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:17 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1920469279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:47:17.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:17 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1920469279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:47:18.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:18 smithi112 ceph-mon[30003]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:47:18.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:18 smithi162 ceph-mon[34800]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:47:18.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:18 smithi150 ceph-mon[34315]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:47:20.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:20 smithi112 ceph-mon[30003]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T05:47:20.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:20 smithi162 ceph-mon[34800]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T05:47:20.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:20 smithi150 ceph-mon[34315]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T05:47:22.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:22 smithi162 ceph-mon[34800]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:22.819 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:22 smithi112 ceph-mon[30003]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:22.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:22 smithi150 ceph-mon[34315]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:24.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:24 smithi112 ceph-mon[30003]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:24.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:24 smithi162 ceph-mon[34800]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:24 smithi150 ceph-mon[34315]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:26.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:26 smithi112 ceph-mon[30003]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:26.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T05:47:26.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:26.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:26 smithi112 ceph-mon[30003]: Deploying daemon osd.6 on smithi112 2024-05-08T05:47:26.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:26 smithi162 ceph-mon[34800]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:26.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T05:47:26.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:26.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:26 smithi162 ceph-mon[34800]: Deploying daemon osd.6 on smithi112 2024-05-08T05:47:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:26 smithi150 ceph-mon[34315]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T05:47:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:26 smithi150 ceph-mon[34315]: Deploying daemon osd.6 on smithi112 2024-05-08T05:47:28.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:28 smithi112 ceph-mon[30003]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:28.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:28 smithi162 ceph-mon[34800]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:28.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:28 smithi150 ceph-mon[34315]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:30.745 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:30 smithi112 ceph-mon[30003]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:30.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:30 smithi162 ceph-mon[34800]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:30.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:30 smithi150 ceph-mon[34315]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:32.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:32 smithi112 ceph-mon[30003]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:32.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:32 smithi162 ceph-mon[34800]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:32.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:32 smithi150 ceph-mon[34315]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:34.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:34 smithi112 ceph-mon[30003]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:34.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:34.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:34.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:34.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:34 smithi150 ceph-mon[34315]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:34.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:34.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:34.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:35.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:34 smithi162 ceph-mon[34800]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:35.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:35.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:35.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:36.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:36 smithi150 ceph-mon[34315]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:37.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:36 smithi112 ceph-mon[30003]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:37.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:36 smithi162 ceph-mon[34800]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:38.281 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 6 on host 'smithi112' 2024-05-08T05:47:38.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:38.281+0000 7ff503400700 1 -- 172.21.15.112:0/2984963886 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff50c0638e0 con 0x7ff4f00b2880 2024-05-08T05:47:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:38.285+0000 7ff50be00700 1 -- 172.21.15.112:0/2984963886 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff4f00b2880 msgr2=0x7ff4f00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:38.285+0000 7ff50be00700 1 --2- 172.21.15.112:0/2984963886 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff4f00b2880 0x7ff4f00b4d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009c60 tx=0x7ff4f8009260 comp rx=0 tx=0).stop 2024-05-08T05:47:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:38.285+0000 7ff50be00700 1 -- 172.21.15.112:0/2984963886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c105640 msgr2=0x7ff50c19b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:38.285+0000 7ff50be00700 1 --2- 172.21.15.112:0/2984963886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c105640 0x7ff50c19b7e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00b580 tx=0x7ff4fc003af0 comp rx=0 tx=0).stop 2024-05-08T05:47:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:38.285+0000 7ff50be00700 1 -- 172.21.15.112:0/2984963886 shutdown_connections 2024-05-08T05:47:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:38.285+0000 7ff50be00700 1 --2- 172.21.15.112:0/2984963886 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff4f00b2880 0x7ff4f00b4d40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:38.285+0000 7ff50be00700 1 --2- 172.21.15.112:0/2984963886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff50c1071e0 0x7ff50c1a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:38.285+0000 7ff50be00700 1 --2- 172.21.15.112:0/2984963886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff50c105640 0x7ff50c19b7e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:38.285+0000 7ff50be00700 1 --2- 172.21.15.112:0/2984963886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff50c106840 0x7ff50c19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:38.285+0000 7ff50be00700 1 -- 172.21.15.112:0/2984963886 >> 172.21.15.112:0/2984963886 conn(0x7ff50c100be0 msgr2=0x7ff50c109bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:38.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:38 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:38.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:38 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:38.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:38 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:38.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:38 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:38.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:38 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:38.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:38 smithi112 ceph-mon[30003]: from='osd.6 [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T05:47:38.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:38 smithi112 ceph-mon[30003]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T05:47:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:38.285+0000 7ff50be00700 1 -- 172.21.15.112:0/2984963886 shutdown_connections 2024-05-08T05:47:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:47:38.286+0000 7ff50be00700 1 -- 172.21.15.112:0/2984963886 wait complete. 2024-05-08T05:47:38.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:38 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:38.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:38 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:38.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:38 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:38.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:38 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:38.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:38 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:38.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:38 smithi162 ceph-mon[34800]: from='osd.6 [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T05:47:38.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:38 smithi162 ceph-mon[34800]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T05:47:38.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:38 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:38.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:38 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:38.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:38 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:38.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:38 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:38.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:38 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:38.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:38 smithi150 ceph-mon[34315]: from='osd.6 [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T05:47:38.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:38 smithi150 ceph-mon[34315]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T05:47:39.090 DEBUG:teuthology.orchestra.run.smithi112:osd.6> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.6.service 2024-05-08T05:47:39.095 INFO:tasks.cephadm:Deploying osd.7 on smithi150 with /dev/vg_nvme/lv_2... 2024-05-08T05:47:39.096 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T05:47:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:39 smithi112 ceph-mon[30003]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:39 smithi112 ceph-mon[30003]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T05:47:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:39 smithi112 ceph-mon[30003]: from='osd.6 [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:47:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:39 smithi112 ceph-mon[30003]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T05:47:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:39 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:47:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:39 smithi112 ceph-mon[30003]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:47:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:39 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:39 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:39 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:39.341 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:47:39.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:39 smithi150 ceph-mon[34315]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:39.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:39 smithi150 ceph-mon[34315]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T05:47:39.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:39 smithi150 ceph-mon[34315]: from='osd.6 [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:47:39.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:39 smithi150 ceph-mon[34315]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T05:47:39.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:39 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:47:39.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:39 smithi150 ceph-mon[34315]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:47:39.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:39 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:39.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:39 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:39.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:39 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:39.570 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:39 smithi162 ceph-mon[34800]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:39.570 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:39 smithi162 ceph-mon[34800]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T05:47:39.570 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:39 smithi162 ceph-mon[34800]: from='osd.6 [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:47:39.570 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:39 smithi162 ceph-mon[34800]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T05:47:39.570 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:39 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:47:39.570 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:39 smithi162 ceph-mon[34800]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:47:39.570 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:39 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:39.571 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:39 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:39.571 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:39 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:40.365 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:40 smithi150 ceph-mon[34315]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-08T05:47:40.365 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:40 smithi150 ceph-mon[34315]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T05:47:40.365 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:40 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:47:40.365 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:40 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:47:40.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:40 smithi112 ceph-mon[30003]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-08T05:47:40.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:40 smithi112 ceph-mon[30003]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T05:47:40.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:40 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:47:40.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:40 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:47:40.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:40 smithi162 ceph-mon[34800]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-08T05:47:40.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:40 smithi162 ceph-mon[34800]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T05:47:40.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:40 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:47:40.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:40 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:47:41.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:41 smithi150 ceph-mon[34315]: purged_snaps scrub starts 2024-05-08T05:47:41.205 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:41 smithi150 ceph-mon[34315]: purged_snaps scrub ok 2024-05-08T05:47:41.205 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:41 smithi150 ceph-mon[34315]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:41.205 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:41 smithi150 ceph-mon[34315]: osd.6 [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186] boot 2024-05-08T05:47:41.205 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:41 smithi150 ceph-mon[34315]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T05:47:41.205 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:41 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:47:41.205 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T05:47:41.205 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T05:47:41.205 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-08T05:47:41.205 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-08T05:47:41.205 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123864 s, 847 MB/s 2024-05-08T05:47:41.206 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-08T05:47:41.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:41 smithi112 ceph-mon[30003]: purged_snaps scrub starts 2024-05-08T05:47:41.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:41 smithi112 ceph-mon[30003]: purged_snaps scrub ok 2024-05-08T05:47:41.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:41 smithi112 ceph-mon[30003]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:41.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:41 smithi112 ceph-mon[30003]: osd.6 [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186] boot 2024-05-08T05:47:41.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:41 smithi112 ceph-mon[30003]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T05:47:41.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:41 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:47:41.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:41 smithi162 ceph-mon[34800]: purged_snaps scrub starts 2024-05-08T05:47:41.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:41 smithi162 ceph-mon[34800]: purged_snaps scrub ok 2024-05-08T05:47:41.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:41 smithi162 ceph-mon[34800]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:47:41.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:41 smithi162 ceph-mon[34800]: osd.6 [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186] boot 2024-05-08T05:47:41.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:41 smithi162 ceph-mon[34800]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T05:47:41.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:41 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:47:41.841 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_2 2024-05-08T05:47:42.087 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:47:42.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:42 smithi112 ceph-mon[30003]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T05:47:42.358 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:42 smithi150 ceph-mon[34315]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T05:47:42.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:42 smithi162 ceph-mon[34800]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T05:47:43.359 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:43 smithi150 ceph-mon[34315]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:43.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:43 smithi112 ceph-mon[30003]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:43.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:43 smithi162 ceph-mon[34800]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:43.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.567+0000 7fd850e40700 1 -- 172.21.15.150:0/3224418317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd84c1047f0 msgr2=0x7fd84c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:43.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.567+0000 7fd850e40700 1 --2- 172.21.15.150:0/3224418317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd84c1047f0 0x7fd84c104c50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009990 tx=0x7fd83c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:47:43.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.568+0000 7fd850e40700 1 -- 172.21.15.150:0/3224418317 shutdown_connections 2024-05-08T05:47:43.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.568+0000 7fd850e40700 1 --2- 172.21.15.150:0/3224418317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd84c105190 0x7fd84c10ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.568+0000 7fd850e40700 1 --2- 172.21.15.150:0/3224418317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd84c101810 0x7fd84c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.568+0000 7fd850e40700 1 --2- 172.21.15.150:0/3224418317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd84c1047f0 0x7fd84c104c50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.568+0000 7fd850e40700 1 -- 172.21.15.150:0/3224418317 >> 172.21.15.150:0/3224418317 conn(0x7fd84c0fd170 msgr2=0x7fd84c0ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.568+0000 7fd850e40700 1 -- 172.21.15.150:0/3224418317 shutdown_connections 2024-05-08T05:47:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.568+0000 7fd850e40700 1 -- 172.21.15.150:0/3224418317 wait complete. 2024-05-08T05:47:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.568+0000 7fd850e40700 1 Processor -- start 2024-05-08T05:47:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.568+0000 7fd850e40700 1 -- start start 2024-05-08T05:47:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.568+0000 7fd850e40700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd84c101810 0x7fd84c1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd850e40700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd84c1047f0 0x7fd84c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd850e40700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd84c105190 0x7fd84c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd850e40700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c10f030 con 0x7fd84c105190 2024-05-08T05:47:43.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd850e40700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c10eeb0 con 0x7fd84c101810 2024-05-08T05:47:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd850e40700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c10f1b0 con 0x7fd84c1047f0 2024-05-08T05:47:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd84a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd84c101810 0x7fd84c1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd84a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd84c101810 0x7fd84c1995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:34322/0 (socket says 172.21.15.150:34322) 2024-05-08T05:47:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd849d9b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd84c1047f0 0x7fd84c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd84a59c700 1 -- 172.21.15.150:0/2236681146 learned_addr learned my addr 172.21.15.150:0/2236681146 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-08T05:47:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd84ad9d700 1 --2- 172.21.15.150:0/2236681146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd84c105190 0x7fd84c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd84a59c700 1 -- 172.21.15.150:0/2236681146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd84c1047f0 msgr2=0x7fd84c199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd84a59c700 1 --2- 172.21.15.150:0/2236681146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd84c1047f0 0x7fd84c199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:43.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd84a59c700 1 -- 172.21.15.150:0/2236681146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd84c105190 msgr2=0x7fd84c1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd84a59c700 1 --2- 172.21.15.150:0/2236681146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd84c105190 0x7fd84c1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd84a59c700 1 -- 172.21.15.150:0/2236681146 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84c1a1250 con 0x7fd84c101810 2024-05-08T05:47:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd849d9b700 1 --2- 172.21.15.150:0/2236681146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd84c1047f0 0x7fd84c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:47:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd84ad9d700 1 --2- 172.21.15.150:0/2236681146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd84c105190 0x7fd84c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd84a59c700 1 --2- 172.21.15.150:0/2236681146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd84c101810 0x7fd84c1995f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd84000c990 tx=0x7fd84000cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd83b7fe700 1 -- 172.21.15.150:0/2236681146 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840007b00 con 0x7fd84c101810 2024-05-08T05:47:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd83b7fe700 1 -- 172.21.15.150:0/2236681146 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd840007ca0 con 0x7fd84c101810 2024-05-08T05:47:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd83b7fe700 1 -- 172.21.15.150:0/2236681146 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840018660 con 0x7fd84c101810 2024-05-08T05:47:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd850e40700 1 -- 172.21.15.150:0/2236681146 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84c1a1540 con 0x7fd84c101810 2024-05-08T05:47:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.569+0000 7fd850e40700 1 -- 172.21.15.150:0/2236681146 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84c1a1ad0 con 0x7fd84c101810 2024-05-08T05:47:43.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.570+0000 7fd850e40700 1 -- 172.21.15.150:0/2236681146 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd84c06a190 con 0x7fd84c101810 2024-05-08T05:47:43.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.572+0000 7fd83b7fe700 1 -- 172.21.15.150:0/2236681146 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fd840004030 con 0x7fd84c101810 2024-05-08T05:47:43.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.572+0000 7fd83b7fe700 1 --2- 172.21.15.150:0/2236681146 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd8340b2900 0x7fd8340b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:43.574 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.573+0000 7fd83b7fe700 1 -- 172.21.15.150:0/2236681146 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7fd8400e14c0 con 0x7fd84c101810 2024-05-08T05:47:43.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.573+0000 7fd849d9b700 1 --2- 172.21.15.150:0/2236681146 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd8340b2900 0x7fd8340b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:43.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.574+0000 7fd849d9b700 1 --2- 172.21.15.150:0/2236681146 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd8340b2900 0x7fd8340b4dc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00b2e0 tx=0x7fd83c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:43.576 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.576+0000 7fd83b7fe700 1 -- 172.21.15.150:0/2236681146 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd840054df0 con 0x7fd84c101810 2024-05-08T05:47:43.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:47:43.788+0000 7fd850e40700 1 -- 172.21.15.150:0/2236681146 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd84c0638e0 con 0x7fd8340b2900 2024-05-08T05:47:44.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:47:44.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:47:44.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:44.407 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:47:44.407 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:47:44.407 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:44.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:47:44.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:47:44.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:45.229 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:45 smithi150 ceph-mon[34315]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:45.229 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:45 smithi150 ceph-mon[34315]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:47:45.229 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:45 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:45.230 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:45 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:45.230 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:45 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:45.230 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:45 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:45.230 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:45 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:45.230 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:45 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:45.230 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:45 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:45.230 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:45 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:45.230 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:45 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:45.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:45 smithi112 ceph-mon[30003]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:45.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:45 smithi112 ceph-mon[30003]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:47:45.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:45 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:45.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:45 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:45.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:45 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:45.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:45 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:45.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:45 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:45.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:45 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:45.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:45 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:45.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:45 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:45.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:45 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:45.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:45 smithi162 ceph-mon[34800]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:45.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:45 smithi162 ceph-mon[34800]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:47:45.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:45 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:45.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:45 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:45.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:45 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:45.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:45 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:45.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:45 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:45.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:45 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:45.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:45 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:45.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:45 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:45.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:45 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:46.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:46 smithi150 ceph-mon[34315]: Detected new or changed devices on smithi112 2024-05-08T05:47:46.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:46 smithi150 ceph-mon[34315]: Adjusting osd_memory_target on smithi112 to 5696M 2024-05-08T05:47:46.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:46 smithi112 ceph-mon[30003]: Detected new or changed devices on smithi112 2024-05-08T05:47:46.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:46 smithi112 ceph-mon[30003]: Adjusting osd_memory_target on smithi112 to 5696M 2024-05-08T05:47:46.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:46 smithi162 ceph-mon[34800]: Detected new or changed devices on smithi112 2024-05-08T05:47:46.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:46 smithi162 ceph-mon[34800]: Adjusting osd_memory_target on smithi112 to 5696M 2024-05-08T05:47:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:47 smithi150 ceph-mon[34315]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:47 smithi150 ceph-mon[34315]: from='client.? 172.21.15.150:0/1991254626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af3629b4-9bed-4669-97a6-48deba503a96"}]: dispatch 2024-05-08T05:47:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:47 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af3629b4-9bed-4669-97a6-48deba503a96"}]: dispatch 2024-05-08T05:47:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:47 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af3629b4-9bed-4669-97a6-48deba503a96"}]': finished 2024-05-08T05:47:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:47 smithi150 ceph-mon[34315]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T05:47:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:47 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:47:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:47 smithi150 ceph-mon[34315]: from='client.? 172.21.15.150:0/4030667230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:47:47.468 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:47 smithi162 ceph-mon[34800]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:47.468 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:47 smithi162 ceph-mon[34800]: from='client.? 172.21.15.150:0/1991254626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af3629b4-9bed-4669-97a6-48deba503a96"}]: dispatch 2024-05-08T05:47:47.468 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:47 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af3629b4-9bed-4669-97a6-48deba503a96"}]: dispatch 2024-05-08T05:47:47.468 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:47 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af3629b4-9bed-4669-97a6-48deba503a96"}]': finished 2024-05-08T05:47:47.468 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:47 smithi162 ceph-mon[34800]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T05:47:47.468 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:47 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:47:47.468 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:47 smithi162 ceph-mon[34800]: from='client.? 172.21.15.150:0/4030667230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:47:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:47 smithi112 ceph-mon[30003]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:47 smithi112 ceph-mon[30003]: from='client.? 172.21.15.150:0/1991254626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af3629b4-9bed-4669-97a6-48deba503a96"}]: dispatch 2024-05-08T05:47:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:47 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af3629b4-9bed-4669-97a6-48deba503a96"}]: dispatch 2024-05-08T05:47:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:47 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af3629b4-9bed-4669-97a6-48deba503a96"}]': finished 2024-05-08T05:47:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:47 smithi112 ceph-mon[30003]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T05:47:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:47 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:47:47.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:47 smithi112 ceph-mon[30003]: from='client.? 172.21.15.150:0/4030667230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:47:49.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:49 smithi150 ceph-mon[34315]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:49.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:49 smithi112 ceph-mon[30003]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:49.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:49 smithi162 ceph-mon[34800]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:51.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:51 smithi150 ceph-mon[34315]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:51.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:51 smithi112 ceph-mon[30003]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:51.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:51 smithi162 ceph-mon[34800]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:53.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:53 smithi150 ceph-mon[34315]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:53.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:53 smithi112 ceph-mon[30003]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:53.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:53 smithi162 ceph-mon[34800]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:54.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:54 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T05:47:54.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:54 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:54.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:54 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T05:47:54.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:54 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:54.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:54 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T05:47:54.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:54 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:55.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:55 smithi150 ceph-mon[34315]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:55.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:55 smithi150 ceph-mon[34315]: Deploying daemon osd.7 on smithi150 2024-05-08T05:47:55.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:55 smithi112 ceph-mon[30003]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:55.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:55 smithi112 ceph-mon[30003]: Deploying daemon osd.7 on smithi150 2024-05-08T05:47:55.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:55 smithi162 ceph-mon[34800]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:55.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:55 smithi162 ceph-mon[34800]: Deploying daemon osd.7 on smithi150 2024-05-08T05:47:57.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:57 smithi150 ceph-mon[34315]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:57.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:57 smithi112 ceph-mon[30003]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:57.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:57 smithi162 ceph-mon[34800]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:59.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:59 smithi150 ceph-mon[34315]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:59.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:59 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:59.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:59 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:59.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:47:59 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:59.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:59 smithi112 ceph-mon[30003]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:59.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:59 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:59.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:59 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:59.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:47:59 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:59.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:59 smithi162 ceph-mon[34800]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:59.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:59 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:59.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:59 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:47:59.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:47:59 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:01.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:01 smithi112 ceph-mon[30003]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:48:01.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:01 smithi162 ceph-mon[34800]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:48:01.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:01 smithi150 ceph-mon[34315]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:48:02.364 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 7 on host 'smithi150' 2024-05-08T05:48:02.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:48:02.363+0000 7fd83b7fe700 1 -- 172.21.15.150:0/2236681146 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd84c0638e0 con 0x7fd8340b2900 2024-05-08T05:48:02.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:48:02.368+0000 7fd850e40700 1 -- 172.21.15.150:0/2236681146 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd8340b2900 msgr2=0x7fd8340b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:02.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:48:02.368+0000 7fd850e40700 1 --2- 172.21.15.150:0/2236681146 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd8340b2900 0x7fd8340b4dc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00b2e0 tx=0x7fd83c01a040 comp rx=0 tx=0).stop 2024-05-08T05:48:02.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:48:02.368+0000 7fd850e40700 1 -- 172.21.15.150:0/2236681146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd84c101810 msgr2=0x7fd84c1995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:02.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:48:02.368+0000 7fd850e40700 1 --2- 172.21.15.150:0/2236681146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd84c101810 0x7fd84c1995f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd84000c990 tx=0x7fd84000cca0 comp rx=0 tx=0).stop 2024-05-08T05:48:02.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:48:02.368+0000 7fd850e40700 1 -- 172.21.15.150:0/2236681146 shutdown_connections 2024-05-08T05:48:02.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:48:02.368+0000 7fd850e40700 1 --2- 172.21.15.150:0/2236681146 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd8340b2900 0x7fd8340b4dc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:02.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:48:02.368+0000 7fd850e40700 1 --2- 172.21.15.150:0/2236681146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd84c105190 0x7fd84c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:02.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:48:02.368+0000 7fd850e40700 1 --2- 172.21.15.150:0/2236681146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd84c101810 0x7fd84c1995f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:02.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:48:02.368+0000 7fd850e40700 1 --2- 172.21.15.150:0/2236681146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd84c1047f0 0x7fd84c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:02.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:48:02.368+0000 7fd850e40700 1 -- 172.21.15.150:0/2236681146 >> 172.21.15.150:0/2236681146 conn(0x7fd84c0fd170 msgr2=0x7fd84c107b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:02.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:48:02.368+0000 7fd850e40700 1 -- 172.21.15.150:0/2236681146 shutdown_connections 2024-05-08T05:48:02.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:48:02.368+0000 7fd850e40700 1 -- 172.21.15.150:0/2236681146 wait complete. 2024-05-08T05:48:03.006 DEBUG:teuthology.orchestra.run.smithi150:osd.7> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.7.service 2024-05-08T05:48:03.008 INFO:tasks.cephadm:Deploying osd.8 on smithi162 with /dev/vg_nvme/lv_2... 2024-05-08T05:48:03.008 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T05:48:03.255 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:48:03.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:03 smithi162 ceph-mon[34800]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:48:03.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:03 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:03.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:03 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:03.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:03 smithi162 ceph-mon[34800]: from='osd.7 [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:48:03.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:03 smithi162 ceph-mon[34800]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:48:03.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:03 smithi150 ceph-mon[34315]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:48:03.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:03 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:03.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:03 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:03.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:03 smithi150 ceph-mon[34315]: from='osd.7 [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:48:03.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:03 smithi150 ceph-mon[34315]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:48:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:03 smithi112 ceph-mon[30003]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:48:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:03 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:03 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:03 smithi112 ceph-mon[30003]: from='osd.7 [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:48:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:03 smithi112 ceph-mon[30003]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:48:04.433 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 05:48:04 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T05:48:04.360+0000 7f2b41094700 -1 osd.7 0 waiting for initial osdmap 2024-05-08T05:48:04.433 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 05:48:04 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T05:48:04.370+0000 7f2b3c686700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:48:04.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:04.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:04.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:04 smithi150 ceph-mon[34315]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T05:48:04.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:04 smithi150 ceph-mon[34315]: from='osd.7 [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:48:04.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:04 smithi150 ceph-mon[34315]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T05:48:04.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:48:04.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:04 smithi150 ceph-mon[34315]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:48:04.468 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:04.468 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:04.468 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:04 smithi112 ceph-mon[30003]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T05:48:04.468 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:04 smithi112 ceph-mon[30003]: from='osd.7 [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:48:04.468 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:04 smithi112 ceph-mon[30003]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T05:48:04.468 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:48:04.468 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:04 smithi112 ceph-mon[30003]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:48:04.490 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:04.490 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:04.490 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:04 smithi162 ceph-mon[34800]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T05:48:04.490 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:04 smithi162 ceph-mon[34800]: from='osd.7 [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:48:04.490 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:04 smithi162 ceph-mon[34800]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T05:48:04.490 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:48:04.490 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:04 smithi162 ceph-mon[34800]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:48:05.040 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T05:48:05.040 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T05:48:05.040 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2024-05-08T05:48:05.040 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2024-05-08T05:48:05.040 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115143 s, 911 MB/s 2024-05-08T05:48:05.040 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2024-05-08T05:48:05.336 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:05 smithi162 ceph-mon[34800]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:48:05.336 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:05 smithi162 ceph-mon[34800]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-08T05:48:05.336 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:05 smithi162 ceph-mon[34800]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T05:48:05.337 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:05 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:48:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:05 smithi150 ceph-mon[34315]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:48:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:05 smithi150 ceph-mon[34315]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-08T05:48:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:05 smithi150 ceph-mon[34315]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T05:48:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:48:05.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:05 smithi112 ceph-mon[30003]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:48:05.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:05 smithi112 ceph-mon[30003]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-08T05:48:05.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:05 smithi112 ceph-mon[30003]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T05:48:05.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:05 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:48:05.698 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_2 2024-05-08T05:48:05.939 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:48:06.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:06 smithi150 ceph-mon[34315]: purged_snaps scrub starts 2024-05-08T05:48:06.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:06 smithi150 ceph-mon[34315]: purged_snaps scrub ok 2024-05-08T05:48:06.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:06 smithi150 ceph-mon[34315]: osd.7 [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564] boot 2024-05-08T05:48:06.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:06 smithi150 ceph-mon[34315]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T05:48:06.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:06 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:48:06.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:06 smithi150 ceph-mon[34315]: pgmap v189: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:06.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:06 smithi112 ceph-mon[30003]: purged_snaps scrub starts 2024-05-08T05:48:06.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:06 smithi112 ceph-mon[30003]: purged_snaps scrub ok 2024-05-08T05:48:06.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:06 smithi112 ceph-mon[30003]: osd.7 [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564] boot 2024-05-08T05:48:06.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:06 smithi112 ceph-mon[30003]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T05:48:06.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:06 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:48:06.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:06 smithi112 ceph-mon[30003]: pgmap v189: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:06.717 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:06 smithi162 ceph-mon[34800]: purged_snaps scrub starts 2024-05-08T05:48:06.717 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:06 smithi162 ceph-mon[34800]: purged_snaps scrub ok 2024-05-08T05:48:06.717 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:06 smithi162 ceph-mon[34800]: osd.7 [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564] boot 2024-05-08T05:48:06.717 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:06 smithi162 ceph-mon[34800]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T05:48:06.717 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:06 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:48:06.718 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:06 smithi162 ceph-mon[34800]: pgmap v189: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:07.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.499+0000 7fa33d83d700 1 -- 172.21.15.162:0/1631647980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3381050a0 msgr2=0x7fa33810b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:07.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.499+0000 7fa33d83d700 1 --2- 172.21.15.162:0/1631647980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3381050a0 0x7fa33810b950 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa33000b0a0 tx=0x7fa33000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:48:07.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.499+0000 7fa33d83d700 1 -- 172.21.15.162:0/1631647980 shutdown_connections 2024-05-08T05:48:07.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.499+0000 7fa33d83d700 1 --2- 172.21.15.162:0/1631647980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3381050a0 0x7fa33810b950 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:07.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.499+0000 7fa33d83d700 1 --2- 172.21.15.162:0/1631647980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa338103500 0x7fa338103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:07.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.499+0000 7fa33d83d700 1 --2- 172.21.15.162:0/1631647980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa338104700 0x7fa338104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:07.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.499+0000 7fa33d83d700 1 -- 172.21.15.162:0/1631647980 >> 172.21.15.162:0/1631647980 conn(0x7fa3380feaa0 msgr2=0x7fa338100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:07.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.499+0000 7fa33d83d700 1 -- 172.21.15.162:0/1631647980 shutdown_connections 2024-05-08T05:48:07.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.499+0000 7fa33d83d700 1 -- 172.21.15.162:0/1631647980 wait complete. 2024-05-08T05:48:07.502 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.500+0000 7fa33d83d700 1 Processor -- start 2024-05-08T05:48:07.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.500+0000 7fa33d83d700 1 -- start start 2024-05-08T05:48:07.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.500+0000 7fa33d83d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa338103500 0x7fa338199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:07.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.500+0000 7fa33d83d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa338104700 0x7fa338199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:07.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.500+0000 7fa33d83d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3381050a0 0x7fa3381a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:07.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.500+0000 7fa33d83d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa33810f090 con 0x7fa338104700 2024-05-08T05:48:07.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.500+0000 7fa33d83d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa33810ef10 con 0x7fa3381050a0 2024-05-08T05:48:07.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.500+0000 7fa33d83d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa33810f210 con 0x7fa338103500 2024-05-08T05:48:07.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.500+0000 7fa336ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa338103500 0x7fa338199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:07.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.500+0000 7fa336ffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa338103500 0x7fa338199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:36390/0 (socket says 172.21.15.162:36390) 2024-05-08T05:48:07.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.500+0000 7fa336ffd700 1 -- 172.21.15.162:0/1153505621 learned_addr learned my addr 172.21.15.162:0/1153505621 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-08T05:48:07.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.501+0000 7fa3377fe700 1 --2- 172.21.15.162:0/1153505621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3381050a0 0x7fa3381a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:07.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.501+0000 7fa336ffd700 1 -- 172.21.15.162:0/1153505621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3381050a0 msgr2=0x7fa3381a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:07.503 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.501+0000 7fa336ffd700 1 --2- 172.21.15.162:0/1153505621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3381050a0 0x7fa3381a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:07.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.501+0000 7fa336ffd700 1 -- 172.21.15.162:0/1153505621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa338104700 msgr2=0x7fa338199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:07.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.501+0000 7fa336ffd700 1 --2- 172.21.15.162:0/1153505621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa338104700 0x7fa338199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:07.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.501+0000 7fa336ffd700 1 -- 172.21.15.162:0/1153505621 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3381a13f0 con 0x7fa338103500 2024-05-08T05:48:07.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.501+0000 7fa336ffd700 1 --2- 172.21.15.162:0/1153505621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa338103500 0x7fa338199790 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa32800b530 tx=0x7fa32800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:07.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.501+0000 7fa33c83b700 1 -- 172.21.15.162:0/1153505621 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328015020 con 0x7fa338103500 2024-05-08T05:48:07.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.501+0000 7fa33d83d700 1 -- 172.21.15.162:0/1153505621 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3381a16e0 con 0x7fa338103500 2024-05-08T05:48:07.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.501+0000 7fa33c83b700 1 -- 172.21.15.162:0/1153505621 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa328011ce0 con 0x7fa338103500 2024-05-08T05:48:07.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.501+0000 7fa33c83b700 1 -- 172.21.15.162:0/1153505621 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328010d60 con 0x7fa338103500 2024-05-08T05:48:07.504 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.501+0000 7fa33d83d700 1 -- 172.21.15.162:0/1153505621 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3381a1ca0 con 0x7fa338103500 2024-05-08T05:48:07.505 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.502+0000 7fa33d83d700 1 -- 172.21.15.162:0/1153505621 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33806a190 con 0x7fa338103500 2024-05-08T05:48:07.505 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.503+0000 7fa33c83b700 1 -- 172.21.15.162:0/1153505621 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fa328018020 con 0x7fa338103500 2024-05-08T05:48:07.505 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.504+0000 7fa33c83b700 1 --2- 172.21.15.162:0/1153505621 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3200b2880 0x7fa3200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:07.505 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.504+0000 7fa33c83b700 1 -- 172.21.15.162:0/1153505621 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7fa3280e0ec0 con 0x7fa338103500 2024-05-08T05:48:07.505 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.504+0000 7fa3367fc700 1 --2- 172.21.15.162:0/1153505621 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3200b2880 0x7fa3200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:07.505 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.505+0000 7fa3367fc700 1 --2- 172.21.15.162:0/1153505621 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3200b2880 0x7fa3200b4d40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0097b0 tx=0x7fa32c006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:07.508 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.508+0000 7fa33c83b700 1 -- 172.21.15.162:0/1153505621 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa32801f020 con 0x7fa338103500 2024-05-08T05:48:07.554 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T05:48:07.554 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: Detected new or changed devices on smithi150 2024-05-08T05:48:07.554 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: Adjusting osd_memory_target on smithi150 to 5696M 2024-05-08T05:48:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:07.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T05:48:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: Detected new or changed devices on smithi150 2024-05-08T05:48:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:07.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:07.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:07.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:07.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: Adjusting osd_memory_target on smithi150 to 5696M 2024-05-08T05:48:07.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:07.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:07.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:07.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:07.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:07.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:07.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:07.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:07.720 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:07.720+0000 7fa33d83d700 1 -- 172.21.15.162:0/1153505621 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa3380638e0 con 0x7fa3200b2880 2024-05-08T05:48:07.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T05:48:07.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: Detected new or changed devices on smithi150 2024-05-08T05:48:07.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:07.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:07.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:07.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:07.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:07.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: Adjusting osd_memory_target on smithi150 to 5696M 2024-05-08T05:48:07.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:07.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:07.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:07.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:07.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:07.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:07.818 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:07.818 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:08.632 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:08 smithi162 ceph-mon[34800]: osdmap e50: 8 total, 8 up, 8 in 2024-05-08T05:48:08.632 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:08 smithi162 ceph-mon[34800]: pgmap v192: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:08.633 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:08 smithi162 ceph-mon[34800]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:48:08.633 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:48:08.633 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:48:08.633 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:08 smithi150 ceph-mon[34315]: osdmap e50: 8 total, 8 up, 8 in 2024-05-08T05:48:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:08 smithi150 ceph-mon[34315]: pgmap v192: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:08 smithi150 ceph-mon[34315]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:48:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:48:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:48:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:08.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:08 smithi112 ceph-mon[30003]: osdmap e50: 8 total, 8 up, 8 in 2024-05-08T05:48:08.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:08 smithi112 ceph-mon[30003]: pgmap v192: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:08.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:08 smithi112 ceph-mon[30003]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:48:08.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:48:08.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:48:08.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:09.491 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:09 smithi162 ceph-mon[34800]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-08T05:48:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:09 smithi150 ceph-mon[34315]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-08T05:48:09.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:09 smithi112 ceph-mon[30003]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-08T05:48:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:10 smithi150 ceph-mon[34315]: pgmap v193: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:10 smithi150 ceph-mon[34315]: from='client.? 172.21.15.162:0/1052564445' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9333e87d-dd78-4467-bbde-c2b44f4ea07f"}]: dispatch 2024-05-08T05:48:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:10 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9333e87d-dd78-4467-bbde-c2b44f4ea07f"}]: dispatch 2024-05-08T05:48:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:10 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9333e87d-dd78-4467-bbde-c2b44f4ea07f"}]': finished 2024-05-08T05:48:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:10 smithi150 ceph-mon[34315]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T05:48:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:10 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:10.700 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:10 smithi162 ceph-mon[34800]: pgmap v193: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:10.700 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:10 smithi162 ceph-mon[34800]: from='client.? 172.21.15.162:0/1052564445' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9333e87d-dd78-4467-bbde-c2b44f4ea07f"}]: dispatch 2024-05-08T05:48:10.700 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:10 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9333e87d-dd78-4467-bbde-c2b44f4ea07f"}]: dispatch 2024-05-08T05:48:10.700 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:10 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9333e87d-dd78-4467-bbde-c2b44f4ea07f"}]': finished 2024-05-08T05:48:10.700 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:10 smithi162 ceph-mon[34800]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T05:48:10.701 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:10 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:10.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:10 smithi112 ceph-mon[30003]: pgmap v193: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:10.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:10 smithi112 ceph-mon[30003]: from='client.? 172.21.15.162:0/1052564445' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9333e87d-dd78-4467-bbde-c2b44f4ea07f"}]: dispatch 2024-05-08T05:48:10.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:10 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9333e87d-dd78-4467-bbde-c2b44f4ea07f"}]: dispatch 2024-05-08T05:48:10.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:10 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9333e87d-dd78-4467-bbde-c2b44f4ea07f"}]': finished 2024-05-08T05:48:10.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:10 smithi112 ceph-mon[30003]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T05:48:10.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:11 smithi150 ceph-mon[34315]: from='client.? 172.21.15.162:0/1553843562' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:48:11.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:11 smithi112 ceph-mon[30003]: from='client.? 172.21.15.162:0/1553843562' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:48:11.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:11 smithi162 ceph-mon[34800]: from='client.? 172.21.15.162:0/1553843562' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:48:12.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:12 smithi150 ceph-mon[34315]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-05-08T05:48:12.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:12 smithi150 ceph-mon[34315]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-08T05:48:12.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:12 smithi150 ceph-mon[34315]: Cluster is now healthy 2024-05-08T05:48:12.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:12 smithi112 ceph-mon[30003]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-05-08T05:48:12.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:12 smithi112 ceph-mon[30003]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-08T05:48:12.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:12 smithi112 ceph-mon[30003]: Cluster is now healthy 2024-05-08T05:48:12.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:12 smithi162 ceph-mon[34800]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-05-08T05:48:12.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:12 smithi162 ceph-mon[34800]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-08T05:48:12.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:12 smithi162 ceph-mon[34800]: Cluster is now healthy 2024-05-08T05:48:14.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:14 smithi162 ceph-mon[34800]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 164 KiB/s, 0 objects/s recovering 2024-05-08T05:48:14.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:14 smithi150 ceph-mon[34315]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 164 KiB/s, 0 objects/s recovering 2024-05-08T05:48:14.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:14 smithi112 ceph-mon[30003]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 164 KiB/s, 0 objects/s recovering 2024-05-08T05:48:16.676 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:16 smithi162 ceph-mon[34800]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-05-08T05:48:16.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:16 smithi150 ceph-mon[34315]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-05-08T05:48:16.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:16 smithi112 ceph-mon[30003]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-05-08T05:48:18.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:18 smithi162 ceph-mon[34800]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T05:48:18.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:18 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T05:48:18.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:18 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:18.555 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:18 smithi162 ceph-mon[34800]: Deploying daemon osd.8 on smithi162 2024-05-08T05:48:18.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:18 smithi150 ceph-mon[34315]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T05:48:18.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:18 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T05:48:18.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:18 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:18.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:18 smithi150 ceph-mon[34315]: Deploying daemon osd.8 on smithi162 2024-05-08T05:48:18.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:18 smithi112 ceph-mon[30003]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T05:48:18.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T05:48:18.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:18.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:18 smithi112 ceph-mon[30003]: Deploying daemon osd.8 on smithi162 2024-05-08T05:48:20.528 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:20 smithi162 ceph-mon[34800]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T05:48:20.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:20 smithi150 ceph-mon[34315]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T05:48:20.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:20 smithi112 ceph-mon[30003]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T05:48:22.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:22 smithi162 ceph-mon[34800]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-05-08T05:48:22.686 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:22 smithi150 ceph-mon[34315]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-05-08T05:48:22.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:22 smithi112 ceph-mon[30003]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-05-08T05:48:24.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:24.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:24.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:24.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:24.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:24.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:24.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:24.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:24.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:25.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:25 smithi162 ceph-mon[34800]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:25.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:25 smithi150 ceph-mon[34315]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:25.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:25 smithi112 ceph-mon[30003]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:26.752 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:26.751+0000 7fa33c83b700 1 -- 172.21.15.162:0/1153505621 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa3380638e0 con 0x7fa3200b2880 2024-05-08T05:48:26.753 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 8 on host 'smithi162' 2024-05-08T05:48:26.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:26.755+0000 7fa33d83d700 1 -- 172.21.15.162:0/1153505621 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3200b2880 msgr2=0x7fa3200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:26.755+0000 7fa33d83d700 1 --2- 172.21.15.162:0/1153505621 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3200b2880 0x7fa3200b4d40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0097b0 tx=0x7fa32c006d20 comp rx=0 tx=0).stop 2024-05-08T05:48:26.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:26.755+0000 7fa33d83d700 1 -- 172.21.15.162:0/1153505621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa338103500 msgr2=0x7fa338199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:26.755+0000 7fa33d83d700 1 --2- 172.21.15.162:0/1153505621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa338103500 0x7fa338199790 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa32800b530 tx=0x7fa32800b8f0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:26.755+0000 7fa33d83d700 1 -- 172.21.15.162:0/1153505621 shutdown_connections 2024-05-08T05:48:26.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:26.755+0000 7fa33d83d700 1 --2- 172.21.15.162:0/1153505621 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3200b2880 0x7fa3200b4d40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:26.755+0000 7fa33d83d700 1 --2- 172.21.15.162:0/1153505621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3381050a0 0x7fa3381a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:26.755+0000 7fa33d83d700 1 --2- 172.21.15.162:0/1153505621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa338103500 0x7fa338199790 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:26.755+0000 7fa33d83d700 1 --2- 172.21.15.162:0/1153505621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa338104700 0x7fa338199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:26.755+0000 7fa33d83d700 1 -- 172.21.15.162:0/1153505621 >> 172.21.15.162:0/1153505621 conn(0x7fa3380feaa0 msgr2=0x7fa338107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:26.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:26.755+0000 7fa33d83d700 1 -- 172.21.15.162:0/1153505621 shutdown_connections 2024-05-08T05:48:26.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:48:26.755+0000 7fa33d83d700 1 -- 172.21.15.162:0/1153505621 wait complete. 2024-05-08T05:48:27.046 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:26 smithi162 ceph-mon[34800]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:27.046 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:27.046 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:27.046 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:27.046 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:27.047 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:27.047 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:27.047 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:27.047 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:27.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:26 smithi112 ceph-mon[30003]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:27.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:27.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:27.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:27.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:27.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:27.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:27.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:27.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:26 smithi150 ceph-mon[34315]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:27.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:27.352 DEBUG:teuthology.orchestra.run.smithi162:osd.8> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.8.service 2024-05-08T05:48:27.354 INFO:tasks.cephadm:Deploying osd.9 on smithi112 with /dev/vg_nvme/lv_1... 2024-05-08T05:48:27.354 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T05:48:27.797 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:48:28.000 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:27 smithi162 ceph-mon[34800]: from='osd.8 [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:48:28.000 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:27 smithi162 ceph-mon[34800]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:48:28.102 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:27 smithi112 ceph-mon[30003]: from='osd.8 [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:48:28.102 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:27 smithi112 ceph-mon[30003]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:48:28.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:27 smithi150 ceph-mon[34315]: from='osd.8 [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:48:28.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:27 smithi150 ceph-mon[34315]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:48:29.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:28 smithi112 ceph-mon[30003]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:29.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:28 smithi112 ceph-mon[30003]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T05:48:29.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:28 smithi112 ceph-mon[30003]: from='osd.8 [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:48:29.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:28 smithi112 ceph-mon[30003]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T05:48:29.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:28 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:29.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:28 smithi112 ceph-mon[30003]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:48:29.187 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:28 smithi150 ceph-mon[34315]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:29.187 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:28 smithi150 ceph-mon[34315]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T05:48:29.187 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:28 smithi150 ceph-mon[34315]: from='osd.8 [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:48:29.187 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:28 smithi150 ceph-mon[34315]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T05:48:29.187 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:28 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:29.187 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:28 smithi150 ceph-mon[34315]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:48:29.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:28 smithi162 ceph-mon[34800]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:29.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:28 smithi162 ceph-mon[34800]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T05:48:29.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:28 smithi162 ceph-mon[34800]: from='osd.8 [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:48:29.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:28 smithi162 ceph-mon[34800]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T05:48:29.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:28 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:29.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:28 smithi162 ceph-mon[34800]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:48:29.816 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 05:48:29 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T05:48:29.756+0000 7f51c4c96700 -1 osd.8 0 waiting for initial osdmap 2024-05-08T05:48:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:29 smithi112 ceph-mon[30003]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-05-08T05:48:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:29 smithi112 ceph-mon[30003]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T05:48:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:29 smithi112 ceph-mon[30003]: from='osd.8 ' entity='osd.8' 2024-05-08T05:48:30.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:29 smithi112 ceph-mon[30003]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T05:48:30.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:30.123 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 05:48:29 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T05:48:29.920+0000 7f51bca81700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:48:30.123 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:29 smithi162 ceph-mon[34800]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-05-08T05:48:30.123 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:29 smithi162 ceph-mon[34800]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T05:48:30.124 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:30.124 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:30.124 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:29 smithi162 ceph-mon[34800]: from='osd.8 ' entity='osd.8' 2024-05-08T05:48:30.124 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:29 smithi162 ceph-mon[34800]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T05:48:30.124 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:29 smithi150 ceph-mon[34315]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-05-08T05:48:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:29 smithi150 ceph-mon[34315]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T05:48:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:29 smithi150 ceph-mon[34315]: from='osd.8 ' entity='osd.8' 2024-05-08T05:48:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:29 smithi150 ceph-mon[34315]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T05:48:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:30.611 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T05:48:30.611 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T05:48:30.611 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-05-08T05:48:30.611 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-05-08T05:48:30.611 INFO:teuthology.orchestra.run.smithi112.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0321899 s, 326 MB/s 2024-05-08T05:48:30.611 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 2024-05-08T05:48:30.612 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-05-08T05:48:30.927 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: purged_snaps scrub starts 2024-05-08T05:48:30.927 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: purged_snaps scrub ok 2024-05-08T05:48:30.927 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:30.928 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: Detected new or changed devices on smithi162 2024-05-08T05:48:30.928 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:30.928 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:30.928 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:30.928 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:30.928 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:30.928 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: Adjusting osd_memory_target on smithi162 to 5696M 2024-05-08T05:48:30.928 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:30.928 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:30.928 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:30.928 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:30.928 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: osd.8 [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606] boot 2024-05-08T05:48:30.928 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T05:48:30.928 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:31.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: purged_snaps scrub starts 2024-05-08T05:48:31.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: purged_snaps scrub ok 2024-05-08T05:48:31.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:31.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: Detected new or changed devices on smithi162 2024-05-08T05:48:31.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:31.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:31.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:31.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:31.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:31.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: Adjusting osd_memory_target on smithi162 to 5696M 2024-05-08T05:48:31.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:31.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:31.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:31.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:31.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: osd.8 [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606] boot 2024-05-08T05:48:31.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T05:48:31.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:30 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:31.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: purged_snaps scrub starts 2024-05-08T05:48:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: purged_snaps scrub ok 2024-05-08T05:48:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:48:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: Detected new or changed devices on smithi162 2024-05-08T05:48:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: Adjusting osd_memory_target on smithi162 to 5696M 2024-05-08T05:48:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:31.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:31.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: osd.8 [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606] boot 2024-05-08T05:48:31.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T05:48:31.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:30 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:48:31.541 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_1 2024-05-08T05:48:31.955 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:48:33.161 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:32 smithi112 ceph-mon[30003]: pgmap v209: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:33.161 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:32 smithi112 ceph-mon[30003]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T05:48:33.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:32 smithi150 ceph-mon[34315]: pgmap v209: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:33.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:32 smithi150 ceph-mon[34315]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T05:48:33.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:32 smithi162 ceph-mon[34800]: pgmap v209: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:33.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:32 smithi162 ceph-mon[34800]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T05:48:33.969 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:33 smithi112 ceph-mon[30003]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T05:48:34.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.145+0000 7f7708c00700 1 -- 172.21.15.112:0/4148568086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7704104710 msgr2=0x7f7704104b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:34.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.145+0000 7f7708c00700 1 --2- 172.21.15.112:0/4148568086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7704104710 0x7f7704104b70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f76f00099c0 tx=0x7f76f0009cd0 comp rx=0 tx=0).stop 2024-05-08T05:48:34.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.145+0000 7f7708c00700 1 -- 172.21.15.112:0/4148568086 shutdown_connections 2024-05-08T05:48:34.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.145+0000 7f7708c00700 1 --2- 172.21.15.112:0/4148568086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77041050b0 0x7f770410b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:34.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.145+0000 7f7708c00700 1 --2- 172.21.15.112:0/4148568086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7704103510 0x7f7704103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:34.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.145+0000 7f7708c00700 1 --2- 172.21.15.112:0/4148568086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7704104710 0x7f7704104b70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:34.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.145+0000 7f7708c00700 1 -- 172.21.15.112:0/4148568086 >> 172.21.15.112:0/4148568086 conn(0x7f77040fea70 msgr2=0x7f7704100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:34.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.145+0000 7f7708c00700 1 -- 172.21.15.112:0/4148568086 shutdown_connections 2024-05-08T05:48:34.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.145+0000 7f7708c00700 1 -- 172.21.15.112:0/4148568086 wait complete. 2024-05-08T05:48:34.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.146+0000 7f7708c00700 1 Processor -- start 2024-05-08T05:48:34.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.146+0000 7f7708c00700 1 -- start start 2024-05-08T05:48:34.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.147+0000 7f7708c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7704103510 0x7f7704077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:34.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.147+0000 7f7708c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7704104710 0x7f7704075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.147+0000 7f7708c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77041050b0 0x7f7704075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.147+0000 7f7708c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f770410ef90 con 0x7f77041050b0 2024-05-08T05:48:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.147+0000 7f7708c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f770410ee10 con 0x7f7704104710 2024-05-08T05:48:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.147+0000 7f7708c00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f770410f110 con 0x7f7704103510 2024-05-08T05:48:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.147+0000 7f7702800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77041050b0 0x7f7704075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.147+0000 7f7701e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7704103510 0x7f7704077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.147+0000 7f7702800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77041050b0 0x7f7704075dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49402/0 (socket says 172.21.15.112:49402) 2024-05-08T05:48:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.147+0000 7f7702800700 1 -- 172.21.15.112:0/2154455139 learned_addr learned my addr 172.21.15.112:0/2154455139 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:48:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.147+0000 7f7701400700 1 --2- 172.21.15.112:0/2154455139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7704104710 0x7f7704075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.148+0000 7f7701e00700 1 -- 172.21.15.112:0/2154455139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7704104710 msgr2=0x7f7704075890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.148+0000 7f7701e00700 1 --2- 172.21.15.112:0/2154455139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7704104710 0x7f7704075890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.148+0000 7f7701e00700 1 -- 172.21.15.112:0/2154455139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77041050b0 msgr2=0x7f7704075dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.148+0000 7f7701e00700 1 --2- 172.21.15.112:0/2154455139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77041050b0 0x7f7704075dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.148+0000 7f7701e00700 1 -- 172.21.15.112:0/2154455139 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7704076680 con 0x7f7704103510 2024-05-08T05:48:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.148+0000 7f7701400700 1 --2- 172.21.15.112:0/2154455139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7704104710 0x7f7704075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:48:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.148+0000 7f7701e00700 1 --2- 172.21.15.112:0/2154455139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7704103510 0x7f7704077240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f76f400e9c0 tx=0x7f76f400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.148+0000 7f76faa00700 1 -- 172.21.15.112:0/2154455139 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f400cb70 con 0x7f7704103510 2024-05-08T05:48:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.148+0000 7f76faa00700 1 -- 172.21.15.112:0/2154455139 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76f400ee60 con 0x7f7704103510 2024-05-08T05:48:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.148+0000 7f76faa00700 1 -- 172.21.15.112:0/2154455139 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4018680 con 0x7f7704103510 2024-05-08T05:48:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.148+0000 7f7708c00700 1 -- 172.21.15.112:0/2154455139 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77041a37a0 con 0x7f7704103510 2024-05-08T05:48:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.148+0000 7f7708c00700 1 -- 172.21.15.112:0/2154455139 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77041a3c60 con 0x7f7704103510 2024-05-08T05:48:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.150+0000 7f7708c00700 1 -- 172.21.15.112:0/2154455139 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770406a190 con 0x7f7704103510 2024-05-08T05:48:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.152+0000 7f76faa00700 1 -- 172.21.15.112:0/2154455139 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f76f401f020 con 0x7f7704103510 2024-05-08T05:48:34.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.153+0000 7f76faa00700 1 --2- 172.21.15.112:0/2154455139 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f76e80b2900 0x7f76e80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:34.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.153+0000 7f76faa00700 1 -- 172.21.15.112:0/2154455139 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5829+0+0 (secure 0 0 0) 0x7f76f40e1680 con 0x7f7704103510 2024-05-08T05:48:34.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.153+0000 7f7701400700 1 --2- 172.21.15.112:0/2154455139 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f76e80b2900 0x7f76e80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:34.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.154+0000 7f7701400700 1 --2- 172.21.15.112:0/2154455139 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f76e80b2900 0x7f76e80b4dc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f76f0000c00 tx=0x7f76f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:34.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.158+0000 7f76faa00700 1 -- 172.21.15.112:0/2154455139 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f76f40a9cb0 con 0x7f7704103510 2024-05-08T05:48:34.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:33 smithi150 ceph-mon[34315]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T05:48:34.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:33 smithi162 ceph-mon[34800]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T05:48:34.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:48:34.409+0000 7f7708c00700 1 -- 172.21.15.112:0/2154455139 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f77040638e0 con 0x7f76e80b2900 2024-05-08T05:48:35.138 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:34 smithi150 ceph-mon[34315]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:35.138 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:48:35.138 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:48:35.138 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:35.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:34 smithi112 ceph-mon[30003]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:35.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:48:35.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:48:35.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:35.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:34 smithi162 ceph-mon[34800]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:35.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:48:35.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:48:35.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:36.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:35 smithi112 ceph-mon[30003]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:48:36.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:35 smithi150 ceph-mon[34315]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:48:36.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:35 smithi162 ceph-mon[34800]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:48:37.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:36 smithi112 ceph-mon[30003]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-08T05:48:37.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:36 smithi150 ceph-mon[34315]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-08T05:48:37.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:36 smithi162 ceph-mon[34800]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-08T05:48:38.932 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:38 smithi112 ceph-mon[30003]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-08T05:48:38.932 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:38 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1220895018' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09723581-1995-42e0-8db7-776ea98a918c"}]: dispatch 2024-05-08T05:48:38.932 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:38 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1220895018' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09723581-1995-42e0-8db7-776ea98a918c"}]': finished 2024-05-08T05:48:38.932 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:38 smithi112 ceph-mon[30003]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T05:48:38.932 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:38 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:39.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:38 smithi150 ceph-mon[34315]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-08T05:48:39.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:38 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1220895018' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09723581-1995-42e0-8db7-776ea98a918c"}]: dispatch 2024-05-08T05:48:39.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:38 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1220895018' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09723581-1995-42e0-8db7-776ea98a918c"}]': finished 2024-05-08T05:48:39.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:38 smithi150 ceph-mon[34315]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T05:48:39.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:38 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:39.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:38 smithi162 ceph-mon[34800]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-08T05:48:39.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:38 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1220895018' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09723581-1995-42e0-8db7-776ea98a918c"}]: dispatch 2024-05-08T05:48:39.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:38 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1220895018' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09723581-1995-42e0-8db7-776ea98a918c"}]': finished 2024-05-08T05:48:39.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:38 smithi162 ceph-mon[34800]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T05:48:39.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:38 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:40.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:39 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/248707600' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:48:40.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:39 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/248707600' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:48:40.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:39 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/248707600' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:48:41.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:40 smithi150 ceph-mon[34315]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-08T05:48:41.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:40 smithi112 ceph-mon[30003]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-08T05:48:41.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:40 smithi162 ceph-mon[34800]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-08T05:48:43.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:42 smithi112 ceph-mon[30003]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-08T05:48:43.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:42 smithi150 ceph-mon[34315]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-08T05:48:43.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:42 smithi162 ceph-mon[34800]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-08T05:48:45.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:44 smithi112 ceph-mon[30003]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:48:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:44 smithi150 ceph-mon[34315]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:48:45.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:44 smithi162 ceph-mon[34800]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:48:47.158 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:46 smithi112 ceph-mon[30003]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:47.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:46 smithi150 ceph-mon[34315]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:47.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:46 smithi162 ceph-mon[34800]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:49.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:48 smithi150 ceph-mon[34315]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:49.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:48 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T05:48:49.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:48 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:49.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:48 smithi150 ceph-mon[34315]: Deploying daemon osd.9 on smithi112 2024-05-08T05:48:49.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:48 smithi112 ceph-mon[30003]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:49.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:48 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T05:48:49.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:48 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:49.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:48 smithi112 ceph-mon[30003]: Deploying daemon osd.9 on smithi112 2024-05-08T05:48:49.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:48 smithi162 ceph-mon[34800]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:49.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:48 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T05:48:49.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:48 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:49.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:48 smithi162 ceph-mon[34800]: Deploying daemon osd.9 on smithi112 2024-05-08T05:48:51.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:50 smithi112 ceph-mon[30003]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:51.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:50 smithi150 ceph-mon[34315]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:51.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:50 smithi162 ceph-mon[34800]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:53.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:52 smithi150 ceph-mon[34315]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:53.205 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:52 smithi112 ceph-mon[30003]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:53.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:52 smithi162 ceph-mon[34800]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:55.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:54 smithi150 ceph-mon[34315]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:55.264 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:54 smithi112 ceph-mon[30003]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:55.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:54 smithi162 ceph-mon[34800]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:57.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:56 smithi112 ceph-mon[30003]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:57.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:56 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:57.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:56 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:57.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:56 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:57.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:56 smithi150 ceph-mon[34315]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:57.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:56 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:57.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:56 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:57.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:56 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:57.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:56 smithi162 ceph-mon[34800]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:57.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:57.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:57.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:48:59.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:48:58 smithi150 ceph-mon[34315]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:59.257 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:48:58 smithi112 ceph-mon[30003]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:59.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:48:58 smithi162 ceph-mon[34800]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:49:01.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:01 smithi112 ceph-mon[30003]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:49:01.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:01 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:01.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:01 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:01.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:01 smithi112 ceph-mon[30003]: from='osd.9 [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T05:49:01.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:01 smithi150 ceph-mon[34315]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:49:01.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:01 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:01.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:01 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:01.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:01 smithi150 ceph-mon[34315]: from='osd.9 [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T05:49:01.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:01.523+0000 7f76faa00700 1 -- 172.21.15.112:0/2154455139 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f77040638e0 con 0x7f76e80b2900 2024-05-08T05:49:01.524 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 9 on host 'smithi112' 2024-05-08T05:49:01.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:01.526+0000 7f7708c00700 1 -- 172.21.15.112:0/2154455139 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f76e80b2900 msgr2=0x7f76e80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:01.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:01.526+0000 7f7708c00700 1 --2- 172.21.15.112:0/2154455139 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f76e80b2900 0x7f76e80b4dc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f76f0000c00 tx=0x7f76f0005fb0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:01.526+0000 7f7708c00700 1 -- 172.21.15.112:0/2154455139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7704103510 msgr2=0x7f7704077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:01.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:01.526+0000 7f7708c00700 1 --2- 172.21.15.112:0/2154455139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7704103510 0x7f7704077240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f76f400e9c0 tx=0x7f76f400ecd0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:01.526+0000 7f7708c00700 1 -- 172.21.15.112:0/2154455139 shutdown_connections 2024-05-08T05:49:01.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:01.526+0000 7f7708c00700 1 --2- 172.21.15.112:0/2154455139 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f76e80b2900 0x7f76e80b4dc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:01.526+0000 7f7708c00700 1 --2- 172.21.15.112:0/2154455139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77041050b0 0x7f7704075dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:01.526+0000 7f7708c00700 1 --2- 172.21.15.112:0/2154455139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7704103510 0x7f7704077240 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:01.526+0000 7f7708c00700 1 --2- 172.21.15.112:0/2154455139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7704104710 0x7f7704075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:01.526+0000 7f7708c00700 1 -- 172.21.15.112:0/2154455139 >> 172.21.15.112:0/2154455139 conn(0x7f77040fea70 msgr2=0x7f7704107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:01.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:01.526+0000 7f7708c00700 1 -- 172.21.15.112:0/2154455139 shutdown_connections 2024-05-08T05:49:01.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:01.526+0000 7f7708c00700 1 -- 172.21.15.112:0/2154455139 wait complete. 2024-05-08T05:49:01.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:01 smithi162 ceph-mon[34800]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:49:01.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:01 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:01.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:01 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:01.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:01 smithi162 ceph-mon[34800]: from='osd.9 [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T05:49:02.333 DEBUG:teuthology.orchestra.run.smithi112:osd.9> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.9.service 2024-05-08T05:49:02.338 INFO:tasks.cephadm:Deploying osd.10 on smithi150 with /dev/vg_nvme/lv_1... 2024-05-08T05:49:02.339 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T05:49:02.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:02 smithi150 ceph-mon[34315]: from='osd.9 [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T05:49:02.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:02 smithi150 ceph-mon[34315]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T05:49:02.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:02 smithi150 ceph-mon[34315]: from='osd.9 [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:49:02.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:02 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:49:02.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:02 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:02.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:02 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:02.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:02 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:02.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:02 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:02.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:02 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:02.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:02 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:02.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:02 smithi112 ceph-mon[30003]: from='osd.9 [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T05:49:02.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:02 smithi112 ceph-mon[30003]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T05:49:02.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:02 smithi112 ceph-mon[30003]: from='osd.9 [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:49:02.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:02 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:49:02.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:02 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:02.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:02 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:02.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:02 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:02.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:02 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:02.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:02 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:02.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:02 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:02.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:02 smithi162 ceph-mon[34800]: from='osd.9 [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T05:49:02.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:02 smithi162 ceph-mon[34800]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T05:49:02.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:02 smithi162 ceph-mon[34800]: from='osd.9 [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-05-08T05:49:02.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:02 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:49:02.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:02 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:02.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:02 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:02.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:02 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:02.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:02 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:02.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:02 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:02.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:02 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:02.584 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:49:03.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:03 smithi112 ceph-mon[30003]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:49:03.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:03 smithi112 ceph-mon[30003]: from='osd.9 [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-08T05:49:03.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:03 smithi112 ceph-mon[30003]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T05:49:03.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:03 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:49:03.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:03 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:49:03.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:03 smithi150 ceph-mon[34315]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:49:03.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:03 smithi150 ceph-mon[34315]: from='osd.9 [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-08T05:49:03.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:03 smithi150 ceph-mon[34315]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T05:49:03.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:03 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:49:03.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:03 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:49:03.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:03 smithi162 ceph-mon[34800]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:49:03.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:03 smithi162 ceph-mon[34800]: from='osd.9 [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-05-08T05:49:03.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:03 smithi162 ceph-mon[34800]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T05:49:03.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:03 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:49:03.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:03 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:49:04.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:04 smithi112 ceph-mon[30003]: purged_snaps scrub starts 2024-05-08T05:49:04.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:04 smithi112 ceph-mon[30003]: purged_snaps scrub ok 2024-05-08T05:49:04.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:04 smithi112 ceph-mon[30003]: osd.9 [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674] boot 2024-05-08T05:49:04.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:04 smithi112 ceph-mon[30003]: osdmap e61: 10 total, 10 up, 10 in 2024-05-08T05:49:04.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:49:04.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:04 smithi150 ceph-mon[34315]: purged_snaps scrub starts 2024-05-08T05:49:04.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:04 smithi150 ceph-mon[34315]: purged_snaps scrub ok 2024-05-08T05:49:04.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:04 smithi150 ceph-mon[34315]: osd.9 [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674] boot 2024-05-08T05:49:04.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:04 smithi150 ceph-mon[34315]: osdmap e61: 10 total, 10 up, 10 in 2024-05-08T05:49:04.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:49:04.556 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T05:49:04.557 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T05:49:04.557 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-08T05:49:04.557 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-08T05:49:04.557 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130884 s, 801 MB/s 2024-05-08T05:49:04.557 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-08T05:49:04.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:04 smithi162 ceph-mon[34800]: purged_snaps scrub starts 2024-05-08T05:49:04.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:04 smithi162 ceph-mon[34800]: purged_snaps scrub ok 2024-05-08T05:49:04.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:04 smithi162 ceph-mon[34800]: osd.9 [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674] boot 2024-05-08T05:49:04.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:04 smithi162 ceph-mon[34800]: osdmap e61: 10 total, 10 up, 10 in 2024-05-08T05:49:04.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:49:05.225 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_1 2024-05-08T05:49:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:05 smithi150 ceph-mon[34315]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:05 smithi150 ceph-mon[34315]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T05:49:05.463 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:49:05.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:05 smithi112 ceph-mon[30003]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:05.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:05 smithi112 ceph-mon[30003]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T05:49:05.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:05 smithi162 ceph-mon[34800]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:05.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:05 smithi162 ceph-mon[34800]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T05:49:06.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.992+0000 7f2415ee9700 1 -- 172.21.15.150:0/2150107918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24101046f0 msgr2=0x7f2410104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:06.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.992+0000 7f2415ee9700 1 --2- 172.21.15.150:0/2150107918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24101046f0 0x7f2410104b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f24040099c0 tx=0x7f2404009cd0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.992+0000 7f2415ee9700 1 -- 172.21.15.150:0/2150107918 shutdown_connections 2024-05-08T05:49:06.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.992+0000 7f2415ee9700 1 --2- 172.21.15.150:0/2150107918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2410105090 0x7f241010b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.992+0000 7f2415ee9700 1 --2- 172.21.15.150:0/2150107918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24101034f0 0x7f2410103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.992+0000 7f2415ee9700 1 --2- 172.21.15.150:0/2150107918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24101046f0 0x7f2410104b50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.992+0000 7f2415ee9700 1 -- 172.21.15.150:0/2150107918 >> 172.21.15.150:0/2150107918 conn(0x7f24100fea90 msgr2=0x7f2410100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:06.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.992+0000 7f2415ee9700 1 -- 172.21.15.150:0/2150107918 shutdown_connections 2024-05-08T05:49:06.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.992+0000 7f2415ee9700 1 -- 172.21.15.150:0/2150107918 wait complete. 2024-05-08T05:49:06.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.992+0000 7f2415ee9700 1 Processor -- start 2024-05-08T05:49:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.992+0000 7f2415ee9700 1 -- start start 2024-05-08T05:49:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f2415ee9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24101034f0 0x7f2410199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f2415ee9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24101046f0 0x7f2410199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f2415ee9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2410105090 0x7f24101a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f2415ee9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f241010f030 con 0x7f24101034f0 2024-05-08T05:49:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f2415ee9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f241010eeb0 con 0x7f24101046f0 2024-05-08T05:49:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f2415ee9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f241010f1b0 con 0x7f2410105090 2024-05-08T05:49:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f240effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24101046f0 0x7f2410199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f240effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24101046f0 0x7f2410199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42390/0 (socket says 172.21.15.150:42390) 2024-05-08T05:49:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f240effd700 1 -- 172.21.15.150:0/3437799523 learned_addr learned my addr 172.21.15.150:0/3437799523 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-08T05:49:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f240ffff700 1 --2- 172.21.15.150:0/3437799523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2410105090 0x7f24101a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f240effd700 1 -- 172.21.15.150:0/3437799523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2410105090 msgr2=0x7f24101a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f240effd700 1 --2- 172.21.15.150:0/3437799523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2410105090 0x7f24101a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f240effd700 1 -- 172.21.15.150:0/3437799523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24101034f0 msgr2=0x7f2410199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f240effd700 1 --2- 172.21.15.150:0/3437799523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24101034f0 0x7f2410199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f240effd700 1 -- 172.21.15.150:0/3437799523 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24101a1380 con 0x7f24101046f0 2024-05-08T05:49:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f240effd700 1 --2- 172.21.15.150:0/3437799523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24101046f0 0x7f2410199c60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2404005910 tx=0x7f240400fab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f240cff9700 1 -- 172.21.15.150:0/3437799523 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404027070 con 0x7f24101046f0 2024-05-08T05:49:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f240cff9700 1 -- 172.21.15.150:0/3437799523 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2404022920 con 0x7f24101046f0 2024-05-08T05:49:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f240cff9700 1 -- 172.21.15.150:0/3437799523 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404009e60 con 0x7f24101046f0 2024-05-08T05:49:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f2415ee9700 1 -- 172.21.15.150:0/3437799523 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24101a1610 con 0x7f24101046f0 2024-05-08T05:49:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.993+0000 7f2415ee9700 1 -- 172.21.15.150:0/3437799523 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24101a1b40 con 0x7f24101046f0 2024-05-08T05:49:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.994+0000 7f2415ee9700 1 -- 172.21.15.150:0/3437799523 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241006a190 con 0x7f24101046f0 2024-05-08T05:49:06.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.996+0000 7f240cff9700 1 -- 172.21.15.150:0/3437799523 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f240402a070 con 0x7f24101046f0 2024-05-08T05:49:06.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.996+0000 7f240cff9700 1 --2- 172.21.15.150:0/3437799523 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f23f80b2900 0x7f23f80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:06.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.996+0000 7f240cff9700 1 -- 172.21.15.150:0/3437799523 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6261+0+0 (secure 0 0 0) 0x7f24040e2930 con 0x7f24101046f0 2024-05-08T05:49:06.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.997+0000 7f240f7fe700 1 --2- 172.21.15.150:0/3437799523 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f23f80b2900 0x7f23f80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:06.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:06.998+0000 7f240f7fe700 1 --2- 172.21.15.150:0/3437799523 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f23f80b2900 0x7f23f80b4dc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2410104550 tx=0x7f2400006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:07.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:07.000+0000 7f240cff9700 1 -- 172.21.15.150:0/3437799523 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2404013070 con 0x7f24101046f0 2024-05-08T05:49:07.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:07.214+0000 7f2415ee9700 1 -- 172.21.15.150:0/3437799523 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f24100638e0 con 0x7f23f80b2900 2024-05-08T05:49:07.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:07 smithi112 ceph-mon[30003]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:07.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:07.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:07 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:07.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:07 smithi162 ceph-mon[34800]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:07.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:07.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:07 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:07 smithi150 ceph-mon[34315]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:07 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:08.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:08 smithi112 ceph-mon[30003]: Detected new or changed devices on smithi112 2024-05-08T05:49:08.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:08.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:08.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:08 smithi112 ceph-mon[30003]: Adjusting osd_memory_target on smithi112 to 4272M 2024-05-08T05:49:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:08 smithi112 ceph-mon[30003]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:49:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:49:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:49:08.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:08.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:08 smithi162 ceph-mon[34800]: Detected new or changed devices on smithi112 2024-05-08T05:49:08.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:08.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:08.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:08.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:08.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:08 smithi162 ceph-mon[34800]: Adjusting osd_memory_target on smithi112 to 4272M 2024-05-08T05:49:08.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:08.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:08.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:08.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:08 smithi162 ceph-mon[34800]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:49:08.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:49:08.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:08.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:49:08.569 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:08 smithi150 ceph-mon[34315]: Detected new or changed devices on smithi112 2024-05-08T05:49:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:08 smithi150 ceph-mon[34315]: Adjusting osd_memory_target on smithi112 to 4272M 2024-05-08T05:49:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:08 smithi150 ceph-mon[34315]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:49:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:49:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:49:08.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:09.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:09 smithi150 ceph-mon[34315]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:09.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:09 smithi112 ceph-mon[30003]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:09.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:09 smithi162 ceph-mon[34800]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:10.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:10 smithi112 ceph-mon[30003]: from='client.? 172.21.15.150:0/3269853905' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34e08e3e-1231-4899-a5b3-76d449105bf4"}]: dispatch 2024-05-08T05:49:10.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:10 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34e08e3e-1231-4899-a5b3-76d449105bf4"}]: dispatch 2024-05-08T05:49:10.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:10 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34e08e3e-1231-4899-a5b3-76d449105bf4"}]': finished 2024-05-08T05:49:10.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:10 smithi112 ceph-mon[30003]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T05:49:10.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:10 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:49:10.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:10 smithi162 ceph-mon[34800]: from='client.? 172.21.15.150:0/3269853905' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34e08e3e-1231-4899-a5b3-76d449105bf4"}]: dispatch 2024-05-08T05:49:10.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:10 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34e08e3e-1231-4899-a5b3-76d449105bf4"}]: dispatch 2024-05-08T05:49:10.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:10 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34e08e3e-1231-4899-a5b3-76d449105bf4"}]': finished 2024-05-08T05:49:10.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:10 smithi162 ceph-mon[34800]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T05:49:10.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:10 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:49:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:10 smithi150 ceph-mon[34315]: from='client.? 172.21.15.150:0/3269853905' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34e08e3e-1231-4899-a5b3-76d449105bf4"}]: dispatch 2024-05-08T05:49:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:10 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34e08e3e-1231-4899-a5b3-76d449105bf4"}]: dispatch 2024-05-08T05:49:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:10 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34e08e3e-1231-4899-a5b3-76d449105bf4"}]': finished 2024-05-08T05:49:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:10 smithi150 ceph-mon[34315]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T05:49:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:10 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:49:11.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:11 smithi112 ceph-mon[30003]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:11.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:11 smithi112 ceph-mon[30003]: from='client.? 172.21.15.150:0/742297319' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:49:11.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:11 smithi162 ceph-mon[34800]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:11.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:11 smithi162 ceph-mon[34800]: from='client.? 172.21.15.150:0/742297319' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:49:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:11 smithi150 ceph-mon[34315]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:11 smithi150 ceph-mon[34315]: from='client.? 172.21.15.150:0/742297319' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:49:13.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:13 smithi112 ceph-mon[30003]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:13.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:13 smithi162 ceph-mon[34800]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:13.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:13 smithi150 ceph-mon[34315]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:14.561 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:14 smithi150 ceph-mon[34315]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:14.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:14 smithi112 ceph-mon[30003]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:14.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:14 smithi162 ceph-mon[34800]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:16.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:16 smithi150 ceph-mon[34315]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:16.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:16 smithi112 ceph-mon[30003]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:16.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:16 smithi162 ceph-mon[34800]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:18.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:18 smithi112 ceph-mon[30003]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:18.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T05:49:18.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:18 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:18.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:18 smithi112 ceph-mon[30003]: Deploying daemon osd.10 on smithi150 2024-05-08T05:49:18.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:18 smithi162 ceph-mon[34800]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:18.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:18 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T05:49:18.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:18 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:18.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:18 smithi162 ceph-mon[34800]: Deploying daemon osd.10 on smithi150 2024-05-08T05:49:18.830 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:18 smithi150 ceph-mon[34315]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:18.831 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:18 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T05:49:18.831 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:18 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:18.831 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:18 smithi150 ceph-mon[34315]: Deploying daemon osd.10 on smithi150 2024-05-08T05:49:20.603 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:20 smithi150 ceph-mon[34315]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:20.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:20 smithi112 ceph-mon[30003]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:20.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:20 smithi162 ceph-mon[34800]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:22.752 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:22 smithi150 ceph-mon[34315]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:22.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:22 smithi112 ceph-mon[30003]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:22.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:22 smithi162 ceph-mon[34800]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:24.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:24.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:24.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:24.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:24.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:24.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:24.372 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:24.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:24.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:25.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:25 smithi112 ceph-mon[30003]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:25.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:25 smithi162 ceph-mon[34800]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:25.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:25 smithi150 ceph-mon[34315]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:26.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:26.255+0000 7f240cff9700 1 -- 172.21.15.150:0/3437799523 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f24100638e0 con 0x7f23f80b2900 2024-05-08T05:49:26.257 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 10 on host 'smithi150' 2024-05-08T05:49:26.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:26.258+0000 7f2415ee9700 1 -- 172.21.15.150:0/3437799523 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f23f80b2900 msgr2=0x7f23f80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:26.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:26.258+0000 7f2415ee9700 1 --2- 172.21.15.150:0/3437799523 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f23f80b2900 0x7f23f80b4dc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2410104550 tx=0x7f2400006d20 comp rx=0 tx=0).stop 2024-05-08T05:49:26.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:26.258+0000 7f2415ee9700 1 -- 172.21.15.150:0/3437799523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24101046f0 msgr2=0x7f2410199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:26.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:26.258+0000 7f2415ee9700 1 --2- 172.21.15.150:0/3437799523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24101046f0 0x7f2410199c60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2404005910 tx=0x7f240400fab0 comp rx=0 tx=0).stop 2024-05-08T05:49:26.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:26.258+0000 7f2415ee9700 1 -- 172.21.15.150:0/3437799523 shutdown_connections 2024-05-08T05:49:26.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:26.258+0000 7f2415ee9700 1 --2- 172.21.15.150:0/3437799523 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f23f80b2900 0x7f23f80b4dc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:26.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:26.258+0000 7f2415ee9700 1 --2- 172.21.15.150:0/3437799523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2410105090 0x7f24101a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:26.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:26.258+0000 7f2415ee9700 1 --2- 172.21.15.150:0/3437799523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24101034f0 0x7f2410199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:26.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:26.258+0000 7f2415ee9700 1 --2- 172.21.15.150:0/3437799523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24101046f0 0x7f2410199c60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:26.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:26.258+0000 7f2415ee9700 1 -- 172.21.15.150:0/3437799523 >> 172.21.15.150:0/3437799523 conn(0x7f24100fea90 msgr2=0x7f2410107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:26.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:26.258+0000 7f2415ee9700 1 -- 172.21.15.150:0/3437799523 shutdown_connections 2024-05-08T05:49:26.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-08T05:49:26.258+0000 7f2415ee9700 1 -- 172.21.15.150:0/3437799523 wait complete. 2024-05-08T05:49:27.200 DEBUG:teuthology.orchestra.run.smithi150:osd.10> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.10.service 2024-05-08T05:49:27.202 INFO:tasks.cephadm:Deploying osd.11 on smithi162 with /dev/vg_nvme/lv_1... 2024-05-08T05:49:27.203 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T05:49:27.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:27 smithi150 ceph-mon[34315]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:27.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:27 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:27.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:27 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:27.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:27 smithi150 ceph-mon[34315]: from='osd.10 [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:49:27.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:27 smithi150 ceph-mon[34315]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:49:27.451 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:49:27.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:27 smithi112 ceph-mon[30003]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:27.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:27 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:27.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:27 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:27.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:27 smithi112 ceph-mon[30003]: from='osd.10 [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:49:27.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:27 smithi112 ceph-mon[30003]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:49:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:27 smithi162 ceph-mon[34800]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:27 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:27 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:27.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:27 smithi162 ceph-mon[34800]: from='osd.10 [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:49:27.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:27 smithi162 ceph-mon[34800]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:49:28.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:28 smithi112 ceph-mon[30003]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T05:49:28.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:28 smithi112 ceph-mon[30003]: from='osd.10 [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:49:28.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:28 smithi112 ceph-mon[30003]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T05:49:28.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:28 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:49:28.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:28 smithi112 ceph-mon[30003]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:49:28.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:28 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:28.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:28 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:28.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:28 smithi162 ceph-mon[34800]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T05:49:28.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:28 smithi162 ceph-mon[34800]: from='osd.10 [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:49:28.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:28 smithi162 ceph-mon[34800]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T05:49:28.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:28 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:49:28.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:28 smithi162 ceph-mon[34800]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:49:28.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:28 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:28.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:28 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:28.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:28 smithi150 ceph-mon[34315]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T05:49:28.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:28 smithi150 ceph-mon[34315]: from='osd.10 [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:49:28.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:28 smithi150 ceph-mon[34315]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T05:49:28.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:28 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:49:28.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:28 smithi150 ceph-mon[34315]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-08T05:49:28.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:28 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:28.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:28 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:28.684 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 05:49:28 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T05:49:28.236+0000 7f307ac94700 -1 osd.10 0 waiting for initial osdmap 2024-05-08T05:49:28.684 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 05:49:28 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T05:49:28.248+0000 7f3076a87700 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:49:29.487 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T05:49:29.487 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T05:49:29.487 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2024-05-08T05:49:29.487 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2024-05-08T05:49:29.488 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115218 s, 910 MB/s 2024-05-08T05:49:29.488 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2024-05-08T05:49:29.511 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:29 smithi162 ceph-mon[34800]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:29.512 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:29 smithi162 ceph-mon[34800]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-08T05:49:29.512 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:29 smithi162 ceph-mon[34800]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T05:49:29.512 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:49:29.512 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:29 smithi162 ceph-mon[34800]: osd.10 [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474] boot 2024-05-08T05:49:29.512 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:29 smithi162 ceph-mon[34800]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T05:49:29.512 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:49:29.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:29 smithi112 ceph-mon[30003]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:29.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:29 smithi112 ceph-mon[30003]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-08T05:49:29.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:29 smithi112 ceph-mon[30003]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T05:49:29.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:49:29.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:29 smithi112 ceph-mon[30003]: osd.10 [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474] boot 2024-05-08T05:49:29.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:29 smithi112 ceph-mon[30003]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T05:49:29.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:49:29.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:29 smithi150 ceph-mon[34315]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:49:29.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:29 smithi150 ceph-mon[34315]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-08T05:49:29.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:29 smithi150 ceph-mon[34315]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T05:49:29.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:49:29.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:29 smithi150 ceph-mon[34315]: osd.10 [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474] boot 2024-05-08T05:49:29.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:29 smithi150 ceph-mon[34315]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T05:49:29.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:49:30.142 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_1 2024-05-08T05:49:30.382 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:49:30.481 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:30 smithi162 ceph-mon[34800]: purged_snaps scrub starts 2024-05-08T05:49:30.481 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:30 smithi162 ceph-mon[34800]: purged_snaps scrub ok 2024-05-08T05:49:30.481 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:30 smithi162 ceph-mon[34800]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T05:49:30.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:30 smithi112 ceph-mon[30003]: purged_snaps scrub starts 2024-05-08T05:49:30.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:30 smithi112 ceph-mon[30003]: purged_snaps scrub ok 2024-05-08T05:49:30.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:30 smithi112 ceph-mon[30003]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T05:49:30.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:30 smithi150 ceph-mon[34315]: purged_snaps scrub starts 2024-05-08T05:49:30.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:30 smithi150 ceph-mon[34315]: purged_snaps scrub ok 2024-05-08T05:49:30.575 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:30 smithi150 ceph-mon[34315]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T05:49:31.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:31.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:31.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:31.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:31 smithi162 ceph-mon[34800]: osdmap e68: 11 total, 11 up, 11 in 2024-05-08T05:49:31.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:31.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:31.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:31.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:31.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:31.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:31.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:31.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:31.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:31.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:31.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:31.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:31.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:31.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:31.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:31.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:31 smithi112 ceph-mon[30003]: osdmap e68: 11 total, 11 up, 11 in 2024-05-08T05:49:31.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:31.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:31.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:31.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:31.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:31.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:31.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:31.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:31.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:31.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:31.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:31.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:31.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:31.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:31.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:31.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:31 smithi150 ceph-mon[34315]: osdmap e68: 11 total, 11 up, 11 in 2024-05-08T05:49:31.728 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.725+0000 7f15ab9c5700 1 -- 172.21.15.162:0/2206401748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15a4109290 msgr2=0x7f15a410b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:31.728 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.725+0000 7f15ab9c5700 1 --2- 172.21.15.162:0/2206401748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15a4109290 0x7f15a410b680 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f159c009990 tx=0x7f159c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:49:31.728 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.725+0000 7f15ab9c5700 1 -- 172.21.15.162:0/2206401748 shutdown_connections 2024-05-08T05:49:31.728 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.725+0000 7f15ab9c5700 1 --2- 172.21.15.162:0/2206401748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15a4109290 0x7f15a410b680 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:31.728 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.725+0000 7f15ab9c5700 1 --2- 172.21.15.162:0/2206401748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4104030 0x7f15a4106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:31.728 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.725+0000 7f15ab9c5700 1 --2- 172.21.15.162:0/2206401748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a4106960 0x7f15a4108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:31.728 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.725+0000 7f15ab9c5700 1 -- 172.21.15.162:0/2206401748 >> 172.21.15.162:0/2206401748 conn(0x7f15a40fd9a0 msgr2=0x7f15a40ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:31.728 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.725+0000 7f15ab9c5700 1 -- 172.21.15.162:0/2206401748 shutdown_connections 2024-05-08T05:49:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.725+0000 7f15ab9c5700 1 -- 172.21.15.162:0/2206401748 wait complete. 2024-05-08T05:49:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15ab9c5700 1 Processor -- start 2024-05-08T05:49:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15ab9c5700 1 -- start start 2024-05-08T05:49:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15ab9c5700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15a4104030 0x7f15a4199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15ab9c5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a4106960 0x7f15a4199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15ab9c5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4109290 0x7f15a41a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15ab9c5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15a410f000 con 0x7f15a4106960 2024-05-08T05:49:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15ab9c5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15a410ee80 con 0x7f15a4109290 2024-05-08T05:49:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15ab9c5700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f15a410f180 con 0x7f15a4104030 2024-05-08T05:49:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15a9761700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15a4104030 0x7f15a4199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15a9761700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15a4104030 0x7f15a4199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:42762/0 (socket says 172.21.15.162:42762) 2024-05-08T05:49:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15a9761700 1 -- 172.21.15.162:0/858966100 learned_addr learned my addr 172.21.15.162:0/858966100 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-08T05:49:31.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15a9f62700 1 --2- 172.21.15.162:0/858966100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4109290 0x7f15a41a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15a9761700 1 -- 172.21.15.162:0/858966100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4109290 msgr2=0x7f15a41a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15a9761700 1 --2- 172.21.15.162:0/858966100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4109290 0x7f15a41a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15a9761700 1 -- 172.21.15.162:0/858966100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a4106960 msgr2=0x7f15a4199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15a9761700 1 --2- 172.21.15.162:0/858966100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a4106960 0x7f15a4199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15a9761700 1 -- 172.21.15.162:0/858966100 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a41a12b0 con 0x7f15a4104030 2024-05-08T05:49:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.726+0000 7f15a9761700 1 --2- 172.21.15.162:0/858966100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15a4104030 0x7f15a4199650 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f15a0007f40 tx=0x7f15a000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.727+0000 7f159a7fc700 1 -- 172.21.15.162:0/858966100 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0004bb0 con 0x7f15a4104030 2024-05-08T05:49:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.727+0000 7f159a7fc700 1 -- 172.21.15.162:0/858966100 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a0004d50 con 0x7f15a4104030 2024-05-08T05:49:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.727+0000 7f15ab9c5700 1 -- 172.21.15.162:0/858966100 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a41a15a0 con 0x7f15a4104030 2024-05-08T05:49:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.727+0000 7f159a7fc700 1 -- 172.21.15.162:0/858966100 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0005a00 con 0x7f15a4104030 2024-05-08T05:49:31.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.727+0000 7f15ab9c5700 1 -- 172.21.15.162:0/858966100 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a41a1b10 con 0x7f15a4104030 2024-05-08T05:49:31.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.728+0000 7f15ab9c5700 1 -- 172.21.15.162:0/858966100 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a406a190 con 0x7f15a4104030 2024-05-08T05:49:31.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.729+0000 7f159a7fc700 1 -- 172.21.15.162:0/858966100 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f15a001a020 con 0x7f15a4104030 2024-05-08T05:49:31.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.729+0000 7f159a7fc700 1 --2- 172.21.15.162:0/858966100 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f15900b2880 0x7f15900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:31.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.730+0000 7f159a7fc700 1 -- 172.21.15.162:0/858966100 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6694+0+0 (secure 0 0 0) 0x7f15a00e13c0 con 0x7f15a4104030 2024-05-08T05:49:31.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.730+0000 7f15a8f60700 1 --2- 172.21.15.162:0/858966100 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f15900b2880 0x7f15900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:31.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.731+0000 7f15a8f60700 1 --2- 172.21.15.162:0/858966100 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f15900b2880 0x7f15900b4d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f15940097b0 tx=0x7f1594006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:31.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.734+0000 7f159a7fc700 1 -- 172.21.15.162:0/858966100 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15a0010070 con 0x7f15a4104030 2024-05-08T05:49:31.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:31.949+0000 7f15ab9c5700 1 -- 172.21.15.162:0/858966100 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f15a40638e0 con 0x7f15900b2880 2024-05-08T05:49:32.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:32 smithi112 ceph-mon[30003]: Detected new or changed devices on smithi150 2024-05-08T05:49:32.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:32 smithi112 ceph-mon[30003]: Adjusting osd_memory_target on smithi150 to 4272M 2024-05-08T05:49:32.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:49:32.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:49:32.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:32.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:32 smithi162 ceph-mon[34800]: Detected new or changed devices on smithi150 2024-05-08T05:49:32.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:32 smithi162 ceph-mon[34800]: Adjusting osd_memory_target on smithi150 to 4272M 2024-05-08T05:49:32.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:32 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:49:32.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:32 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:49:32.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:32 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:32.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:32 smithi150 ceph-mon[34315]: Detected new or changed devices on smithi150 2024-05-08T05:49:32.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:32 smithi150 ceph-mon[34315]: Adjusting osd_memory_target on smithi150 to 4272M 2024-05-08T05:49:32.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:32 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:49:32.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:32 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:49:32.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:32 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:33.529 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:33 smithi162 ceph-mon[34800]: pgmap v252: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:33.530 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:33 smithi162 ceph-mon[34800]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:49:33.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:33 smithi112 ceph-mon[30003]: pgmap v252: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:33.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:33 smithi112 ceph-mon[30003]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:49:33.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:33 smithi150 ceph-mon[34315]: pgmap v252: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:33.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:33 smithi150 ceph-mon[34315]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:49:34.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:34 smithi112 ceph-mon[30003]: pgmap v253: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:34.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:34 smithi162 ceph-mon[34800]: pgmap v253: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:34.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:34 smithi150 ceph-mon[34315]: pgmap v253: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:35.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:35 smithi112 ceph-mon[30003]: from='client.? 172.21.15.162:0/3671354107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fc34ec0-78ca-403c-acb7-b29d83ff64e8"}]: dispatch 2024-05-08T05:49:35.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:35 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fc34ec0-78ca-403c-acb7-b29d83ff64e8"}]: dispatch 2024-05-08T05:49:35.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:35 smithi112 ceph-mon[30003]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fc34ec0-78ca-403c-acb7-b29d83ff64e8"}]': finished 2024-05-08T05:49:35.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:35 smithi112 ceph-mon[30003]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T05:49:35.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:35 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:49:35.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:35 smithi112 ceph-mon[30003]: from='client.? 172.21.15.162:0/555030038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:49:35.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:35 smithi150 ceph-mon[34315]: from='client.? 172.21.15.162:0/3671354107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fc34ec0-78ca-403c-acb7-b29d83ff64e8"}]: dispatch 2024-05-08T05:49:35.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:35 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fc34ec0-78ca-403c-acb7-b29d83ff64e8"}]: dispatch 2024-05-08T05:49:35.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:35 smithi150 ceph-mon[34315]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fc34ec0-78ca-403c-acb7-b29d83ff64e8"}]': finished 2024-05-08T05:49:35.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:35 smithi150 ceph-mon[34315]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T05:49:35.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:35 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:49:35.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:35 smithi150 ceph-mon[34315]: from='client.? 172.21.15.162:0/555030038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:49:35.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:35 smithi162 ceph-mon[34800]: from='client.? 172.21.15.162:0/3671354107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fc34ec0-78ca-403c-acb7-b29d83ff64e8"}]: dispatch 2024-05-08T05:49:35.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:35 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fc34ec0-78ca-403c-acb7-b29d83ff64e8"}]: dispatch 2024-05-08T05:49:35.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:35 smithi162 ceph-mon[34800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fc34ec0-78ca-403c-acb7-b29d83ff64e8"}]': finished 2024-05-08T05:49:35.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:35 smithi162 ceph-mon[34800]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T05:49:35.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:35 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:49:35.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:35 smithi162 ceph-mon[34800]: from='client.? 172.21.15.162:0/555030038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:49:36.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:36 smithi150 ceph-mon[34315]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:49:36.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:36 smithi112 ceph-mon[30003]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:49:36.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:36 smithi162 ceph-mon[34800]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:49:38.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:38 smithi162 ceph-mon[34800]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-08T05:49:38.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:38 smithi112 ceph-mon[30003]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-08T05:49:38.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:38 smithi150 ceph-mon[34315]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-08T05:49:40.732 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:40 smithi162 ceph-mon[34800]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-08T05:49:40.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:40 smithi112 ceph-mon[30003]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-08T05:49:40.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:40 smithi150 ceph-mon[34315]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-08T05:49:42.693 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:42 smithi162 ceph-mon[34800]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:49:42.693 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:42 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T05:49:42.693 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:42 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:42.694 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:42 smithi162 ceph-mon[34800]: Deploying daemon osd.11 on smithi162 2024-05-08T05:49:42.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:42 smithi112 ceph-mon[30003]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:49:42.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T05:49:42.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:42.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:42 smithi112 ceph-mon[30003]: Deploying daemon osd.11 on smithi162 2024-05-08T05:49:42.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:42 smithi150 ceph-mon[34315]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:49:42.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:42 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T05:49:42.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:42 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:42.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:42 smithi150 ceph-mon[34315]: Deploying daemon osd.11 on smithi162 2024-05-08T05:49:44.760 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:44 smithi162 ceph-mon[34800]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:49:44.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:44 smithi112 ceph-mon[30003]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:49:44.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:44 smithi150 ceph-mon[34315]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:49:46.611 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:46 smithi162 ceph-mon[34800]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-08T05:49:46.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:46 smithi112 ceph-mon[30003]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-08T05:49:46.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:46 smithi150 ceph-mon[34315]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-08T05:49:48.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:48 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:48.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:48 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:48.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:48 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:48.553 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:48 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:48.553 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:48 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:48.553 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:48 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:48.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:48 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:48.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:48 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:48.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:48 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:49.304 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:49 smithi162 ceph-mon[34800]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:49.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:49 smithi112 ceph-mon[30003]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:49.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:49 smithi150 ceph-mon[34315]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:50.729 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 11 on host 'smithi162' 2024-05-08T05:49:50.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:50.729+0000 7f159a7fc700 1 -- 172.21.15.162:0/858966100 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f15a40638e0 con 0x7f15900b2880 2024-05-08T05:49:50.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:50.732+0000 7f15ab9c5700 1 -- 172.21.15.162:0/858966100 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f15900b2880 msgr2=0x7f15900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:50.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:50.732+0000 7f15ab9c5700 1 --2- 172.21.15.162:0/858966100 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f15900b2880 0x7f15900b4d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f15940097b0 tx=0x7f1594006d20 comp rx=0 tx=0).stop 2024-05-08T05:49:50.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:50.732+0000 7f15ab9c5700 1 -- 172.21.15.162:0/858966100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15a4104030 msgr2=0x7f15a4199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:50.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:50.732+0000 7f15ab9c5700 1 --2- 172.21.15.162:0/858966100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15a4104030 0x7f15a4199650 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f15a0007f40 tx=0x7f15a000d3b0 comp rx=0 tx=0).stop 2024-05-08T05:49:50.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:50.732+0000 7f15ab9c5700 1 -- 172.21.15.162:0/858966100 shutdown_connections 2024-05-08T05:49:50.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:50.732+0000 7f15ab9c5700 1 --2- 172.21.15.162:0/858966100 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f15900b2880 0x7f15900b4d40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:50.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:50.732+0000 7f15ab9c5700 1 --2- 172.21.15.162:0/858966100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4109290 0x7f15a41a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:50.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:50.732+0000 7f15ab9c5700 1 --2- 172.21.15.162:0/858966100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15a4104030 0x7f15a4199650 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:50.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:50.732+0000 7f15ab9c5700 1 --2- 172.21.15.162:0/858966100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a4106960 0x7f15a4199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:50.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:50.732+0000 7f15ab9c5700 1 -- 172.21.15.162:0/858966100 >> 172.21.15.162:0/858966100 conn(0x7f15a40fd9a0 msgr2=0x7f15a4104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:50.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:50.732+0000 7f15ab9c5700 1 -- 172.21.15.162:0/858966100 shutdown_connections 2024-05-08T05:49:50.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:49:50.732+0000 7f15ab9c5700 1 -- 172.21.15.162:0/858966100 wait complete. 2024-05-08T05:49:51.300 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:51 smithi162 ceph-mon[34800]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:51.301 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:51 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:51.301 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:51 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:51.301 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:51 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:51.301 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:51 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:51.301 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:51 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:51.301 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:51 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:51.301 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:51 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:51.301 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:51 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:51.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:51 smithi112 ceph-mon[30003]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:51.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:51 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:51.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:51 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:51.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:51 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:51.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:51 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:51.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:51 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:51.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:51 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:51.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:51 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:51.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:51 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:51.329 DEBUG:teuthology.orchestra.run.smithi162:osd.11> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.11.service 2024-05-08T05:49:51.331 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-08T05:49:51.331 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T05:49:51.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:51 smithi150 ceph-mon[34315]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:51.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:51 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:51.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:51 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:51.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:51 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:51.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:51 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:51.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:51 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:51.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:51 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:51.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:51 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:51.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:51 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:51.769 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:52.248 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:52 smithi112 ceph-mon[30003]: from='osd.11 [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:49:52.248 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:52 smithi112 ceph-mon[30003]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:49:52.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:52 smithi162 ceph-mon[34800]: from='osd.11 [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:49:52.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:52 smithi162 ceph-mon[34800]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:49:52.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:52 smithi150 ceph-mon[34315]: from='osd.11 [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:49:52.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:52 smithi150 ceph-mon[34315]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:49:53.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:53 smithi112 ceph-mon[30003]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:53.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:53 smithi112 ceph-mon[30003]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T05:49:53.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:53 smithi112 ceph-mon[30003]: from='osd.11 [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:49:53.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:53 smithi112 ceph-mon[30003]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T05:49:53.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:53 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:49:53.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:53 smithi112 ceph-mon[30003]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:49:53.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:53 smithi112 ceph-mon[30003]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-05-08T05:49:53.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:53 smithi162 ceph-mon[34800]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:53.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:53 smithi162 ceph-mon[34800]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T05:49:53.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:53 smithi162 ceph-mon[34800]: from='osd.11 [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:49:53.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:53 smithi162 ceph-mon[34800]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T05:49:53.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:53 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:49:53.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:53 smithi162 ceph-mon[34800]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:49:53.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:53 smithi162 ceph-mon[34800]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-05-08T05:49:53.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 05:49:53 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T05:49:53.035+0000 7f4e90fea700 -1 osd.11 0 waiting for initial osdmap 2024-05-08T05:49:53.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 05:49:53 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T05:49:53.055+0000 7f4e8cddd700 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:49:53.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:53 smithi150 ceph-mon[34315]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:53.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:53 smithi150 ceph-mon[34315]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T05:49:53.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:53 smithi150 ceph-mon[34315]: from='osd.11 [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:49:53.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:53 smithi150 ceph-mon[34315]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T05:49:53.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:53 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:49:53.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:53 smithi150 ceph-mon[34315]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-05-08T05:49:53.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:53 smithi150 ceph-mon[34315]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-05-08T05:49:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.910+0000 7f97d1000700 1 -- 172.21.15.112:0/716148250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97cc1071c0 msgr2=0x7f97cc10da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.910+0000 7f97d1000700 1 --2- 172.21.15.112:0/716148250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97cc1071c0 0x7f97cc10da70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f97c400b0d0 tx=0x7f97c400b3e0 comp rx=0 tx=0).stop 2024-05-08T05:49:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.910+0000 7f97d1000700 1 -- 172.21.15.112:0/716148250 shutdown_connections 2024-05-08T05:49:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.910+0000 7f97d1000700 1 --2- 172.21.15.112:0/716148250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97cc1071c0 0x7f97cc10da70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.910+0000 7f97d1000700 1 --2- 172.21.15.112:0/716148250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97cc105620 0x7f97cc105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.910+0000 7f97d1000700 1 --2- 172.21.15.112:0/716148250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97cc106820 0x7f97cc106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.910+0000 7f97d1000700 1 -- 172.21.15.112:0/716148250 >> 172.21.15.112:0/716148250 conn(0x7f97cc100bc0 msgr2=0x7f97cc103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.910+0000 7f97d1000700 1 -- 172.21.15.112:0/716148250 shutdown_connections 2024-05-08T05:49:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.910+0000 7f97d1000700 1 -- 172.21.15.112:0/716148250 wait complete. 2024-05-08T05:49:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.911+0000 7f97d1000700 1 Processor -- start 2024-05-08T05:49:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.911+0000 7f97d1000700 1 -- start start 2024-05-08T05:49:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.912+0000 7f97d1000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97cc105620 0x7f97cc19b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.912+0000 7f97d1000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97cc106820 0x7f97cc19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.912+0000 7f97d1000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97cc1071c0 0x7f97cc1a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.912+0000 7f97d1000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97cc1110b0 con 0x7f97cc1071c0 2024-05-08T05:49:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.912+0000 7f97d1000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f97cc110f30 con 0x7f97cc106820 2024-05-08T05:49:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.912+0000 7f97d1000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f97cc111230 con 0x7f97cc105620 2024-05-08T05:49:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.912+0000 7f97cbe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97cc1071c0 0x7f97cc1a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.912+0000 7f97caa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97cc106820 0x7f97cc19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.912+0000 7f97cbe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97cc1071c0 0x7f97cc1a2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47202/0 (socket says 172.21.15.112:47202) 2024-05-08T05:49:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.912+0000 7f97cbe00700 1 -- 172.21.15.112:0/3012130567 learned_addr learned my addr 172.21.15.112:0/3012130567 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:49:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.912+0000 7f97cb400700 1 --2- 172.21.15.112:0/3012130567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97cc105620 0x7f97cc19b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.913+0000 7f97cbe00700 1 -- 172.21.15.112:0/3012130567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97cc105620 msgr2=0x7f97cc19b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.913+0000 7f97cbe00700 1 --2- 172.21.15.112:0/3012130567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97cc105620 0x7f97cc19b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.913+0000 7f97cbe00700 1 -- 172.21.15.112:0/3012130567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97cc106820 msgr2=0x7f97cc19bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.913+0000 7f97cbe00700 1 --2- 172.21.15.112:0/3012130567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97cc106820 0x7f97cc19bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.913+0000 7f97cbe00700 1 -- 172.21.15.112:0/3012130567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97cc1a32f0 con 0x7f97cc1071c0 2024-05-08T05:49:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.913+0000 7f97caa00700 1 --2- 172.21.15.112:0/3012130567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97cc106820 0x7f97cc19bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:49:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.913+0000 7f97cb400700 1 --2- 172.21.15.112:0/3012130567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97cc105620 0x7f97cc19b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:49:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.913+0000 7f97cbe00700 1 --2- 172.21.15.112:0/3012130567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97cc1071c0 0x7f97cc1a2c50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f97c4004ce0 tx=0x7f97c4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.913+0000 7f97c3e00700 1 -- 172.21.15.112:0/3012130567 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c401f030 con 0x7f97cc1071c0 2024-05-08T05:49:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.913+0000 7f97d1000700 1 -- 172.21.15.112:0/3012130567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97cc1a3580 con 0x7f97cc1071c0 2024-05-08T05:49:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.913+0000 7f97c3e00700 1 -- 172.21.15.112:0/3012130567 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97c4005040 con 0x7f97cc1071c0 2024-05-08T05:49:53.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.913+0000 7f97c3e00700 1 -- 172.21.15.112:0/3012130567 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c401b690 con 0x7f97cc1071c0 2024-05-08T05:49:53.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.913+0000 7f97d1000700 1 -- 172.21.15.112:0/3012130567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97cc1a3a80 con 0x7f97cc1071c0 2024-05-08T05:49:53.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.915+0000 7f97d1000700 1 -- 172.21.15.112:0/3012130567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97cc06a190 con 0x7f97cc1071c0 2024-05-08T05:49:53.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.917+0000 7f97c3e00700 1 -- 172.21.15.112:0/3012130567 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f97c4022030 con 0x7f97cc1071c0 2024-05-08T05:49:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.919+0000 7f97c3e00700 1 --2- 172.21.15.112:0/3012130567 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f97b00b2900 0x7f97b00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.919+0000 7f97c3e00700 1 -- 172.21.15.112:0/3012130567 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6847+0+0 (secure 0 0 0) 0x7f97c400f020 con 0x7f97cc1071c0 2024-05-08T05:49:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.919+0000 7f97cb400700 1 --2- 172.21.15.112:0/3012130567 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f97b00b2900 0x7f97b00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:53.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.920+0000 7f97cb400700 1 --2- 172.21.15.112:0/3012130567 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f97b00b2900 0x7f97b00b4dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f97cc106680 tx=0x7f97bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:53.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:53.921+0000 7f97c3e00700 1 -- 172.21.15.112:0/3012130567 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f97c4029020 con 0x7f97cc1071c0 2024-05-08T05:49:54.105 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:54 smithi162 ceph-mon[34800]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T05:49:54.105 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:54 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:49:54.105 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:54 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:49:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:54.158+0000 7f97d1000700 1 -- 172.21.15.112:0/3012130567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f97cc02e510 con 0x7f97cc1071c0 2024-05-08T05:49:54.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:54.159+0000 7f97c3e00700 1 -- 172.21.15.112:0/3012130567 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f97c4009cb0 con 0x7f97cc1071c0 2024-05-08T05:49:54.160 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:49:54.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:54.164+0000 7f97d1000700 1 -- 172.21.15.112:0/3012130567 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f97b00b2900 msgr2=0x7f97b00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:54.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:54.164+0000 7f97d1000700 1 --2- 172.21.15.112:0/3012130567 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f97b00b2900 0x7f97b00b4dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f97cc106680 tx=0x7f97bc009260 comp rx=0 tx=0).stop 2024-05-08T05:49:54.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:54.164+0000 7f97d1000700 1 -- 172.21.15.112:0/3012130567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97cc1071c0 msgr2=0x7f97cc1a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:54.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:54.164+0000 7f97d1000700 1 --2- 172.21.15.112:0/3012130567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97cc1071c0 0x7f97cc1a2c50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f97c4004ce0 tx=0x7f97c4004d10 comp rx=0 tx=0).stop 2024-05-08T05:49:54.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:54.164+0000 7f97d1000700 1 -- 172.21.15.112:0/3012130567 shutdown_connections 2024-05-08T05:49:54.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:54.164+0000 7f97d1000700 1 --2- 172.21.15.112:0/3012130567 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f97b00b2900 0x7f97b00b4dc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:54.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:54.164+0000 7f97d1000700 1 --2- 172.21.15.112:0/3012130567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97cc1071c0 0x7f97cc1a2c50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:54.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:54.164+0000 7f97d1000700 1 --2- 172.21.15.112:0/3012130567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97cc105620 0x7f97cc19b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:54.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:54.164+0000 7f97d1000700 1 --2- 172.21.15.112:0/3012130567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97cc106820 0x7f97cc19bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:54.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:54.164+0000 7f97d1000700 1 -- 172.21.15.112:0/3012130567 >> 172.21.15.112:0/3012130567 conn(0x7f97cc100bc0 msgr2=0x7f97cc109b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:54.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:54.165+0000 7f97d1000700 1 -- 172.21.15.112:0/3012130567 shutdown_connections 2024-05-08T05:49:54.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:54.165+0000 7f97d1000700 1 -- 172.21.15.112:0/3012130567 wait complete. 2024-05-08T05:49:54.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:54 smithi112 ceph-mon[30003]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T05:49:54.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:54 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:49:54.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:54 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:49:54.334 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:54 smithi150 ceph-mon[34315]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T05:49:54.334 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:54 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:49:54.334 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:54 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:49:55.086 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1715147394,"num_in_osds":12,"osd_in_since":1715147374,"num_remapped_pgs":0} 2024-05-08T05:49:55.087 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:49:55.153 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: purged_snaps scrub starts 2024-05-08T05:49:55.154 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: purged_snaps scrub ok 2024-05-08T05:49:55.154 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:55.154 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: osd.11 [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331] boot 2024-05-08T05:49:55.154 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: osdmap e72: 12 total, 12 up, 12 in 2024-05-08T05:49:55.154 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:49:55.154 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: Detected new or changed devices on smithi162 2024-05-08T05:49:55.154 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3012130567' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T05:49:55.154 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:55.154 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:55.154 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:55.154 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:55.154 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:55.154 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:55.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: Adjusting osd_memory_target on smithi162 to 4272M 2024-05-08T05:49:55.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:55.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:55.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:55.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:55.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: purged_snaps scrub starts 2024-05-08T05:49:55.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: purged_snaps scrub ok 2024-05-08T05:49:55.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:55.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: osd.11 [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331] boot 2024-05-08T05:49:55.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: osdmap e72: 12 total, 12 up, 12 in 2024-05-08T05:49:55.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:49:55.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: Detected new or changed devices on smithi162 2024-05-08T05:49:55.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3012130567' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T05:49:55.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:55.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:55.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:55.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:55.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:55.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:55.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: Adjusting osd_memory_target on smithi162 to 4272M 2024-05-08T05:49:55.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:55.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:55.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:55.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:55.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: purged_snaps scrub starts 2024-05-08T05:49:55.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: purged_snaps scrub ok 2024-05-08T05:49:55.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:49:55.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: osd.11 [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331] boot 2024-05-08T05:49:55.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: osdmap e72: 12 total, 12 up, 12 in 2024-05-08T05:49:55.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:49:55.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: Detected new or changed devices on smithi162 2024-05-08T05:49:55.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3012130567' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T05:49:55.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:55.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:55.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:55.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:55.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:55.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:49:55.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: Adjusting osd_memory_target on smithi162 to 4272M 2024-05-08T05:49:55.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:55.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:55.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:55.435 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:49:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:56.193 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:56 smithi112 ceph-mon[30003]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T05:49:56.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:56 smithi150 ceph-mon[34315]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T05:49:56.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:56 smithi162 ceph-mon[34800]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T05:49:57.231 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:57 smithi112 ceph-mon[30003]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:57.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:57 smithi150 ceph-mon[34315]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:57.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:57 smithi162 ceph-mon[34800]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.731+0000 7f989e000700 1 -- 172.21.15.112:0/3033265413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98981099e0 msgr2=0x7f9898109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.731+0000 7f989e000700 1 --2- 172.21.15.112:0/3033265413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98981099e0 0x7f9898109e60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9884009990 tx=0x7f9884009ca0 comp rx=0 tx=0).stop 2024-05-08T05:49:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.731+0000 7f989e000700 1 -- 172.21.15.112:0/3033265413 shutdown_connections 2024-05-08T05:49:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.731+0000 7f989e000700 1 --2- 172.21.15.112:0/3033265413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f989810a3a0 0x7f9898116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.731+0000 7f989e000700 1 --2- 172.21.15.112:0/3033265413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f989810e8b0 0x7f989810ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.731+0000 7f989e000700 1 --2- 172.21.15.112:0/3033265413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98981099e0 0x7f9898109e60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.731+0000 7f989e000700 1 -- 172.21.15.112:0/3033265413 >> 172.21.15.112:0/3033265413 conn(0x7f9898079230 msgr2=0x7f98981082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:57.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.731+0000 7f989e000700 1 -- 172.21.15.112:0/3033265413 shutdown_connections 2024-05-08T05:49:57.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.731+0000 7f989e000700 1 -- 172.21.15.112:0/3033265413 wait complete. 2024-05-08T05:49:57.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.732+0000 7f989e000700 1 Processor -- start 2024-05-08T05:49:57.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.732+0000 7f989e000700 1 -- start start 2024-05-08T05:49:57.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.733+0000 7f989e000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98981099e0 0x7f98981a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:57.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.733+0000 7f989e000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989810a3a0 0x7f98981a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:57.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.733+0000 7f989e000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f989810e8b0 0x7f98981a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:57.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.733+0000 7f989e000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9898119be0 con 0x7f989810a3a0 2024-05-08T05:49:57.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.733+0000 7f989e000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9898119a60 con 0x7f98981099e0 2024-05-08T05:49:57.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.733+0000 7f989e000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9898119d60 con 0x7f989810e8b0 2024-05-08T05:49:57.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.733+0000 7f9897e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989810a3a0 0x7f98981a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:57.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.733+0000 7f9897e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989810a3a0 0x7f98981a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47218/0 (socket says 172.21.15.112:47218) 2024-05-08T05:49:57.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.733+0000 7f9897e00700 1 -- 172.21.15.112:0/3491060371 learned_addr learned my addr 172.21.15.112:0/3491060371 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:49:57.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.733+0000 7f989cc00700 1 --2- 172.21.15.112:0/3491060371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98981099e0 0x7f98981a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.733+0000 7f989d600700 1 --2- 172.21.15.112:0/3491060371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f989810e8b0 0x7f98981a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.734+0000 7f9897e00700 1 -- 172.21.15.112:0/3491060371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f989810e8b0 msgr2=0x7f98981a4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.734+0000 7f9897e00700 1 --2- 172.21.15.112:0/3491060371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f989810e8b0 0x7f98981a4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.734+0000 7f9897e00700 1 -- 172.21.15.112:0/3491060371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98981099e0 msgr2=0x7f98981a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.734+0000 7f9897e00700 1 --2- 172.21.15.112:0/3491060371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98981099e0 0x7f98981a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.734+0000 7f9897e00700 1 -- 172.21.15.112:0/3491060371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98981a50d0 con 0x7f989810a3a0 2024-05-08T05:49:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.734+0000 7f9897e00700 1 --2- 172.21.15.112:0/3491060371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989810a3a0 0x7f98981a0650 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9884005910 tx=0x7f988400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.734+0000 7f9895600700 1 -- 172.21.15.112:0/3491060371 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884027070 con 0x7f989810a3a0 2024-05-08T05:49:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.734+0000 7f9895600700 1 -- 172.21.15.112:0/3491060371 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9884022470 con 0x7f989810a3a0 2024-05-08T05:49:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.734+0000 7f989e000700 1 -- 172.21.15.112:0/3491060371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98981a5360 con 0x7f989810a3a0 2024-05-08T05:49:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.735+0000 7f9895600700 1 -- 172.21.15.112:0/3491060371 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884017650 con 0x7f989810a3a0 2024-05-08T05:49:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.735+0000 7f989e000700 1 -- 172.21.15.112:0/3491060371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98981acc40 con 0x7f989810a3a0 2024-05-08T05:49:57.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.736+0000 7f989e000700 1 -- 172.21.15.112:0/3491060371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f989806a190 con 0x7f989810a3a0 2024-05-08T05:49:57.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.738+0000 7f9895600700 1 -- 172.21.15.112:0/3491060371 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f988402a070 con 0x7f989810a3a0 2024-05-08T05:49:57.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.739+0000 7f9895600700 1 --2- 172.21.15.112:0/3491060371 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f98800b2880 0x7f98800b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:57.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.740+0000 7f9895600700 1 -- 172.21.15.112:0/3491060371 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f98840e2b00 con 0x7f989810a3a0 2024-05-08T05:49:57.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.740+0000 7f989cc00700 1 --2- 172.21.15.112:0/3491060371 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f98800b2880 0x7f98800b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:57.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.740+0000 7f989cc00700 1 --2- 172.21.15.112:0/3491060371 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f98800b2880 0x7f98800b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f988c007b60 tx=0x7f988c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:57.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.744+0000 7f9895600700 1 -- 172.21.15.112:0/3491060371 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9884013070 con 0x7f989810a3a0 2024-05-08T05:49:57.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.975+0000 7f989e000700 1 -- 172.21.15.112:0/3491060371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f989802e510 con 0x7f989810a3a0 2024-05-08T05:49:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.976+0000 7f9895600700 1 -- 172.21.15.112:0/3491060371 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18524 (secure 0 0 0) 0x7f98840aacb0 con 0x7f989810a3a0 2024-05-08T05:49:57.978 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:49:57.978 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":73,"fsid":"7a07ff60-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:48.498871+0000","modified":"2024-05-08T05:49:55.049566+0000","last_up_change":"2024-05-08T05:49:54.039380+0000","last_in_change":"2024-05-08T05:49:34.443850+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:47.485105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b378542-1bbe-4614-8805-8a6b875ada55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6803","nonce":3419091820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6805","nonce":3419091820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6809","nonce":3419091820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6807","nonce":3419091820}]},"public_addr":"172.21.15.112:6803/3419091820","cluster_addr":"172.21.15.112:6805/3419091820","heartbeat_back_addr":"172.21.15.112:6809/3419091820","heartbeat_front_addr":"172.21.15.112:6807/3419091820","state":["exists","up"]},{"osd":1,"uuid":"826ef036-2db7-470e-9956-d9a58b9181e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6801","nonce":2728419797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6803","nonce":2728419797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6807","nonce":2728419797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6805","nonce":2728419797}]},"public_addr":"172.21.15.150:6801/2728419797","cluster_addr":"172.21.15.150:6803/2728419797","heartbeat_back_addr":"172.21.15.150:6807/2728419797","heartbeat_front_addr":"172.21.15.150:6805/2728419797","state":["exists","up"]},{"osd":2,"uuid":"32e710e7-2a87-4863-bc7a-906d31aa4ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6801","nonce":1556203215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6803","nonce":1556203215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6807","nonce":1556203215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6805","nonce":1556203215}]},"public_addr":"172.21.15.162:6801/1556203215","cluster_addr":"172.21.15.162:6803/1556203215","heartbeat_back_addr":"172.21.15.162:6807/1556203215","heartbeat_front_addr":"172.21.15.162:6805/1556203215","state":["exists","up"]},{"osd":3,"uuid":"ba1d4a5f-6a95-4b9e-8a48-8524a899b004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6811","nonce":3325718755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6813","nonce":3325718755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6817","nonce":3325718755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6815","nonce":3325718755}]},"public_addr":"172.21.15.112:6811/3325718755","cluster_addr":"172.21.15.112:6813/3325718755","heartbeat_back_addr":"172.21.15.112:6817/3325718755","heartbeat_front_addr":"172.21.15.112:6815/3325718755","state":["exists","up"]},{"osd":4,"uuid":"cb4f1a42-8de8-4b50-b3bc-602de9f4803a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6809","nonce":1548914683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6811","nonce":1548914683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6815","nonce":1548914683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6813","nonce":1548914683}]},"public_addr":"172.21.15.150:6809/1548914683","cluster_addr":"172.21.15.150:6811/1548914683","heartbeat_back_addr":"172.21.15.150:6815/1548914683","heartbeat_front_addr":"172.21.15.150:6813/1548914683","state":["exists","up"]},{"osd":5,"uuid":"a26c03b8-050c-4a2c-8f23-5801fffb7c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6809","nonce":409836750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6811","nonce":409836750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6815","nonce":409836750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6813","nonce":409836750}]},"public_addr":"172.21.15.162:6809/409836750","cluster_addr":"172.21.15.162:6811/409836750","heartbeat_back_addr":"172.21.15.162:6815/409836750","heartbeat_front_addr":"172.21.15.162:6813/409836750","state":["exists","up"]},{"osd":6,"uuid":"8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6819","nonce":1098984186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6821","nonce":1098984186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6825","nonce":1098984186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6823","nonce":1098984186}]},"public_addr":"172.21.15.112:6819/1098984186","cluster_addr":"172.21.15.112:6821/1098984186","heartbeat_back_addr":"172.21.15.112:6825/1098984186","heartbeat_front_addr":"172.21.15.112:6823/1098984186","state":["exists","up"]},{"osd":7,"uuid":"af3629b4-9bed-4669-97a6-48deba503a96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6817","nonce":4009836564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6819","nonce":4009836564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6823","nonce":4009836564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6821","nonce":4009836564}]},"public_addr":"172.21.15.150:6817/4009836564","cluster_addr":"172.21.15.150:6819/4009836564","heartbeat_back_addr":"172.21.15.150:6823/4009836564","heartbeat_front_addr":"172.21.15.150:6821/4009836564","state":["exists","up"]},{"osd":8,"uuid":"9333e87d-dd78-4467-bbde-c2b44f4ea07f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6817","nonce":852791606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6819","nonce":852791606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6823","nonce":852791606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6821","nonce":852791606}]},"public_addr":"172.21.15.162:6817/852791606","cluster_addr":"172.21.15.162:6819/852791606","heartbeat_back_addr":"172.21.15.162:6823/852791606","heartbeat_front_addr":"172.21.15.162:6821/852791606","state":["exists","up"]},{"osd":9,"uuid":"09723581-1995-42e0-8db7-776ea98a918c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6827","nonce":2186764674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6829","nonce":2186764674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6833","nonce":2186764674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6831","nonce":2186764674}]},"public_addr":"172.21.15.112:6827/2186764674","cluster_addr":"172.21.15.112:6829/2186764674","heartbeat_back_addr":"172.21.15.112:6833/2186764674","heartbeat_front_addr":"172.21.15.112:6831/2186764674","state":["exists","up"]},{"osd":10,"uuid":"34e08e3e-1231-4899-a5b3-76d449105bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6825","nonce":2838995474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6827","nonce":2838995474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6831","nonce":2838995474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6829","nonce":2838995474}]},"public_addr":"172.21.15.150:6825/2838995474","cluster_addr":"172.21.15.150:6827/2838995474","heartbeat_back_addr":"172.21.15.150:6831/2838995474","heartbeat_front_addr":"172.21.15.150:6829/2838995474","state":["exists","up"]},{"osd":11,"uuid":"6fc34ec0-78ca-403c-acb7-b29d83ff64e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6825","nonce":3702614331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6827","nonce":3702614331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6831","nonce":3702614331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6829","nonce":3702614331}]},"public_addr":"172.21.15.162:6825/3702614331","cluster_addr":"172.21.15.162:6827/3702614331","heartbeat_back_addr":"172.21.15.162:6831/3702614331","heartbeat_front_addr":"172.21.15.162:6829/3702614331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:54.518702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:19.939543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:44.425586+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:16.126471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:41.696071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:05.517405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:38.549318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:03.799649+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:28.007282+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:01.395459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:27.963637+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:52.181212+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3667148497":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6800/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3425327158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1404611205":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3765334043":"2024-05-09T05:42:29.819307+0000","172.21.15.112:6800/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/2814303790":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1091751583":"2024-05-09T05:42:29.819307+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:49:57.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.980+0000 7f989e000700 1 -- 172.21.15.112:0/3491060371 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f98800b2880 msgr2=0x7f98800b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:57.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.980+0000 7f989e000700 1 --2- 172.21.15.112:0/3491060371 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f98800b2880 0x7f98800b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f988c007b60 tx=0x7f988c0073d0 comp rx=0 tx=0).stop 2024-05-08T05:49:57.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.980+0000 7f989e000700 1 -- 172.21.15.112:0/3491060371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989810a3a0 msgr2=0x7f98981a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:57.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.980+0000 7f989e000700 1 --2- 172.21.15.112:0/3491060371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989810a3a0 0x7f98981a0650 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9884005910 tx=0x7f988400b670 comp rx=0 tx=0).stop 2024-05-08T05:49:57.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.981+0000 7f989e000700 1 -- 172.21.15.112:0/3491060371 shutdown_connections 2024-05-08T05:49:57.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.981+0000 7f989e000700 1 --2- 172.21.15.112:0/3491060371 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f98800b2880 0x7f98800b4d40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.981+0000 7f989e000700 1 --2- 172.21.15.112:0/3491060371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f989810e8b0 0x7f98981a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.981+0000 7f989e000700 1 --2- 172.21.15.112:0/3491060371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98981099e0 0x7f98981a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.981+0000 7f989e000700 1 --2- 172.21.15.112:0/3491060371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f989810a3a0 0x7f98981a0650 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.981+0000 7f989e000700 1 -- 172.21.15.112:0/3491060371 >> 172.21.15.112:0/3491060371 conn(0x7f9898079230 msgr2=0x7f9898079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.981+0000 7f989e000700 1 -- 172.21.15.112:0/3491060371 shutdown_connections 2024-05-08T05:49:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:49:57.981+0000 7f989e000700 1 -- 172.21.15.112:0/3491060371 wait complete. 2024-05-08T05:49:58.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:58 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3491060371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:49:58.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:58 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3491060371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:49:58.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:58 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3491060371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:49:58.858 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T05:45:47.485105+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T05:49:58.859 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-08T05:49:59.290 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:59.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:49:59 smithi150 ceph-mon[34315]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:59.438 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:49:59 smithi112 ceph-mon[30003]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:59.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:49:59 smithi162 ceph-mon[34800]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:00.350 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:00 smithi112 ceph-mon[30003]: overall HEALTH_OK 2024-05-08T05:50:00.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:00 smithi150 ceph-mon[34315]: overall HEALTH_OK 2024-05-08T05:50:00.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:00 smithi162 ceph-mon[34800]: overall HEALTH_OK 2024-05-08T05:50:01.217 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:01 smithi112 ceph-mon[30003]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:01.377 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:01 smithi162 ceph-mon[34800]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:01.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:01 smithi150 ceph-mon[34315]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:01.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.449+0000 7fcfbba00700 1 -- 172.21.15.112:0/3089596030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfb4106050 msgr2=0x7fcfb4112560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:01.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.449+0000 7fcfbba00700 1 --2- 172.21.15.112:0/3089596030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfb4106050 0x7fcfb4112560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00b0a0 tx=0x7fcfac00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.449+0000 7fcfbba00700 1 -- 172.21.15.112:0/3089596030 shutdown_connections 2024-05-08T05:50:01.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.449+0000 7fcfbba00700 1 --2- 172.21.15.112:0/3089596030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfb4106050 0x7fcfb4112560 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.449+0000 7fcfbba00700 1 --2- 172.21.15.112:0/3089596030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfb410b690 0x7fcfb410ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.449+0000 7fcfbba00700 1 --2- 172.21.15.112:0/3089596030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb4105690 0x7fcfb4105b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.449+0000 7fcfbba00700 1 -- 172.21.15.112:0/3089596030 >> 172.21.15.112:0/3089596030 conn(0x7fcfb4101180 msgr2=0x7fcfb41035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:01.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.449+0000 7fcfbba00700 1 -- 172.21.15.112:0/3089596030 shutdown_connections 2024-05-08T05:50:01.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.449+0000 7fcfbba00700 1 -- 172.21.15.112:0/3089596030 wait complete. 2024-05-08T05:50:01.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.450+0000 7fcfbba00700 1 Processor -- start 2024-05-08T05:50:01.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.451+0000 7fcfbba00700 1 -- start start 2024-05-08T05:50:01.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.451+0000 7fcfbba00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfb4105690 0x7fcfb419bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:01.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.451+0000 7fcfbba00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfb4106050 0x7fcfb419c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:01.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.451+0000 7fcfbba00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb410b690 0x7fcfb41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:01.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.451+0000 7fcfbba00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41158d0 con 0x7fcfb4106050 2024-05-08T05:50:01.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.451+0000 7fcfbba00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4115750 con 0x7fcfb410b690 2024-05-08T05:50:01.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.451+0000 7fcfbba00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4115a50 con 0x7fcfb4105690 2024-05-08T05:50:01.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.451+0000 7fcfb3e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfb4106050 0x7fcfb419c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:01.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.451+0000 7fcfb9000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfb4105690 0x7fcfb419bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.451+0000 7fcfb9000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfb4105690 0x7fcfb419bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:35500/0 (socket says 172.21.15.112:35500) 2024-05-08T05:50:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.451+0000 7fcfb3e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfb4106050 0x7fcfb419c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47234/0 (socket says 172.21.15.112:47234) 2024-05-08T05:50:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.451+0000 7fcfb9000700 1 -- 172.21.15.112:0/883492579 learned_addr learned my addr 172.21.15.112:0/883492579 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:50:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.451+0000 7fcfb9a00700 1 --2- 172.21.15.112:0/883492579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb410b690 0x7fcfb41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.452+0000 7fcfb9000700 1 -- 172.21.15.112:0/883492579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb410b690 msgr2=0x7fcfb41a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.452+0000 7fcfb9000700 1 --2- 172.21.15.112:0/883492579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb410b690 0x7fcfb41a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.452+0000 7fcfb9000700 1 -- 172.21.15.112:0/883492579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfb4106050 msgr2=0x7fcfb419c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.452+0000 7fcfb9000700 1 --2- 172.21.15.112:0/883492579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfb4106050 0x7fcfb419c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.452+0000 7fcfb9000700 1 -- 172.21.15.112:0/883492579 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb41a0d50 con 0x7fcfb4105690 2024-05-08T05:50:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.452+0000 7fcfb9a00700 1 --2- 172.21.15.112:0/883492579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb410b690 0x7fcfb41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.452+0000 7fcfb3e00700 1 --2- 172.21.15.112:0/883492579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfb4106050 0x7fcfb419c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.452+0000 7fcfb9000700 1 --2- 172.21.15.112:0/883492579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfb4105690 0x7fcfb419bd40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800e9c0 tx=0x7fcfa800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:01.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.452+0000 7fcfb1600700 1 -- 172.21.15.112:0/883492579 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa800cbd0 con 0x7fcfb4105690 2024-05-08T05:50:01.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.452+0000 7fcfb1600700 1 -- 172.21.15.112:0/883492579 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa800ee60 con 0x7fcfb4105690 2024-05-08T05:50:01.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.452+0000 7fcfbba00700 1 -- 172.21.15.112:0/883492579 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb41a1040 con 0x7fcfb4105690 2024-05-08T05:50:01.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.452+0000 7fcfb1600700 1 -- 172.21.15.112:0/883492579 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80055c0 con 0x7fcfb4105690 2024-05-08T05:50:01.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.453+0000 7fcfbba00700 1 -- 172.21.15.112:0/883492579 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb41a88c0 con 0x7fcfb4105690 2024-05-08T05:50:01.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.454+0000 7fcfbba00700 1 -- 172.21.15.112:0/883492579 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb406a190 con 0x7fcfb4105690 2024-05-08T05:50:01.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.456+0000 7fcfb1600700 1 -- 172.21.15.112:0/883492579 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fcfa8004750 con 0x7fcfb4105690 2024-05-08T05:50:01.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.457+0000 7fcfb1600700 1 --2- 172.21.15.112:0/883492579 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcf9c0b2880 0x7fcf9c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:01.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.458+0000 7fcfb1600700 1 -- 172.21.15.112:0/883492579 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fcfa80e2050 con 0x7fcfb4105690 2024-05-08T05:50:01.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.458+0000 7fcfb3e00700 1 --2- 172.21.15.112:0/883492579 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcf9c0b2880 0x7fcf9c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:01.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.458+0000 7fcfb3e00700 1 --2- 172.21.15.112:0/883492579 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcf9c0b2880 0x7fcf9c0b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009680 tx=0x7fcfa0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:01.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.461+0000 7fcfb1600700 1 -- 172.21.15.112:0/883492579 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcfa80aa170 con 0x7fcfb4105690 2024-05-08T05:50:01.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.693+0000 7fcfbba00700 1 -- 172.21.15.112:0/883492579 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcfb419d0a0 con 0x7fcfb4105690 2024-05-08T05:50:01.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.693+0000 7fcfb1600700 1 -- 172.21.15.112:0/883492579 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7fcfa80a9a70 con 0x7fcfb4105690 2024-05-08T05:50:01.694 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-05-08T05:50:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.696+0000 7fcfbba00700 1 -- 172.21.15.112:0/883492579 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcf9c0b2880 msgr2=0x7fcf9c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.696+0000 7fcfbba00700 1 --2- 172.21.15.112:0/883492579 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcf9c0b2880 0x7fcf9c0b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009680 tx=0x7fcfa0009260 comp rx=0 tx=0).stop 2024-05-08T05:50:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.696+0000 7fcfbba00700 1 -- 172.21.15.112:0/883492579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfb4105690 msgr2=0x7fcfb419bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.696+0000 7fcfbba00700 1 --2- 172.21.15.112:0/883492579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfb4105690 0x7fcfb419bd40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800e9c0 tx=0x7fcfa800ecd0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.697+0000 7fcfbba00700 1 -- 172.21.15.112:0/883492579 shutdown_connections 2024-05-08T05:50:01.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.697+0000 7fcfbba00700 1 --2- 172.21.15.112:0/883492579 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcf9c0b2880 0x7fcf9c0b4d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.697+0000 7fcfbba00700 1 --2- 172.21.15.112:0/883492579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb410b690 0x7fcfb41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.697+0000 7fcfbba00700 1 --2- 172.21.15.112:0/883492579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfb4105690 0x7fcfb419bd40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.697+0000 7fcfbba00700 1 --2- 172.21.15.112:0/883492579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfb4106050 0x7fcfb419c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.697+0000 7fcfbba00700 1 -- 172.21.15.112:0/883492579 >> 172.21.15.112:0/883492579 conn(0x7fcfb4101180 msgr2=0x7fcfb41016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:01.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.697+0000 7fcfbba00700 1 -- 172.21.15.112:0/883492579 shutdown_connections 2024-05-08T05:50:01.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:01.697+0000 7fcfbba00700 1 -- 172.21.15.112:0/883492579 wait complete. 2024-05-08T05:50:02.418 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:02 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/883492579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T05:50:02.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:02 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/883492579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T05:50:02.488 INFO:tasks.cephadm:Adding mds.a on smithi112 2024-05-08T05:50:02.488 INFO:tasks.cephadm:Adding mds.d on smithi112 2024-05-08T05:50:02.488 INFO:tasks.cephadm:Adding mds.g on smithi112 2024-05-08T05:50:02.488 INFO:tasks.cephadm:Adding mds.j on smithi112 2024-05-08T05:50:02.488 INFO:tasks.cephadm:Adding mds.b on smithi150 2024-05-08T05:50:02.488 INFO:tasks.cephadm:Adding mds.e on smithi150 2024-05-08T05:50:02.488 INFO:tasks.cephadm:Adding mds.h on smithi150 2024-05-08T05:50:02.488 INFO:tasks.cephadm:Adding mds.k on smithi150 2024-05-08T05:50:02.488 INFO:tasks.cephadm:Adding mds.c on smithi162 2024-05-08T05:50:02.488 INFO:tasks.cephadm:Adding mds.f on smithi162 2024-05-08T05:50:02.488 INFO:tasks.cephadm:Adding mds.i on smithi162 2024-05-08T05:50:02.488 INFO:tasks.cephadm:Adding mds.l on smithi162 2024-05-08T05:50:02.489 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l' 2024-05-08T05:50:02.517 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:02 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/883492579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T05:50:02.740 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:50:03.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:03 smithi150 ceph-mon[34315]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:03.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:03 smithi112 ceph-mon[30003]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:03.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:03 smithi162 ceph-mon[34800]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:04.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.183+0000 7f9d04219700 1 -- 172.21.15.162:0/3373318671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cfc1039f0 msgr2=0x7f9cfc103e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:04.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.183+0000 7f9d04219700 1 --2- 172.21.15.162:0/3373318671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cfc1039f0 0x7f9cfc103e10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009990 tx=0x7f9cf8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:50:04.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.183+0000 7f9d04219700 1 -- 172.21.15.162:0/3373318671 shutdown_connections 2024-05-08T05:50:04.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.183+0000 7f9d04219700 1 --2- 172.21.15.162:0/3373318671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cfc103040 0x7f9cfc10a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:04.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.183+0000 7f9d04219700 1 --2- 172.21.15.162:0/3373318671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cfc1039f0 0x7f9cfc103e10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:04.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.183+0000 7f9d04219700 1 --2- 172.21.15.162:0/3373318671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cfc1026a0 0x7f9cfc102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:04.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.183+0000 7f9d04219700 1 -- 172.21.15.162:0/3373318671 >> 172.21.15.162:0/3373318671 conn(0x7f9cfc0fd110 msgr2=0x7f9cfc0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:04.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.183+0000 7f9d04219700 1 -- 172.21.15.162:0/3373318671 shutdown_connections 2024-05-08T05:50:04.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.183+0000 7f9d04219700 1 -- 172.21.15.162:0/3373318671 wait complete. 2024-05-08T05:50:04.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.183+0000 7f9d04219700 1 Processor -- start 2024-05-08T05:50:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.183+0000 7f9d04219700 1 -- start start 2024-05-08T05:50:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d04219700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cfc1026a0 0x7f9cfc102150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d04219700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cfc103040 0x7f9cfc19e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d04219700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cfc1039f0 0x7f9cfc1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d04219700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc10df10 con 0x7f9cfc103040 2024-05-08T05:50:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d04219700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc10dd90 con 0x7f9cfc1026a0 2024-05-08T05:50:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d04219700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc10e090 con 0x7f9cfc1039f0 2024-05-08T05:50:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d027b6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cfc1039f0 0x7f9cfc1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d027b6700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cfc1039f0 0x7f9cfc1a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51450/0 (socket says 172.21.15.162:51450) 2024-05-08T05:50:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d027b6700 1 -- 172.21.15.162:0/2148901680 learned_addr learned my addr 172.21.15.162:0/2148901680 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-08T05:50:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d01fb5700 1 --2- 172.21.15.162:0/2148901680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cfc1026a0 0x7f9cfc102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d027b6700 1 -- 172.21.15.162:0/2148901680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cfc1026a0 msgr2=0x7f9cfc102150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d027b6700 1 --2- 172.21.15.162:0/2148901680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cfc1026a0 0x7f9cfc102150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:04.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d027b6700 1 -- 172.21.15.162:0/2148901680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cfc103040 msgr2=0x7f9cfc19e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:04.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d027b6700 1 --2- 172.21.15.162:0/2148901680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cfc103040 0x7f9cfc19e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:04.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d027b6700 1 -- 172.21.15.162:0/2148901680 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cfc1a1320 con 0x7f9cfc1039f0 2024-05-08T05:50:04.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d027b6700 1 --2- 172.21.15.162:0/2148901680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cfc1039f0 0x7f9cfc1a0a50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400b970 tx=0x7f9cf400bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:04.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9cf2ffd700 1 -- 172.21.15.162:0/2148901680 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4019070 con 0x7f9cfc1039f0 2024-05-08T05:50:04.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9cf2ffd700 1 -- 172.21.15.162:0/2148901680 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cf4014440 con 0x7f9cfc1039f0 2024-05-08T05:50:04.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d04219700 1 -- 172.21.15.162:0/2148901680 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cfc1a1610 con 0x7f9cfc1039f0 2024-05-08T05:50:04.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9cf2ffd700 1 -- 172.21.15.162:0/2148901680 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf400ce60 con 0x7f9cfc1039f0 2024-05-08T05:50:04.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.184+0000 7f9d04219700 1 -- 172.21.15.162:0/2148901680 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cfc1a1b80 con 0x7f9cfc1039f0 2024-05-08T05:50:04.188 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.185+0000 7f9d04219700 1 -- 172.21.15.162:0/2148901680 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cfc06a190 con 0x7f9cfc1039f0 2024-05-08T05:50:04.188 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.187+0000 7f9cf2ffd700 1 -- 172.21.15.162:0/2148901680 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f9cf401c070 con 0x7f9cfc1039f0 2024-05-08T05:50:04.188 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.187+0000 7f9cf2ffd700 1 --2- 172.21.15.162:0/2148901680 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9ce80b2880 0x7f9ce80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:04.188 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.187+0000 7f9cf2ffd700 1 -- 172.21.15.162:0/2148901680 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f9cf40e16b0 con 0x7f9cfc1039f0 2024-05-08T05:50:04.188 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.188+0000 7f9d01fb5700 1 --2- 172.21.15.162:0/2148901680 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9ce80b2880 0x7f9ce80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:04.188 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.189+0000 7f9d01fb5700 1 --2- 172.21.15.162:0/2148901680 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9ce80b2880 0x7f9ce80b4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800b2e0 tx=0x7f9cf801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:04.190 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.191+0000 7f9cf2ffd700 1 -- 172.21.15.162:0/2148901680 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9cf40a9870 con 0x7f9cfc1039f0 2024-05-08T05:50:04.403 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.403+0000 7f9d04219700 1 -- 172.21.15.162:0/2148901680 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l", "target": ["mon-mgr", ""]}) -- 0x7f9cfc063ae0 con 0x7f9ce80b2880 2024-05-08T05:50:04.421 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.421+0000 7f9cf2ffd700 1 -- 172.21.15.162:0/2148901680 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f9cfc063ae0 con 0x7f9ce80b2880 2024-05-08T05:50:04.421 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled mds.all update... 2024-05-08T05:50:04.424 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.423+0000 7f9d04219700 1 -- 172.21.15.162:0/2148901680 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9ce80b2880 msgr2=0x7f9ce80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:04.424 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.423+0000 7f9d04219700 1 --2- 172.21.15.162:0/2148901680 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9ce80b2880 0x7f9ce80b4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800b2e0 tx=0x7f9cf801a040 comp rx=0 tx=0).stop 2024-05-08T05:50:04.424 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.423+0000 7f9d04219700 1 -- 172.21.15.162:0/2148901680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cfc1039f0 msgr2=0x7f9cfc1a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:04.424 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.423+0000 7f9d04219700 1 --2- 172.21.15.162:0/2148901680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cfc1039f0 0x7f9cfc1a0a50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400b970 tx=0x7f9cf400bd30 comp rx=0 tx=0).stop 2024-05-08T05:50:04.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.423+0000 7f9d04219700 1 -- 172.21.15.162:0/2148901680 shutdown_connections 2024-05-08T05:50:04.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.423+0000 7f9d04219700 1 --2- 172.21.15.162:0/2148901680 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9ce80b2880 0x7f9ce80b4d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:04.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.423+0000 7f9d04219700 1 --2- 172.21.15.162:0/2148901680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cfc1039f0 0x7f9cfc1a0a50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:04.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.423+0000 7f9d04219700 1 --2- 172.21.15.162:0/2148901680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cfc1026a0 0x7f9cfc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:04.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.423+0000 7f9d04219700 1 --2- 172.21.15.162:0/2148901680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cfc103040 0x7f9cfc19e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:04.425 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.423+0000 7f9d04219700 1 -- 172.21.15.162:0/2148901680 >> 172.21.15.162:0/2148901680 conn(0x7f9cfc0fd110 msgr2=0x7f9cfc106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:04.426 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.423+0000 7f9d04219700 1 -- 172.21.15.162:0/2148901680 shutdown_connections 2024-05-08T05:50:04.426 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-08T05:50:04.423+0000 7f9d04219700 1 -- 172.21.15.162:0/2148901680 wait complete. 2024-05-08T05:50:04.979 DEBUG:teuthology.orchestra.run.smithi112:mds.a> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.a.service 2024-05-08T05:50:04.987 DEBUG:teuthology.orchestra.run.smithi112:mds.d> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.d.service 2024-05-08T05:50:05.033 DEBUG:teuthology.orchestra.run.smithi112:mds.g> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.g.service 2024-05-08T05:50:05.079 DEBUG:teuthology.orchestra.run.smithi112:mds.j> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.j.service 2024-05-08T05:50:05.125 DEBUG:teuthology.orchestra.run.smithi150:mds.b> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.b.service 2024-05-08T05:50:05.128 DEBUG:teuthology.orchestra.run.smithi150:mds.e> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.e.service 2024-05-08T05:50:05.171 DEBUG:teuthology.orchestra.run.smithi150:mds.h> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.h.service 2024-05-08T05:50:05.214 DEBUG:teuthology.orchestra.run.smithi150:mds.k> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.k.service 2024-05-08T05:50:05.258 DEBUG:teuthology.orchestra.run.smithi162:mds.c> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.c.service 2024-05-08T05:50:05.261 DEBUG:teuthology.orchestra.run.smithi162:mds.f> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.f.service 2024-05-08T05:50:05.305 DEBUG:teuthology.orchestra.run.smithi162:mds.i> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.i.service 2024-05-08T05:50:05.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:05 smithi112 ceph-mon[30003]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:05.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:05 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:05.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:05 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:50:05.348 DEBUG:teuthology.orchestra.run.smithi162:mds.l> sudo journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.l.service 2024-05-08T05:50:05.388 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:05 smithi162 ceph-mon[34800]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:05.388 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:05 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:05.388 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:05 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:50:05.389 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-08T05:50:05.390 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:50:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:05 smithi150 ceph-mon[34315]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:50:05.810 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:06.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:06 smithi112 ceph-mon[30003]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:50:06.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:06 smithi112 ceph-mon[30003]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l;count:12 2024-05-08T05:50:06.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:06 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:06.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:06 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:50:06.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:06 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:06.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:06 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:06.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:06 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:06.255 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:06 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:06.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:06 smithi150 ceph-mon[34315]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:50:06.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:06 smithi150 ceph-mon[34315]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l;count:12 2024-05-08T05:50:06.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:06 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:06.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:06 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:50:06.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:06 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:06.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:06 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:06.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:06 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:06.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:06 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:06.467 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:06 smithi162 ceph-mon[34800]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:50:06.468 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:06 smithi162 ceph-mon[34800]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l;count:12 2024-05-08T05:50:06.468 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:06 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:06.468 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:06 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:50:06.468 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:06 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:06.468 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:06 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:06.468 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:06 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:06.469 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:06 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:07.413 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:07 smithi162 ceph-mon[34800]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:07.413 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:07 smithi162 ceph-mon[34800]: Deploying daemon mds.l on smithi162 2024-05-08T05:50:07.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:07 smithi150 ceph-mon[34315]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:07.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:07 smithi150 ceph-mon[34315]: Deploying daemon mds.l on smithi162 2024-05-08T05:50:07.438 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:07 smithi112 ceph-mon[30003]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:07.438 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:07 smithi112 ceph-mon[30003]: Deploying daemon mds.l on smithi162 2024-05-08T05:50:07.816 INFO:journalctl@ceph.mds.l.smithi162.stdout:May 08 05:50:07 smithi162 systemd[1]: Starting Ceph mds.l for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:50:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.919+0000 7f114a400700 1 -- 172.21.15.112:0/4093103925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11441056c0 msgr2=0x7f1144105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.919+0000 7f114a400700 1 --2- 172.21.15.112:0/4093103925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11441056c0 0x7f1144105b40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f112c009990 tx=0x7f112c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.919+0000 7f114a400700 1 -- 172.21.15.112:0/4093103925 shutdown_connections 2024-05-08T05:50:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.919+0000 7f114a400700 1 --2- 172.21.15.112:0/4093103925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1144106080 0x7f1144112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.919+0000 7f114a400700 1 --2- 172.21.15.112:0/4093103925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f114410b6c0 0x7f114410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.919+0000 7f114a400700 1 --2- 172.21.15.112:0/4093103925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11441056c0 0x7f1144105b40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.919+0000 7f114a400700 1 -- 172.21.15.112:0/4093103925 >> 172.21.15.112:0/4093103925 conn(0x7f11441011b0 msgr2=0x7f11441035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.919+0000 7f114a400700 1 -- 172.21.15.112:0/4093103925 shutdown_connections 2024-05-08T05:50:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.919+0000 7f114a400700 1 -- 172.21.15.112:0/4093103925 wait complete. 2024-05-08T05:50:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.920+0000 7f114a400700 1 Processor -- start 2024-05-08T05:50:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.920+0000 7f114a400700 1 -- start start 2024-05-08T05:50:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.920+0000 7f114a400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11441056c0 0x7f114419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f114a400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1144106080 0x7f114419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f114a400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f114410b6c0 0x7f11441a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f114a400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1144115950 con 0x7f11441056c0 2024-05-08T05:50:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f114a400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f11441157d0 con 0x7f1144106080 2024-05-08T05:50:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f114a400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1144115ad0 con 0x7f114410b6c0 2024-05-08T05:50:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f1143400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11441056c0 0x7f114419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f1143e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f114410b6c0 0x7f11441a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f1143400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11441056c0 0x7f114419bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44864/0 (socket says 172.21.15.112:44864) 2024-05-08T05:50:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f1143e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f114410b6c0 0x7f11441a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50776/0 (socket says 172.21.15.112:50776) 2024-05-08T05:50:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f113aa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1144106080 0x7f114419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f1143400700 1 -- 172.21.15.112:0/1585149735 learned_addr learned my addr 172.21.15.112:0/1585149735 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:50:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f1143400700 1 -- 172.21.15.112:0/1585149735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f114410b6c0 msgr2=0x7f11441a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f1143400700 1 --2- 172.21.15.112:0/1585149735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f114410b6c0 0x7f11441a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f1143400700 1 -- 172.21.15.112:0/1585149735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1144106080 msgr2=0x7f114419c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f1143400700 1 --2- 172.21.15.112:0/1585149735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1144106080 0x7f114419c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f1143400700 1 -- 172.21.15.112:0/1585149735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11441a0db0 con 0x7f11441056c0 2024-05-08T05:50:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f1143e00700 1 --2- 172.21.15.112:0/1585149735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f114410b6c0 0x7f11441a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.921+0000 7f113aa00700 1 --2- 172.21.15.112:0/1585149735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1144106080 0x7f114419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.922+0000 7f1143400700 1 --2- 172.21.15.112:0/1585149735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11441056c0 0x7f114419bda0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f113400eb30 tx=0x7f113400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.922+0000 7f1140c00700 1 -- 172.21.15.112:0/1585149735 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113400cbd0 con 0x7f11441056c0 2024-05-08T05:50:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.922+0000 7f1140c00700 1 -- 172.21.15.112:0/1585149735 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11340044d0 con 0x7f11441056c0 2024-05-08T05:50:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.922+0000 7f114a400700 1 -- 172.21.15.112:0/1585149735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11441a10a0 con 0x7f11441056c0 2024-05-08T05:50:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.922+0000 7f1140c00700 1 -- 172.21.15.112:0/1585149735 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134003e30 con 0x7f11441056c0 2024-05-08T05:50:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.922+0000 7f114a400700 1 -- 172.21.15.112:0/1585149735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11441a8920 con 0x7f11441056c0 2024-05-08T05:50:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.923+0000 7f114a400700 1 -- 172.21.15.112:0/1585149735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f114406a190 con 0x7f11441056c0 2024-05-08T05:50:07.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.926+0000 7f1140c00700 1 -- 172.21.15.112:0/1585149735 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f1134004000 con 0x7f11441056c0 2024-05-08T05:50:07.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.927+0000 7f1140c00700 1 --2- 172.21.15.112:0/1585149735 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f11280b2880 0x7f11280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.927+0000 7f1140c00700 1 -- 172.21.15.112:0/1585149735 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f11340e2170 con 0x7f11441056c0 2024-05-08T05:50:07.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.927+0000 7f113aa00700 1 --2- 172.21.15.112:0/1585149735 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f11280b2880 0x7f11280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.928+0000 7f113aa00700 1 --2- 172.21.15.112:0/1585149735 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f11280b2880 0x7f11280b4d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1144103830 tx=0x7f112c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:07.932+0000 7f1140c00700 1 -- 172.21.15.112:0/1585149735 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f11340042f0 con 0x7f11441056c0 2024-05-08T05:50:08.067 INFO:journalctl@ceph.mds.l.smithi162.stdout:May 08 05:50:07 smithi162 podman[64624]: 2024-05-08 05:50:07.816094934 +0000 UTC m=+0.115308794 container create 0ecda3dd22a53e2730167bceb89be4d1e063c4da35e21af50ee5d953c602d602 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-l, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-05-08T05:50:08.067 INFO:journalctl@ceph.mds.l.smithi162.stdout:May 08 05:50:07 smithi162 podman[64624]: 2024-05-08 05:50:07.719042184 +0000 UTC m=+0.018256011 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:50:08.067 INFO:journalctl@ceph.mds.l.smithi162.stdout:May 08 05:50:08 smithi162 podman[64624]: 2024-05-08 05:50:08.036304252 +0000 UTC m=+0.335518068 container init 0ecda3dd22a53e2730167bceb89be4d1e063c4da35e21af50ee5d953c602d602 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-08T05:50:08.067 INFO:journalctl@ceph.mds.l.smithi162.stdout:May 08 05:50:08 smithi162 podman[64624]: 2024-05-08 05:50:08.038942355 +0000 UTC m=+0.338156166 container start 0ecda3dd22a53e2730167bceb89be4d1e063c4da35e21af50ee5d953c602d602 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-l, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-08T05:50:08.067 INFO:journalctl@ceph.mds.l.smithi162.stdout:May 08 05:50:08 smithi162 ceph-mds[64643]: -- 172.21.15.162:0/1654586782 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5571591eda40 con 0x5571591c1000 2024-05-08T05:50:08.067 INFO:journalctl@ceph.mds.l.smithi162.stdout:May 08 05:50:08 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-l[64639]: starting mds.l at 2024-05-08T05:50:08.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:08.221+0000 7f114a400700 1 -- 172.21.15.112:0/1585149735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f114419d100 con 0x7f11441056c0 2024-05-08T05:50:08.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:08.222+0000 7f1140c00700 1 -- 172.21.15.112:0/1585149735 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f11340aa290 con 0x7f11441056c0 2024-05-08T05:50:08.222 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:50:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-08T05:50:08.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:08.225+0000 7f114a400700 1 -- 172.21.15.112:0/1585149735 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f11280b2880 msgr2=0x7f11280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:08.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:08.225+0000 7f114a400700 1 --2- 172.21.15.112:0/1585149735 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f11280b2880 0x7f11280b4d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1144103830 tx=0x7f112c00b260 comp rx=0 tx=0).stop 2024-05-08T05:50:08.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:08.225+0000 7f114a400700 1 -- 172.21.15.112:0/1585149735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11441056c0 msgr2=0x7f114419bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:08.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:08.225+0000 7f114a400700 1 --2- 172.21.15.112:0/1585149735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11441056c0 0x7f114419bda0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f113400eb30 tx=0x7f113400ee40 comp rx=0 tx=0).stop 2024-05-08T05:50:08.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:08.225+0000 7f114a400700 1 -- 172.21.15.112:0/1585149735 shutdown_connections 2024-05-08T05:50:08.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:08.225+0000 7f114a400700 1 --2- 172.21.15.112:0/1585149735 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f11280b2880 0x7f11280b4d40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:08.225+0000 7f114a400700 1 --2- 172.21.15.112:0/1585149735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f114410b6c0 0x7f11441a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:08.225+0000 7f114a400700 1 --2- 172.21.15.112:0/1585149735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11441056c0 0x7f114419bda0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:08.225+0000 7f114a400700 1 --2- 172.21.15.112:0/1585149735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1144106080 0x7f114419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:08.225+0000 7f114a400700 1 -- 172.21.15.112:0/1585149735 >> 172.21.15.112:0/1585149735 conn(0x7f11441011b0 msgr2=0x7f11441019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:08.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:08.225+0000 7f114a400700 1 -- 172.21.15.112:0/1585149735 shutdown_connections 2024-05-08T05:50:08.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:08.225+0000 7f114a400700 1 -- 172.21.15.112:0/1585149735 wait complete. 2024-05-08T05:50:08.227 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 2 2024-05-08T05:50:08.566 INFO:journalctl@ceph.mds.l.smithi162.stdout:May 08 05:50:08 smithi162 bash[64624]: 0ecda3dd22a53e2730167bceb89be4d1e063c4da35e21af50ee5d953c602d602 2024-05-08T05:50:08.567 INFO:journalctl@ceph.mds.l.smithi162.stdout:May 08 05:50:08 smithi162 systemd[1]: Started Ceph mds.l for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:50:09.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:09 smithi150 ceph-mon[34315]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:09.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:09 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.162:6832/3668390959,v1:172.21.15.162:6833/3668390959] up:boot 2024-05-08T05:50:09.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:09 smithi150 ceph-mon[34315]: fsmap 1 up:standby 2024-05-08T05:50:09.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:09 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:50:09.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:09 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:09.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:09 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:09.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:09 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1585149735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:09.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:09 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:09.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:09 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:09.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:09 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:09.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:09 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:09.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:09 smithi150 ceph-mon[34315]: Deploying daemon mds.k on smithi150 2024-05-08T05:50:09.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:09 smithi112 ceph-mon[30003]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:09.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:09 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.162:6832/3668390959,v1:172.21.15.162:6833/3668390959] up:boot 2024-05-08T05:50:09.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:09 smithi112 ceph-mon[30003]: fsmap 1 up:standby 2024-05-08T05:50:09.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:09 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:50:09.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:09 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:09.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:09 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:09.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:09 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1585149735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:09.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:09 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:09.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:09 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:09.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:09 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:09.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:09 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:09.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:09 smithi112 ceph-mon[30003]: Deploying daemon mds.k on smithi150 2024-05-08T05:50:09.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:09 smithi162 ceph-mon[34800]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:09.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:09 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.162:6832/3668390959,v1:172.21.15.162:6833/3668390959] up:boot 2024-05-08T05:50:09.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:09 smithi162 ceph-mon[34800]: fsmap 1 up:standby 2024-05-08T05:50:09.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:09 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:50:09.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:09 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:09.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:09 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:09.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:09 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1585149735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:09.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:09 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:09.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:09 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:09.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:09 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:09.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:09 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:09.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:09 smithi162 ceph-mon[34800]: Deploying daemon mds.k on smithi150 2024-05-08T05:50:10.618 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 08 05:50:10 smithi150 systemd[1]: Starting Ceph mds.k for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:50:10.934 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 08 05:50:10 smithi150 podman[63209]: 2024-05-08 05:50:10.521721322 +0000 UTC m=+0.018140905 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:50:10.934 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 08 05:50:10 smithi150 podman[63209]: 2024-05-08 05:50:10.625062139 +0000 UTC m=+0.121481721 container create 16086b0e30d0de7f66a461aec29fdcb715537c7374f3200b9e2522db20f401c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-08T05:50:10.934 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 08 05:50:10 smithi150 podman[63209]: 2024-05-08 05:50:10.835629681 +0000 UTC m=+0.332049263 container init 16086b0e30d0de7f66a461aec29fdcb715537c7374f3200b9e2522db20f401c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-k, ceph=True, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-08T05:50:10.934 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 08 05:50:10 smithi150 podman[63209]: 2024-05-08 05:50:10.838267107 +0000 UTC m=+0.334686695 container start 16086b0e30d0de7f66a461aec29fdcb715537c7374f3200b9e2522db20f401c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-k, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-08T05:50:10.934 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 08 05:50:10 smithi150 ceph-mds[63228]: -- 172.21.15.150:0/311400170 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be3db87a40 con 0x55be3db5ac00 2024-05-08T05:50:10.934 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 08 05:50:10 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-k[63224]: starting mds.k at 2024-05-08T05:50:11.019 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:50:11.433 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 08 05:50:10 smithi150 bash[63209]: 16086b0e30d0de7f66a461aec29fdcb715537c7374f3200b9e2522db20f401c6 2024-05-08T05:50:11.433 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 08 05:50:10 smithi150 systemd[1]: Started Ceph mds.k for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:50:11.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:11 smithi150 ceph-mon[34315]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:11.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:11 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:11.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:11 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:11.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:11 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:11.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:11 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:11.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:11 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:11.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:11 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:11.463 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:11.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:11 smithi112 ceph-mon[30003]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:11.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:11 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:11.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:11 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:11.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:11 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:11.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:11 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:11.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:11 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:11.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:11 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:11.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:11 smithi162 ceph-mon[34800]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:11.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:11 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:11.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:11 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:11.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:11 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:11.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:11 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:11.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:11 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:11.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:11 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:12.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:12 smithi112 ceph-mon[30003]: Deploying daemon mds.j on smithi112 2024-05-08T05:50:12.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:12 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.150:6832/442303489,v1:172.21.15.150:6833/442303489] up:boot 2024-05-08T05:50:12.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:12 smithi112 ceph-mon[30003]: fsmap 2 up:standby 2024-05-08T05:50:12.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:12 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:50:12.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:12 smithi150 ceph-mon[34315]: Deploying daemon mds.j on smithi112 2024-05-08T05:50:12.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:12 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.150:6832/442303489,v1:172.21.15.150:6833/442303489] up:boot 2024-05-08T05:50:12.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:12 smithi150 ceph-mon[34315]: fsmap 2 up:standby 2024-05-08T05:50:12.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:12 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:50:12.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:12 smithi162 ceph-mon[34800]: Deploying daemon mds.j on smithi112 2024-05-08T05:50:12.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:12 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.150:6832/442303489,v1:172.21.15.150:6833/442303489] up:boot 2024-05-08T05:50:12.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:12 smithi162 ceph-mon[34800]: fsmap 2 up:standby 2024-05-08T05:50:12.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:12 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:50:13.271 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:13 smithi150 ceph-mon[34315]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:13.484 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:13 smithi112 ceph-mon[30003]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:13.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:13 smithi162 ceph-mon[34800]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:14.465 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:50:14.465 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-08T05:50:14.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.073+0000 7fbef7800700 1 -- 172.21.15.112:0/3256844585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef0075ff0 msgr2=0x7fbef010f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:14.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.073+0000 7fbef7800700 1 --2- 172.21.15.112:0/3256844585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef0075ff0 0x7fbef010f9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009990 tx=0x7fbed8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.073+0000 7fbef7800700 1 -- 172.21.15.112:0/3256844585 shutdown_connections 2024-05-08T05:50:14.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.073+0000 7fbef7800700 1 --2- 172.21.15.112:0/3256844585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbef010ff00 0x7fbef01122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.073+0000 7fbef7800700 1 --2- 172.21.15.112:0/3256844585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef00756d0 0x7fbef0075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.073+0000 7fbef7800700 1 --2- 172.21.15.112:0/3256844585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef0075ff0 0x7fbef010f9c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.073+0000 7fbef7800700 1 -- 172.21.15.112:0/3256844585 >> 172.21.15.112:0/3256844585 conn(0x7fbef007a600 msgr2=0x7fbef007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:14.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.073+0000 7fbef7800700 1 -- 172.21.15.112:0/3256844585 shutdown_connections 2024-05-08T05:50:14.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.073+0000 7fbef7800700 1 -- 172.21.15.112:0/3256844585 wait complete. 2024-05-08T05:50:14.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.074+0000 7fbef7800700 1 Processor -- start 2024-05-08T05:50:14.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.075+0000 7fbef7800700 1 -- start start 2024-05-08T05:50:14.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.075+0000 7fbef7800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef00756d0 0x7fbef01a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.075+0000 7fbef7800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbef0075ff0 0x7fbef01a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.075+0000 7fbef7800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef010ff00 0x7fbef019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.075+0000 7fbef7800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01158a0 con 0x7fbef00756d0 2024-05-08T05:50:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.075+0000 7fbef7800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbef0115720 con 0x7fbef010ff00 2024-05-08T05:50:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.075+0000 7fbef7800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbef0115a20 con 0x7fbef0075ff0 2024-05-08T05:50:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.075+0000 7fbef4e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef00756d0 0x7fbef01a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.076+0000 7fbef5800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef010ff00 0x7fbef019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.075+0000 7fbef4e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef00756d0 0x7fbef01a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47444/0 (socket says 172.21.15.112:47444) 2024-05-08T05:50:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.076+0000 7fbee7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbef0075ff0 0x7fbef01a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.076+0000 7fbef4e00700 1 -- 172.21.15.112:0/1929275957 learned_addr learned my addr 172.21.15.112:0/1929275957 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:50:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.076+0000 7fbef5800700 1 -- 172.21.15.112:0/1929275957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbef0075ff0 msgr2=0x7fbef01a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.076+0000 7fbef5800700 1 --2- 172.21.15.112:0/1929275957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbef0075ff0 0x7fbef01a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.076+0000 7fbef5800700 1 -- 172.21.15.112:0/1929275957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef00756d0 msgr2=0x7fbef01a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.076+0000 7fbef5800700 1 --2- 172.21.15.112:0/1929275957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef00756d0 0x7fbef01a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.076+0000 7fbef5800700 1 -- 172.21.15.112:0/1929275957 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef019c460 con 0x7fbef010ff00 2024-05-08T05:50:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.076+0000 7fbef4e00700 1 --2- 172.21.15.112:0/1929275957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef00756d0 0x7fbef01a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.077+0000 7fbef5800700 1 --2- 172.21.15.112:0/1929275957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef010ff00 0x7fbef019bb80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbee8007ee0 tx=0x7fbee800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.077+0000 7fbeee000700 1 -- 172.21.15.112:0/1929275957 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8004db0 con 0x7fbef010ff00 2024-05-08T05:50:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.077+0000 7fbeee000700 1 -- 172.21.15.112:0/1929275957 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee8009e20 con 0x7fbef010ff00 2024-05-08T05:50:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.077+0000 7fbef7800700 1 -- 172.21.15.112:0/1929275957 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef019c750 con 0x7fbef010ff00 2024-05-08T05:50:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.077+0000 7fbeee000700 1 -- 172.21.15.112:0/1929275957 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee80059b0 con 0x7fbef010ff00 2024-05-08T05:50:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.077+0000 7fbef7800700 1 -- 172.21.15.112:0/1929275957 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef01a89e0 con 0x7fbef010ff00 2024-05-08T05:50:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.078+0000 7fbef7800700 1 -- 172.21.15.112:0/1929275957 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef0076d90 con 0x7fbef010ff00 2024-05-08T05:50:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.080+0000 7fbeee000700 1 -- 172.21.15.112:0/1929275957 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fbee80050b0 con 0x7fbef010ff00 2024-05-08T05:50:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.082+0000 7fbeee000700 1 --2- 172.21.15.112:0/1929275957 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbed40b2900 0x7fbed40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.082+0000 7fbeee000700 1 -- 172.21.15.112:0/1929275957 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fbee80e1620 con 0x7fbef010ff00 2024-05-08T05:50:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.082+0000 7fbef4e00700 1 --2- 172.21.15.112:0/1929275957 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbed40b2900 0x7fbed40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.082+0000 7fbef4e00700 1 --2- 172.21.15.112:0/1929275957 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbed40b2900 0x7fbed40b4dc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbee0011550 tx=0x7fbee0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.086+0000 7fbeee000700 1 -- 172.21.15.112:0/1929275957 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbee8010070 con 0x7fbef010ff00 2024-05-08T05:50:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.370+0000 7fbef7800700 1 -- 172.21.15.112:0/1929275957 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbef019cff0 con 0x7fbef010ff00 2024-05-08T05:50:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.370+0000 7fbeee000700 1 -- 172.21.15.112:0/1929275957 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fbee80a9740 con 0x7fbef010ff00 2024-05-08T05:50:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.375+0000 7fbef7800700 1 -- 172.21.15.112:0/1929275957 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbed40b2900 msgr2=0x7fbed40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.375+0000 7fbef7800700 1 --2- 172.21.15.112:0/1929275957 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbed40b2900 0x7fbed40b4dc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbee0011550 tx=0x7fbee0009260 comp rx=0 tx=0).stop 2024-05-08T05:50:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.375+0000 7fbef7800700 1 -- 172.21.15.112:0/1929275957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef010ff00 msgr2=0x7fbef019bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.375+0000 7fbef7800700 1 --2- 172.21.15.112:0/1929275957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef010ff00 0x7fbef019bb80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbee8007ee0 tx=0x7fbee800d3b0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.375+0000 7fbef7800700 1 -- 172.21.15.112:0/1929275957 shutdown_connections 2024-05-08T05:50:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.375+0000 7fbef7800700 1 --2- 172.21.15.112:0/1929275957 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbed40b2900 0x7fbed40b4dc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.375+0000 7fbef7800700 1 --2- 172.21.15.112:0/1929275957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef010ff00 0x7fbef019bb80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.375+0000 7fbef7800700 1 --2- 172.21.15.112:0/1929275957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef00756d0 0x7fbef01a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.375+0000 7fbef7800700 1 --2- 172.21.15.112:0/1929275957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbef0075ff0 0x7fbef01a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.375+0000 7fbef7800700 1 -- 172.21.15.112:0/1929275957 >> 172.21.15.112:0/1929275957 conn(0x7fbef007a600 msgr2=0x7fbef0100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.375+0000 7fbef7800700 1 -- 172.21.15.112:0/1929275957 shutdown_connections 2024-05-08T05:50:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:14.375+0000 7fbef7800700 1 -- 172.21.15.112:0/1929275957 wait complete. 2024-05-08T05:50:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 3 2024-05-08T05:50:15.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:15 smithi112 ceph-mon[30003]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:15.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:15 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1929275957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:15.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:15 smithi150 ceph-mon[34315]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:15.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:15 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1929275957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:15.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:15 smithi162 ceph-mon[34800]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:15 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1929275957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:16.315 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 08 05:50:16 smithi112 systemd[1]: Started Ceph mds.j for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:50:17.366 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:17 smithi112 ceph-mon[30003]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:17.366 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:17 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.112:6834/3258450798,v1:172.21.15.112:6835/3258450798] up:boot 2024-05-08T05:50:17.366 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:17 smithi112 ceph-mon[30003]: fsmap 3 up:standby 2024-05-08T05:50:17.366 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:17 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T05:50:17.367 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:17 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:17.367 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:17 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:17.367 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:17 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:17.367 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:17 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:17.367 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:17 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:17.367 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:17 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:17.367 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:17 smithi112 ceph-mon[30003]: Deploying daemon mds.d on smithi112 2024-05-08T05:50:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:17 smithi150 ceph-mon[34315]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:17 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.112:6834/3258450798,v1:172.21.15.112:6835/3258450798] up:boot 2024-05-08T05:50:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:17 smithi150 ceph-mon[34315]: fsmap 3 up:standby 2024-05-08T05:50:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:17 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T05:50:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:17 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:17 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:17.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:17 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:17.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:17 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:17.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:17 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:17.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:17 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:17.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:17 smithi150 ceph-mon[34315]: Deploying daemon mds.d on smithi112 2024-05-08T05:50:17.477 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:50:17.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:17 smithi162 ceph-mon[34800]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:17.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:17 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.112:6834/3258450798,v1:172.21.15.112:6835/3258450798] up:boot 2024-05-08T05:50:17.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:17 smithi162 ceph-mon[34800]: fsmap 3 up:standby 2024-05-08T05:50:17.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:17 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T05:50:17.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:17 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:17.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:17 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:17.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:17 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:17.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:17 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:17.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:17 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:17.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:17 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:17.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:17 smithi162 ceph-mon[34800]: Deploying daemon mds.d on smithi112 2024-05-08T05:50:18.452 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:19.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:19 smithi112 ceph-mon[30003]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:19.316 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 05:50:19 smithi112 systemd[1]: Starting Ceph mds.d for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:50:19.369 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:19 smithi162 ceph-mon[34800]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:19.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:19 smithi150 ceph-mon[34315]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:20.067 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 05:50:20 smithi112 podman[68641]: 2024-05-08 05:50:19.937092086 +0000 UTC m=+0.041080391 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:50:20.067 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 05:50:20 smithi112 podman[68641]: 2024-05-08 05:50:20.045636565 +0000 UTC m=+0.149624914 container create cb81d29a3c70fe71970fa25cfeff0d7060ce0d5e81d73335d687234a5f8d9d99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-08T05:50:20.754 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 05:50:20 smithi112 podman[68641]: 2024-05-08 05:50:20.555593887 +0000 UTC m=+0.659582249 container init cb81d29a3c70fe71970fa25cfeff0d7060ce0d5e81d73335d687234a5f8d9d99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-08T05:50:20.754 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 05:50:20 smithi112 podman[68641]: 2024-05-08 05:50:20.567386589 +0000 UTC m=+0.671374938 container start cb81d29a3c70fe71970fa25cfeff0d7060ce0d5e81d73335d687234a5f8d9d99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:50:20.754 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 05:50:20 smithi112 ceph-mds[68683]: -- 172.21.15.112:0/1950311282 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d96f15fa40 con 0x55d96f132800 2024-05-08T05:50:20.755 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 05:50:20 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-d[68674]: starting mds.d at 2024-05-08T05:50:20.755 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 05:50:20 smithi112 bash[68641]: cb81d29a3c70fe71970fa25cfeff0d7060ce0d5e81d73335d687234a5f8d9d99 2024-05-08T05:50:20.755 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 05:50:20 smithi112 systemd[1]: Started Ceph mds.d for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:50:20.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.926+0000 7f10e9400700 1 -- 172.21.15.112:0/3733845612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10e41056c0 msgr2=0x7f10e4105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:20.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.926+0000 7f10e9400700 1 --2- 172.21.15.112:0/3733845612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10e41056c0 0x7f10e4105b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009990 tx=0x7f10d0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:50:20.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.926+0000 7f10e9400700 1 -- 172.21.15.112:0/3733845612 shutdown_connections 2024-05-08T05:50:20.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.926+0000 7f10e9400700 1 --2- 172.21.15.112:0/3733845612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10e4106080 0x7f10e4112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:20.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.926+0000 7f10e9400700 1 --2- 172.21.15.112:0/3733845612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10e410b6c0 0x7f10e410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:20.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.926+0000 7f10e9400700 1 --2- 172.21.15.112:0/3733845612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10e41056c0 0x7f10e4105b40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:20.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.926+0000 7f10e9400700 1 -- 172.21.15.112:0/3733845612 >> 172.21.15.112:0/3733845612 conn(0x7f10e41011b0 msgr2=0x7f10e41035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:20.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.926+0000 7f10e9400700 1 -- 172.21.15.112:0/3733845612 shutdown_connections 2024-05-08T05:50:20.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.926+0000 7f10e9400700 1 -- 172.21.15.112:0/3733845612 wait complete. 2024-05-08T05:50:20.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.927+0000 7f10e9400700 1 Processor -- start 2024-05-08T05:50:20.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.927+0000 7f10e9400700 1 -- start start 2024-05-08T05:50:20.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e9400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10e41056c0 0x7f10e419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:20.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e9400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10e4106080 0x7f10e419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:20.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e9400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10e410b6c0 0x7f10e41a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:20.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e9400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10e4115900 con 0x7f10e410b6c0 2024-05-08T05:50:20.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e9400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f10e4115780 con 0x7f10e41056c0 2024-05-08T05:50:20.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e9400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f10e4115a80 con 0x7f10e4106080 2024-05-08T05:50:20.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10e41056c0 0x7f10e419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:20.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10e41056c0 0x7f10e419bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:36538/0 (socket says 172.21.15.112:36538) 2024-05-08T05:50:20.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e3200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10e410b6c0 0x7f10e41a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:20.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e2800700 1 -- 172.21.15.112:0/794810499 learned_addr learned my addr 172.21.15.112:0/794810499 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:50:20.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e1e00700 1 --2- 172.21.15.112:0/794810499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10e4106080 0x7f10e419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:20.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e2800700 1 -- 172.21.15.112:0/794810499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10e4106080 msgr2=0x7f10e419c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:20.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e2800700 1 --2- 172.21.15.112:0/794810499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10e4106080 0x7f10e419c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:20.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e2800700 1 -- 172.21.15.112:0/794810499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10e410b6c0 msgr2=0x7f10e41a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:20.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e2800700 1 --2- 172.21.15.112:0/794810499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10e410b6c0 0x7f10e41a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:20.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.928+0000 7f10e2800700 1 -- 172.21.15.112:0/794810499 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e41a0d60 con 0x7f10e41056c0 2024-05-08T05:50:20.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.929+0000 7f10e2800700 1 --2- 172.21.15.112:0/794810499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10e41056c0 0x7f10e419bd50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f10d400e9a0 tx=0x7f10d400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:20.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.929+0000 7f10db400700 1 -- 172.21.15.112:0/794810499 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d400ca10 con 0x7f10e41056c0 2024-05-08T05:50:20.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.929+0000 7f10e9400700 1 -- 172.21.15.112:0/794810499 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e41a1050 con 0x7f10e41056c0 2024-05-08T05:50:20.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.929+0000 7f10db400700 1 -- 172.21.15.112:0/794810499 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d400cbb0 con 0x7f10e41056c0 2024-05-08T05:50:20.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.929+0000 7f10db400700 1 -- 172.21.15.112:0/794810499 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4018680 con 0x7f10e41056c0 2024-05-08T05:50:20.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.929+0000 7f10e9400700 1 -- 172.21.15.112:0/794810499 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e41a88d0 con 0x7f10e41056c0 2024-05-08T05:50:20.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.930+0000 7f10e9400700 1 -- 172.21.15.112:0/794810499 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e406a190 con 0x7f10e41056c0 2024-05-08T05:50:20.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.933+0000 7f10db400700 1 -- 172.21.15.112:0/794810499 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f10d4010430 con 0x7f10e41056c0 2024-05-08T05:50:20.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.934+0000 7f10db400700 1 --2- 172.21.15.112:0/794810499 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f10c80b2880 0x7f10c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:20.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.935+0000 7f10db400700 1 -- 172.21.15.112:0/794810499 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f10d40e2280 con 0x7f10e41056c0 2024-05-08T05:50:20.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.935+0000 7f10e1e00700 1 --2- 172.21.15.112:0/794810499 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f10c80b2880 0x7f10c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:20.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.935+0000 7f10e1e00700 1 --2- 172.21.15.112:0/794810499 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f10c80b2880 0x7f10c80b4d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f10d00098e0 tx=0x7f10d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:20.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:20.936+0000 7f10db400700 1 -- 172.21.15.112:0/794810499 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10d40aa3a0 con 0x7f10e41056c0 2024-05-08T05:50:21.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:21.221+0000 7f10e9400700 1 -- 172.21.15.112:0/794810499 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10e402e510 con 0x7f10e41056c0 2024-05-08T05:50:21.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:21.222+0000 7f10db400700 1 -- 172.21.15.112:0/794810499 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3877 (secure 0 0 0) 0x7f10d40a9ca0 con 0x7f10e41056c0 2024-05-08T05:50:21.223 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:50:21.223 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-08T05:50:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:21.225+0000 7f10e9400700 1 -- 172.21.15.112:0/794810499 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f10c80b2880 msgr2=0x7f10c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:21.225+0000 7f10e9400700 1 --2- 172.21.15.112:0/794810499 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f10c80b2880 0x7f10c80b4d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f10d00098e0 tx=0x7f10d0005960 comp rx=0 tx=0).stop 2024-05-08T05:50:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:21.225+0000 7f10e9400700 1 -- 172.21.15.112:0/794810499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10e41056c0 msgr2=0x7f10e419bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:21.225+0000 7f10e9400700 1 --2- 172.21.15.112:0/794810499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10e41056c0 0x7f10e419bd50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f10d400e9a0 tx=0x7f10d400ecb0 comp rx=0 tx=0).stop 2024-05-08T05:50:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:21.225+0000 7f10e9400700 1 -- 172.21.15.112:0/794810499 shutdown_connections 2024-05-08T05:50:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:21.225+0000 7f10e9400700 1 --2- 172.21.15.112:0/794810499 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f10c80b2880 0x7f10c80b4d40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:21.225+0000 7f10e9400700 1 --2- 172.21.15.112:0/794810499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10e410b6c0 0x7f10e41a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:21.225+0000 7f10e9400700 1 --2- 172.21.15.112:0/794810499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10e41056c0 0x7f10e419bd50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:21.225+0000 7f10e9400700 1 --2- 172.21.15.112:0/794810499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10e4106080 0x7f10e419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:21.225+0000 7f10e9400700 1 -- 172.21.15.112:0/794810499 >> 172.21.15.112:0/794810499 conn(0x7f10e41011b0 msgr2=0x7f10e4101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:21.226+0000 7f10e9400700 1 -- 172.21.15.112:0/794810499 shutdown_connections 2024-05-08T05:50:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:21.226+0000 7f10e9400700 1 -- 172.21.15.112:0/794810499 wait complete. 2024-05-08T05:50:21.228 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 5 2024-05-08T05:50:21.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:21 smithi150 ceph-mon[34315]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:21.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:21 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:21.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:21 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:21.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:21 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:21.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:21 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:21.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:21 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:21.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:21 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:21.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:21 smithi112 ceph-mon[30003]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:21.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:21 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:21.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:21 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:21.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:21 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:21.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:21 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:21.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:21 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:21.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:21 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:21.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:21 smithi162 ceph-mon[34800]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:21.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:21 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:21.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:21 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:21.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:21 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:21.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:21 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:21.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:21 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:21.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:21 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:22.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:22 smithi112 ceph-mon[30003]: Deploying daemon mds.e on smithi150 2024-05-08T05:50:22.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:22 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.112:6836/2849670124,v1:172.21.15.112:6837/2849670124] up:boot 2024-05-08T05:50:22.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:22 smithi112 ceph-mon[30003]: fsmap 4 up:standby 2024-05-08T05:50:22.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:22 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T05:50:22.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:22 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/794810499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:22.408 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:22 smithi150 ceph-mon[34315]: Deploying daemon mds.e on smithi150 2024-05-08T05:50:22.408 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:22 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.112:6836/2849670124,v1:172.21.15.112:6837/2849670124] up:boot 2024-05-08T05:50:22.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:22 smithi150 ceph-mon[34315]: fsmap 4 up:standby 2024-05-08T05:50:22.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:22 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T05:50:22.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:22 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/794810499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:22.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:22 smithi162 ceph-mon[34800]: Deploying daemon mds.e on smithi150 2024-05-08T05:50:22.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:22 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.112:6836/2849670124,v1:172.21.15.112:6837/2849670124] up:boot 2024-05-08T05:50:22.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:22 smithi162 ceph-mon[34800]: fsmap 4 up:standby 2024-05-08T05:50:22.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:22 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T05:50:22.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:22 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/794810499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:23.081 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 05:50:22 smithi150 systemd[1]: Starting Ceph mds.e for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:50:23.363 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:23 smithi150 ceph-mon[34315]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:23.364 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 05:50:23 smithi150 podman[63527]: 2024-05-08 05:50:22.984140718 +0000 UTC m=+0.018391414 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:50:23.364 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 05:50:23 smithi150 podman[63527]: 2024-05-08 05:50:23.086478401 +0000 UTC m=+0.120729110 container create 92a9e02d224d2bdf587e48bd5ecd3b19384c26ae0506e4e5b38aac74ba199bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-e, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:50:23.364 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 05:50:23 smithi150 podman[63527]: 2024-05-08 05:50:23.364297858 +0000 UTC m=+0.398548525 container init 92a9e02d224d2bdf587e48bd5ecd3b19384c26ae0506e4e5b38aac74ba199bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-e, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-08T05:50:23.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:23 smithi112 ceph-mon[30003]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:23.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:23 smithi162 ceph-mon[34800]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:23.683 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 05:50:23 smithi150 podman[63527]: 2024-05-08 05:50:23.36700212 +0000 UTC m=+0.401252794 container start 92a9e02d224d2bdf587e48bd5ecd3b19384c26ae0506e4e5b38aac74ba199bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:50:23.683 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 05:50:23 smithi150 ceph-mds[63546]: -- 172.21.15.150:0/2568342893 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5565ae0dfa40 con 0x5565ae0b2c00 2024-05-08T05:50:23.683 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 05:50:23 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-e[63542]: starting mds.e at 2024-05-08T05:50:23.683 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 05:50:23 smithi150 bash[63527]: 92a9e02d224d2bdf587e48bd5ecd3b19384c26ae0506e4e5b38aac74ba199bb8 2024-05-08T05:50:23.683 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 05:50:23 smithi150 systemd[1]: Started Ceph mds.e for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:50:23.980 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:50:24.442 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:24.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:24 smithi112 ceph-mon[30003]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:24.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:24.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:24.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:24.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:24.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:24.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:24.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:24 smithi112 ceph-mon[30003]: Deploying daemon mds.f on smithi162 2024-05-08T05:50:24.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:24 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.150:6834/1945132252,v1:172.21.15.150:6835/1945132252] up:boot 2024-05-08T05:50:24.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:24 smithi112 ceph-mon[30003]: fsmap 5 up:standby 2024-05-08T05:50:24.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:50:24.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:24 smithi162 ceph-mon[34800]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:24.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:24.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:24.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:24.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:24.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:24.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:24.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:24 smithi162 ceph-mon[34800]: Deploying daemon mds.f on smithi162 2024-05-08T05:50:24.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:24 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.150:6834/1945132252,v1:172.21.15.150:6835/1945132252] up:boot 2024-05-08T05:50:24.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:24 smithi162 ceph-mon[34800]: fsmap 5 up:standby 2024-05-08T05:50:24.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:50:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:24 smithi150 ceph-mon[34315]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:24 smithi150 ceph-mon[34315]: Deploying daemon mds.f on smithi162 2024-05-08T05:50:24.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:24 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.150:6834/1945132252,v1:172.21.15.150:6835/1945132252] up:boot 2024-05-08T05:50:24.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:24 smithi150 ceph-mon[34315]: fsmap 5 up:standby 2024-05-08T05:50:24.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:50:25.991 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 05:50:25 smithi162 systemd[1]: Starting Ceph mds.f for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:50:26.254 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 05:50:25 smithi162 podman[64946]: 2024-05-08 05:50:25.89445723 +0000 UTC m=+0.017819026 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:50:26.254 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 05:50:25 smithi162 podman[64946]: 2024-05-08 05:50:25.993359249 +0000 UTC m=+0.116721078 container create f14a5f6fd940d1762bb23647832997d389bc12de26df02a96918d1c5a847da24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-08T05:50:26.254 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 05:50:26 smithi162 podman[64946]: 2024-05-08 05:50:26.195816346 +0000 UTC m=+0.319178129 container init f14a5f6fd940d1762bb23647832997d389bc12de26df02a96918d1c5a847da24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-f, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507) 2024-05-08T05:50:26.254 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 05:50:26 smithi162 podman[64946]: 2024-05-08 05:50:26.198377449 +0000 UTC m=+0.321739239 container start f14a5f6fd940d1762bb23647832997d389bc12de26df02a96918d1c5a847da24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-f, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:50:26.254 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 05:50:26 smithi162 ceph-mds[64965]: -- 172.21.15.162:0/921123136 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c5c269a40 con 0x563c5c23c800 2024-05-08T05:50:26.254 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 05:50:26 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-f[64961]: starting mds.f at 2024-05-08T05:50:26.254 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 05:50:26 smithi162 bash[64946]: f14a5f6fd940d1762bb23647832997d389bc12de26df02a96918d1c5a847da24 2024-05-08T05:50:26.523 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 05:50:26 smithi162 systemd[1]: Started Ceph mds.f for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:50:26.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:26 smithi112 ceph-mon[30003]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:26.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:26.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:26.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:26.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:26.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:26.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:26.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.590+0000 7fa9a5e00700 1 -- 172.21.15.112:0/998874261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a00756d0 msgr2=0x7fa9a0075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:26.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.590+0000 7fa9a5e00700 1 --2- 172.21.15.112:0/998874261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a00756d0 0x7fa9a0075ab0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa990009990 tx=0x7fa990009ca0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.590+0000 7fa9a5e00700 1 -- 172.21.15.112:0/998874261 shutdown_connections 2024-05-08T05:50:26.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.590+0000 7fa9a5e00700 1 --2- 172.21.15.112:0/998874261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9a0114670 0x7fa9a0116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.590+0000 7fa9a5e00700 1 --2- 172.21.15.112:0/998874261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a00756d0 0x7fa9a0075ab0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.590+0000 7fa9a5e00700 1 --2- 172.21.15.112:0/998874261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9a0075ff0 0x7fa9a0114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.590+0000 7fa9a5e00700 1 -- 172.21.15.112:0/998874261 >> 172.21.15.112:0/998874261 conn(0x7fa9a007a600 msgr2=0x7fa9a007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:26.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.590+0000 7fa9a5e00700 1 -- 172.21.15.112:0/998874261 shutdown_connections 2024-05-08T05:50:26.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.590+0000 7fa9a5e00700 1 -- 172.21.15.112:0/998874261 wait complete. 2024-05-08T05:50:26.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.592+0000 7fa9a5e00700 1 Processor -- start 2024-05-08T05:50:26.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.592+0000 7fa9a5e00700 1 -- start start 2024-05-08T05:50:26.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa9a5e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9a00756d0 0x7fa9a01a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:26.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa9a5e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9a0075ff0 0x7fa9a01a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:26.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa9a5e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a0114670 0x7fa9a01a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:26.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa9a5e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0119ca0 con 0x7fa9a0114670 2024-05-08T05:50:26.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa9a5e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0119b20 con 0x7fa9a00756d0 2024-05-08T05:50:26.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa9a5e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0119e20 con 0x7fa9a0075ff0 2024-05-08T05:50:26.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa99e000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9a0075ff0 0x7fa9a01a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:26.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa99e000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9a0075ff0 0x7fa9a01a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50070/0 (socket says 172.21.15.112:50070) 2024-05-08T05:50:26.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa99ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9a00756d0 0x7fa9a01a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:26.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa99e000700 1 -- 172.21.15.112:0/2800361775 learned_addr learned my addr 172.21.15.112:0/2800361775 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:50:26.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa99f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a0114670 0x7fa9a01a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:26.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa99e000700 1 -- 172.21.15.112:0/2800361775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9a00756d0 msgr2=0x7fa9a01a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:26.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa99e000700 1 --2- 172.21.15.112:0/2800361775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9a00756d0 0x7fa9a01a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa99e000700 1 -- 172.21.15.112:0/2800361775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a0114670 msgr2=0x7fa9a01a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:26.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa99e000700 1 --2- 172.21.15.112:0/2800361775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a0114670 0x7fa9a01a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa99e000700 1 -- 172.21.15.112:0/2800361775 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a01a5240 con 0x7fa9a0075ff0 2024-05-08T05:50:26.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.593+0000 7fa99ea00700 1 --2- 172.21.15.112:0/2800361775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9a00756d0 0x7fa9a01a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:26.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.594+0000 7fa99e000700 1 --2- 172.21.15.112:0/2800361775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9a0075ff0 0x7fa9a01a0770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00eb30 tx=0x7fa98c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.594+0000 7fa9a4a00700 1 -- 172.21.15.112:0/2800361775 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c00cbd0 con 0x7fa9a0075ff0 2024-05-08T05:50:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.594+0000 7fa9a5e00700 1 -- 172.21.15.112:0/2800361775 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a01a5530 con 0x7fa9a0075ff0 2024-05-08T05:50:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.594+0000 7fa9a4a00700 1 -- 172.21.15.112:0/2800361775 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa98c0044d0 con 0x7fa9a0075ff0 2024-05-08T05:50:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.594+0000 7fa9a4a00700 1 -- 172.21.15.112:0/2800361775 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c003e30 con 0x7fa9a0075ff0 2024-05-08T05:50:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.594+0000 7fa9a5e00700 1 -- 172.21.15.112:0/2800361775 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a01acdb0 con 0x7fa9a0075ff0 2024-05-08T05:50:26.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.595+0000 7fa9a5e00700 1 -- 172.21.15.112:0/2800361775 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a006a190 con 0x7fa9a0075ff0 2024-05-08T05:50:26.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.599+0000 7fa9a4a00700 1 -- 172.21.15.112:0/2800361775 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fa98c004000 con 0x7fa9a0075ff0 2024-05-08T05:50:26.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.601+0000 7fa9a4a00700 1 --2- 172.21.15.112:0/2800361775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa9840b2900 0x7fa9840b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:26.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.601+0000 7fa9a4a00700 1 -- 172.21.15.112:0/2800361775 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fa98c0e21e0 con 0x7fa9a0075ff0 2024-05-08T05:50:26.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.601+0000 7fa99ea00700 1 --2- 172.21.15.112:0/2800361775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa9840b2900 0x7fa9840b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:26.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.602+0000 7fa99ea00700 1 --2- 172.21.15.112:0/2800361775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa9840b2900 0x7fa9840b4dc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa990006010 tx=0x7fa99000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:26.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.606+0000 7fa9a4a00700 1 -- 172.21.15.112:0/2800361775 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa98c0042f0 con 0x7fa9a0075ff0 2024-05-08T05:50:26.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:26 smithi162 ceph-mon[34800]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:26.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:26.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:26.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:26.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:26.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:26.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:26.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.888+0000 7fa9a5e00700 1 -- 172.21.15.112:0/2800361775 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9a01a1590 con 0x7fa9a0075ff0 2024-05-08T05:50:26.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.889+0000 7fa9a4a00700 1 -- 172.21.15.112:0/2800361775 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5533 (secure 0 0 0) 0x7fa9a01a1590 con 0x7fa9a0075ff0 2024-05-08T05:50:26.890 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:50:26.890 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-05-08T05:50:26.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.892+0000 7fa9a5e00700 1 -- 172.21.15.112:0/2800361775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa9840b2900 msgr2=0x7fa9840b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:26.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.892+0000 7fa9a5e00700 1 --2- 172.21.15.112:0/2800361775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa9840b2900 0x7fa9840b4dc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa990006010 tx=0x7fa99000b260 comp rx=0 tx=0).stop 2024-05-08T05:50:26.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.892+0000 7fa9a5e00700 1 -- 172.21.15.112:0/2800361775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9a0075ff0 msgr2=0x7fa9a01a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:26.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.892+0000 7fa9a5e00700 1 --2- 172.21.15.112:0/2800361775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9a0075ff0 0x7fa9a01a0770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00eb30 tx=0x7fa98c00ee40 comp rx=0 tx=0).stop 2024-05-08T05:50:26.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.892+0000 7fa9a5e00700 1 -- 172.21.15.112:0/2800361775 shutdown_connections 2024-05-08T05:50:26.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.892+0000 7fa9a5e00700 1 --2- 172.21.15.112:0/2800361775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa9840b2900 0x7fa9840b4dc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.892+0000 7fa9a5e00700 1 --2- 172.21.15.112:0/2800361775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a0114670 0x7fa9a01a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.892+0000 7fa9a5e00700 1 --2- 172.21.15.112:0/2800361775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9a00756d0 0x7fa9a01a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.892+0000 7fa9a5e00700 1 --2- 172.21.15.112:0/2800361775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9a0075ff0 0x7fa9a01a0770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.892+0000 7fa9a5e00700 1 -- 172.21.15.112:0/2800361775 >> 172.21.15.112:0/2800361775 conn(0x7fa9a007a600 msgr2=0x7fa9a01134b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:26.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.893+0000 7fa9a5e00700 1 -- 172.21.15.112:0/2800361775 shutdown_connections 2024-05-08T05:50:26.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:26.893+0000 7fa9a5e00700 1 -- 172.21.15.112:0/2800361775 wait complete. 2024-05-08T05:50:26.895 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2024-05-08T05:50:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:26 smithi150 ceph-mon[34315]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:27.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:27 smithi162 ceph-mon[34800]: Deploying daemon mds.c on smithi162 2024-05-08T05:50:27.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:27 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.162:6834/3060051414,v1:172.21.15.162:6835/3060051414] up:boot 2024-05-08T05:50:27.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:27 smithi162 ceph-mon[34800]: fsmap 6 up:standby 2024-05-08T05:50:27.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:27 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:50:27.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:27 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/2800361775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:27.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:27 smithi112 ceph-mon[30003]: Deploying daemon mds.c on smithi162 2024-05-08T05:50:27.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:27 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.162:6834/3060051414,v1:172.21.15.162:6835/3060051414] up:boot 2024-05-08T05:50:27.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:27 smithi112 ceph-mon[30003]: fsmap 6 up:standby 2024-05-08T05:50:27.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:27 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:50:27.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:27 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2800361775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:27.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:27 smithi150 ceph-mon[34315]: Deploying daemon mds.c on smithi162 2024-05-08T05:50:27.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:27 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.162:6834/3060051414,v1:172.21.15.162:6835/3060051414] up:boot 2024-05-08T05:50:27.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:27 smithi150 ceph-mon[34315]: fsmap 6 up:standby 2024-05-08T05:50:27.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:27 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:50:27.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:27 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/2800361775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:28.521 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 05:50:28 smithi162 systemd[1]: Starting Ceph mds.c for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:50:28.791 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:28 smithi162 ceph-mon[34800]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:28.791 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 05:50:28 smithi162 podman[65257]: 2024-05-08 05:50:28.521308704 +0000 UTC m=+0.106203788 container create e053083a8d5fc3a8259501c3a1da76e5543387563693f165f48565de0170e051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-08T05:50:28.791 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 05:50:28 smithi162 podman[65257]: 2024-05-08 05:50:28.433270545 +0000 UTC m=+0.018165628 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:50:28.791 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 05:50:28 smithi162 podman[65257]: 2024-05-08 05:50:28.76576047 +0000 UTC m=+0.350655554 container init e053083a8d5fc3a8259501c3a1da76e5543387563693f165f48565de0170e051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True) 2024-05-08T05:50:28.791 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 05:50:28 smithi162 podman[65257]: 2024-05-08 05:50:28.768335649 +0000 UTC m=+0.353230752 container start e053083a8d5fc3a8259501c3a1da76e5543387563693f165f48565de0170e051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-08T05:50:28.791 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 05:50:28 smithi162 ceph-mds[65276]: -- 172.21.15.162:0/4242448763 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563418cefa40 con 0x563418cc2c00 2024-05-08T05:50:28.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:28 smithi112 ceph-mon[30003]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:28.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:28 smithi150 ceph-mon[34315]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:29.066 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 05:50:28 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-c[65272]: starting mds.c at 2024-05-08T05:50:29.067 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 05:50:28 smithi162 bash[65257]: e053083a8d5fc3a8259501c3a1da76e5543387563693f165f48565de0170e051 2024-05-08T05:50:29.067 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 05:50:28 smithi162 systemd[1]: Started Ceph mds.c for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:50:29.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:50:30.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:29 smithi112 ceph-mon[30003]: Deploying daemon mds.h on smithi150 2024-05-08T05:50:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:29 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.162:6836/3092620994,v1:172.21.15.162:6837/3092620994] up:boot 2024-05-08T05:50:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:29 smithi112 ceph-mon[30003]: fsmap 7 up:standby 2024-05-08T05:50:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:50:30.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:50:30.137 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:29 smithi150 ceph-mon[34315]: Deploying daemon mds.h on smithi150 2024-05-08T05:50:30.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:29 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.162:6836/3092620994,v1:172.21.15.162:6837/3092620994] up:boot 2024-05-08T05:50:30.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:29 smithi150 ceph-mon[34315]: fsmap 7 up:standby 2024-05-08T05:50:30.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:50:30.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:50:30.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:30.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:30.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:30.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:30.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:30.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:30.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:29 smithi162 ceph-mon[34800]: Deploying daemon mds.h on smithi150 2024-05-08T05:50:30.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:29 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.162:6836/3092620994,v1:172.21.15.162:6837/3092620994] up:boot 2024-05-08T05:50:30.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:29 smithi162 ceph-mon[34800]: fsmap 7 up:standby 2024-05-08T05:50:30.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:50:30.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:50:31.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:31 smithi150 ceph-mon[34315]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:31.183 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 05:50:31 smithi150 systemd[1]: Starting Ceph mds.h for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:50:31.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:31 smithi112 ceph-mon[30003]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:31.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:31 smithi162 ceph-mon[34800]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:31.542 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 05:50:31 smithi150 podman[63842]: 2024-05-08 05:50:31.189791232 +0000 UTC m=+0.018377658 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:50:31.542 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 05:50:31 smithi150 podman[63842]: 2024-05-08 05:50:31.290021833 +0000 UTC m=+0.118608243 container create cb34aefd585ee76e60737edd0acaa21f8ade4435982703008f9b2e2292c349c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-08T05:50:31.542 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 05:50:31 smithi150 podman[63842]: 2024-05-08 05:50:31.542346403 +0000 UTC m=+0.370932812 container init cb34aefd585ee76e60737edd0acaa21f8ade4435982703008f9b2e2292c349c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-08T05:50:31.933 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 05:50:31 smithi150 podman[63842]: 2024-05-08 05:50:31.544890154 +0000 UTC m=+0.373476564 container start cb34aefd585ee76e60737edd0acaa21f8ade4435982703008f9b2e2292c349c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-h, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True) 2024-05-08T05:50:31.933 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 05:50:31 smithi150 ceph-mds[63861]: -- 172.21.15.150:0/3212638952 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d0245cda40 con 0x55d0245a1000 2024-05-08T05:50:31.933 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 05:50:31 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-h[63857]: starting mds.h at 2024-05-08T05:50:31.933 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 05:50:31 smithi150 bash[63842]: cb34aefd585ee76e60737edd0acaa21f8ade4435982703008f9b2e2292c349c7 2024-05-08T05:50:31.933 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 05:50:31 smithi150 systemd[1]: Started Ceph mds.h for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:50:32.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.392+0000 7f9ff5c00700 1 -- 172.21.15.112:0/1035150529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff006c630 msgr2=0x7f9ff0110360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:32.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.392+0000 7f9ff5c00700 1 --2- 172.21.15.112:0/1035150529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff006c630 0x7f9ff0110360 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800b0d0 tx=0x7f9fe800b3e0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.392+0000 7f9ff5c00700 1 -- 172.21.15.112:0/1035150529 shutdown_connections 2024-05-08T05:50:32.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.392+0000 7f9ff5c00700 1 --2- 172.21.15.112:0/1035150529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff006c630 0x7f9ff0110360 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.392+0000 7f9ff5c00700 1 --2- 172.21.15.112:0/1035150529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ff0109590 0x7f9ff0109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.392+0000 7f9ff5c00700 1 --2- 172.21.15.112:0/1035150529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff006bc70 0x7f9ff006c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.392+0000 7f9ff5c00700 1 -- 172.21.15.112:0/1035150529 >> 172.21.15.112:0/1035150529 conn(0x7f9ff0079440 msgr2=0x7f9ff0079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:32.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.392+0000 7f9ff5c00700 1 -- 172.21.15.112:0/1035150529 shutdown_connections 2024-05-08T05:50:32.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.392+0000 7f9ff5c00700 1 -- 172.21.15.112:0/1035150529 wait complete. 2024-05-08T05:50:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.394+0000 7f9ff5c00700 1 Processor -- start 2024-05-08T05:50:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.394+0000 7f9ff5c00700 1 -- start start 2024-05-08T05:50:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.394+0000 7f9ff5c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff006bc70 0x7f9ff01a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.394+0000 7f9ff5c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ff006c630 0x7f9ff01a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.394+0000 7f9ff5c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff0109590 0x7f9ff019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.394+0000 7f9ff5c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff01136a0 con 0x7f9ff006bc70 2024-05-08T05:50:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.394+0000 7f9ff5c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0113520 con 0x7f9ff0109590 2024-05-08T05:50:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.394+0000 7f9ff5c00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0113820 con 0x7f9ff006c630 2024-05-08T05:50:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.395+0000 7f9fee000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ff006c630 0x7f9ff01a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.395+0000 7f9fee000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ff006c630 0x7f9ff01a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50092/0 (socket says 172.21.15.112:50092) 2024-05-08T05:50:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.395+0000 7f9fee000700 1 -- 172.21.15.112:0/397190399 learned_addr learned my addr 172.21.15.112:0/397190399 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:50:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.395+0000 7f9feea00700 1 --2- 172.21.15.112:0/397190399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff006bc70 0x7f9ff01a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:32.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.395+0000 7f9fef400700 1 --2- 172.21.15.112:0/397190399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff0109590 0x7f9ff019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:32.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.395+0000 7f9fee000700 1 -- 172.21.15.112:0/397190399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff0109590 msgr2=0x7f9ff019bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:32.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.395+0000 7f9fee000700 1 --2- 172.21.15.112:0/397190399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff0109590 0x7f9ff019bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.395+0000 7f9fee000700 1 -- 172.21.15.112:0/397190399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff006bc70 msgr2=0x7f9ff01a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:32.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.395+0000 7f9fee000700 1 --2- 172.21.15.112:0/397190399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff006bc70 0x7f9ff01a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.395+0000 7f9fee000700 1 -- 172.21.15.112:0/397190399 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff019c260 con 0x7f9ff006c630 2024-05-08T05:50:32.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.396+0000 7f9fee000700 1 --2- 172.21.15.112:0/397190399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ff006c630 0x7f9ff01a1e80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00e9a0 tx=0x7f9fdc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:32.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.396+0000 7f9fe7400700 1 -- 172.21.15.112:0/397190399 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc00ca10 con 0x7f9ff006c630 2024-05-08T05:50:32.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.396+0000 7f9fe7400700 1 -- 172.21.15.112:0/397190399 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fdc00cbb0 con 0x7f9ff006c630 2024-05-08T05:50:32.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.396+0000 7f9ff5c00700 1 -- 172.21.15.112:0/397190399 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff019c550 con 0x7f9ff006c630 2024-05-08T05:50:32.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.396+0000 7f9ff5c00700 1 -- 172.21.15.112:0/397190399 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff01a8810 con 0x7f9ff006c630 2024-05-08T05:50:32.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.396+0000 7f9fe7400700 1 -- 172.21.15.112:0/397190399 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc018680 con 0x7f9ff006c630 2024-05-08T05:50:32.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.397+0000 7f9ff5c00700 1 -- 172.21.15.112:0/397190399 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd0005330 con 0x7f9ff006c630 2024-05-08T05:50:32.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.400+0000 7f9fe7400700 1 -- 172.21.15.112:0/397190399 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f9fdc018820 con 0x7f9ff006c630 2024-05-08T05:50:32.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.401+0000 7f9fe7400700 1 --2- 172.21.15.112:0/397190399 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9fd40b2900 0x7f9fd40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:32.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.401+0000 7f9fe7400700 1 -- 172.21.15.112:0/397190399 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f9fdc0e1de0 con 0x7f9ff006c630 2024-05-08T05:50:32.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.402+0000 7f9feea00700 1 --2- 172.21.15.112:0/397190399 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9fd40b2900 0x7f9fd40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:32.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.402+0000 7f9feea00700 1 --2- 172.21.15.112:0/397190399 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9fd40b2900 0x7f9fd40b4dc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009c60 tx=0x7f9fe0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.406+0000 7f9fe7400700 1 -- 172.21.15.112:0/397190399 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9fdc0a9f00 con 0x7f9ff006c630 2024-05-08T05:50:32.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.686+0000 7f9ff5c00700 1 -- 172.21.15.112:0/397190399 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fd0005620 con 0x7f9ff006c630 2024-05-08T05:50:32.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.689+0000 7f9fe7400700 1 -- 172.21.15.112:0/397190399 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7189 (secure 0 0 0) 0x7f9fdc0a9800 con 0x7f9ff006c630 2024-05-08T05:50:32.690 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:50:32.690 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-08T05:50:32.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.692+0000 7f9ff5c00700 1 -- 172.21.15.112:0/397190399 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9fd40b2900 msgr2=0x7f9fd40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:32.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.692+0000 7f9ff5c00700 1 --2- 172.21.15.112:0/397190399 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9fd40b2900 0x7f9fd40b4dc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009c60 tx=0x7f9fe0009260 comp rx=0 tx=0).stop 2024-05-08T05:50:32.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.692+0000 7f9ff5c00700 1 -- 172.21.15.112:0/397190399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ff006c630 msgr2=0x7f9ff01a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:32.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.692+0000 7f9ff5c00700 1 --2- 172.21.15.112:0/397190399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ff006c630 0x7f9ff01a1e80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00e9a0 tx=0x7f9fdc00ecb0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.693+0000 7f9ff5c00700 1 -- 172.21.15.112:0/397190399 shutdown_connections 2024-05-08T05:50:32.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.693+0000 7f9ff5c00700 1 --2- 172.21.15.112:0/397190399 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9fd40b2900 0x7f9fd40b4dc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.693+0000 7f9ff5c00700 1 --2- 172.21.15.112:0/397190399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff0109590 0x7f9ff019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.693+0000 7f9ff5c00700 1 --2- 172.21.15.112:0/397190399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff006bc70 0x7f9ff01a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.693+0000 7f9ff5c00700 1 --2- 172.21.15.112:0/397190399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ff006c630 0x7f9ff01a1e80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.693+0000 7f9ff5c00700 1 -- 172.21.15.112:0/397190399 >> 172.21.15.112:0/397190399 conn(0x7f9ff0079440 msgr2=0x7f9ff010e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:32.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.693+0000 7f9ff5c00700 1 -- 172.21.15.112:0/397190399 shutdown_connections 2024-05-08T05:50:32.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:32.693+0000 7f9ff5c00700 1 -- 172.21.15.112:0/397190399 wait complete. 2024-05-08T05:50:32.695 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 9 2024-05-08T05:50:32.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:32 smithi150 ceph-mon[34315]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:32.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:32 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:32.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:32 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:32.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:32 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:32.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:32 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:32.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:32 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:32.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:32 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:32.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:32 smithi150 ceph-mon[34315]: Deploying daemon mds.i on smithi162 2024-05-08T05:50:32.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:32 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.150:6836/4103977076,v1:172.21.15.150:6837/4103977076] up:boot 2024-05-08T05:50:32.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:32 smithi150 ceph-mon[34315]: fsmap 8 up:standby 2024-05-08T05:50:32.935 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:32 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:50:33.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:32 smithi162 ceph-mon[34800]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:32 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:32 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:32 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:32 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:32 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:32 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:32 smithi162 ceph-mon[34800]: Deploying daemon mds.i on smithi162 2024-05-08T05:50:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:32 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.150:6836/4103977076,v1:172.21.15.150:6837/4103977076] up:boot 2024-05-08T05:50:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:32 smithi162 ceph-mon[34800]: fsmap 8 up:standby 2024-05-08T05:50:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:32 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:50:33.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:32 smithi112 ceph-mon[30003]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:33.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:33.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:33.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:33.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:33.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:33.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:33.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:32 smithi112 ceph-mon[30003]: Deploying daemon mds.i on smithi162 2024-05-08T05:50:33.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:32 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.150:6836/4103977076,v1:172.21.15.150:6837/4103977076] up:boot 2024-05-08T05:50:33.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:32 smithi112 ceph-mon[30003]: fsmap 8 up:standby 2024-05-08T05:50:33.068 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:50:33.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:33 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/397190399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:34.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:33 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/397190399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:34.067 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 05:50:33 smithi162 systemd[1]: Starting Ceph mds.i for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:50:34.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:33 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/397190399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:34.422 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 05:50:34 smithi162 podman[65564]: 2024-05-08 05:50:34.053804287 +0000 UTC m=+0.018032875 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:50:34.422 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 05:50:34 smithi162 podman[65564]: 2024-05-08 05:50:34.169963465 +0000 UTC m=+0.134192054 container create 2f5505b532fa10eabf37d1d4bdf6f4276c3bf3a3432e557a34fbd27a998e458a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-08T05:50:34.422 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 05:50:34 smithi162 podman[65564]: 2024-05-08 05:50:34.364179189 +0000 UTC m=+0.328407780 container init 2f5505b532fa10eabf37d1d4bdf6f4276c3bf3a3432e557a34fbd27a998e458a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-i, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-08T05:50:34.422 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 05:50:34 smithi162 podman[65564]: 2024-05-08 05:50:34.366708877 +0000 UTC m=+0.330937466 container start 2f5505b532fa10eabf37d1d4bdf6f4276c3bf3a3432e557a34fbd27a998e458a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:50:34.422 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 05:50:34 smithi162 ceph-mds[65583]: -- 172.21.15.162:0/3366100223 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561755bb3a40 con 0x561755b86c00 2024-05-08T05:50:34.423 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 05:50:34 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-i[65579]: starting mds.i at 2024-05-08T05:50:34.423 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 05:50:34 smithi162 bash[65564]: 2f5505b532fa10eabf37d1d4bdf6f4276c3bf3a3432e557a34fbd27a998e458a 2024-05-08T05:50:34.707 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 05:50:34 smithi162 systemd[1]: Started Ceph mds.i for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:50:34.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:34 smithi150 ceph-mon[34315]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:34.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:34.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:34.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:34.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:34.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:34.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:35.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:34 smithi112 ceph-mon[30003]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:35.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:35.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:35.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:35.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:35.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:35.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:35.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:34 smithi162 ceph-mon[34800]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:35.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:35.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:35.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:35.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:35.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:35.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:35.514 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:50:35.751 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:35 smithi162 ceph-mon[34800]: Deploying daemon mds.g on smithi112 2024-05-08T05:50:35.752 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:35 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] up:boot 2024-05-08T05:50:35.752 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:35 smithi162 ceph-mon[34800]: fsmap 9 up:standby 2024-05-08T05:50:35.752 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:35 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:50:35.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:35 smithi112 ceph-mon[30003]: Deploying daemon mds.g on smithi112 2024-05-08T05:50:35.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:35 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] up:boot 2024-05-08T05:50:35.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:35 smithi112 ceph-mon[30003]: fsmap 9 up:standby 2024-05-08T05:50:35.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:35 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:50:36.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:35 smithi150 ceph-mon[34315]: Deploying daemon mds.g on smithi112 2024-05-08T05:50:36.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:35 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] up:boot 2024-05-08T05:50:36.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:35 smithi150 ceph-mon[34315]: fsmap 9 up:standby 2024-05-08T05:50:36.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:35 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:50:36.737 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:36.889 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:36 smithi112 ceph-mon[30003]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:37.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:36 smithi162 ceph-mon[34800]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:37.186 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:36 smithi150 ceph-mon[34315]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:38.565 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 08 05:50:38 smithi112 podman[69424]: 2024-05-08 05:50:38.42442134 +0000 UTC m=+0.136314681 container create f5f0933b3f22d0b6350fa5001b8d3fa50efa351f2482d84157cf6b805e50749c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-g, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-08T05:50:38.902 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:38 smithi112 ceph-mon[30003]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:38.903 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 08 05:50:38 smithi112 podman[69424]: 2024-05-08 05:50:38.663670072 +0000 UTC m=+0.375563486 container init f5f0933b3f22d0b6350fa5001b8d3fa50efa351f2482d84157cf6b805e50749c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-g, ceph=True, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-08T05:50:38.903 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 08 05:50:38 smithi112 podman[69424]: 2024-05-08 05:50:38.673768735 +0000 UTC m=+0.385662059 container start f5f0933b3f22d0b6350fa5001b8d3fa50efa351f2482d84157cf6b805e50749c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-g, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-08T05:50:38.903 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 08 05:50:38 smithi112 ceph-mds[69461]: -- 172.21.15.112:0/2150386921 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e779505a40 con 0x55e7794d8800 2024-05-08T05:50:38.903 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 08 05:50:38 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-g[69455]: starting mds.g at 2024-05-08T05:50:38.903 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 08 05:50:38 smithi112 bash[69424]: f5f0933b3f22d0b6350fa5001b8d3fa50efa351f2482d84157cf6b805e50749c 2024-05-08T05:50:38.903 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 08 05:50:38 smithi112 systemd[1]: Started Ceph mds.g for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:50:39.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:38 smithi162 ceph-mon[34800]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:39.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:38 smithi150 ceph-mon[34315]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:39.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.396+0000 7f4d47c00700 1 -- 172.21.15.112:0/2423555312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d4006bb40 msgr2=0x7f4d40110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:39.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.396+0000 7f4d47c00700 1 --2- 172.21.15.112:0/2423555312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d4006bb40 0x7f4d40110330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800b0a0 tx=0x7f4d3800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:50:39.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.397+0000 7f4d47c00700 1 -- 172.21.15.112:0/2423555312 shutdown_connections 2024-05-08T05:50:39.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.397+0000 7f4d47c00700 1 --2- 172.21.15.112:0/2423555312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d4006bb40 0x7f4d40110330 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:39.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.397+0000 7f4d47c00700 1 --2- 172.21.15.112:0/2423555312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d40109510 0x7f4d401098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:39.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.397+0000 7f4d47c00700 1 --2- 172.21.15.112:0/2423555312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d4006b180 0x7f4d4006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:39.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.397+0000 7f4d47c00700 1 -- 172.21.15.112:0/2423555312 >> 172.21.15.112:0/2423555312 conn(0x7f4d40079350 msgr2=0x7f4d40079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:39.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.397+0000 7f4d47c00700 1 -- 172.21.15.112:0/2423555312 shutdown_connections 2024-05-08T05:50:39.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.397+0000 7f4d47c00700 1 -- 172.21.15.112:0/2423555312 wait complete. 2024-05-08T05:50:39.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.398+0000 7f4d47c00700 1 Processor -- start 2024-05-08T05:50:39.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.398+0000 7f4d47c00700 1 -- start start 2024-05-08T05:50:39.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.398+0000 7f4d47c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d4006b180 0x7f4d4019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:39.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.398+0000 7f4d47c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d4006bb40 0x7f4d4019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:39.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.398+0000 7f4d47c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40109510 0x7f4d40199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:39.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.398+0000 7f4d47c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d40113670 con 0x7f4d4006b180 2024-05-08T05:50:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.398+0000 7f4d47c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401134f0 con 0x7f4d40109510 2024-05-08T05:50:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.398+0000 7f4d47c00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401137f0 con 0x7f4d4006bb40 2024-05-08T05:50:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.399+0000 7f4d45200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d4006b180 0x7f4d4019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.399+0000 7f4d45200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d4006b180 0x7f4d4019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51892/0 (socket says 172.21.15.112:51892) 2024-05-08T05:50:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.399+0000 7f4d45200700 1 -- 172.21.15.112:0/245732940 learned_addr learned my addr 172.21.15.112:0/245732940 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:50:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.399+0000 7f4d45c00700 1 --2- 172.21.15.112:0/245732940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40109510 0x7f4d40199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.399+0000 7f4d3fe00700 1 --2- 172.21.15.112:0/245732940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d4006bb40 0x7f4d4019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.399+0000 7f4d45c00700 1 -- 172.21.15.112:0/245732940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d4006bb40 msgr2=0x7f4d4019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.399+0000 7f4d45c00700 1 --2- 172.21.15.112:0/245732940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d4006bb40 0x7f4d4019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.399+0000 7f4d45c00700 1 -- 172.21.15.112:0/245732940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d4006b180 msgr2=0x7f4d4019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.399+0000 7f4d45c00700 1 --2- 172.21.15.112:0/245732940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d4006b180 0x7f4d4019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.399+0000 7f4d45c00700 1 -- 172.21.15.112:0/245732940 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4019a0d0 con 0x7f4d40109510 2024-05-08T05:50:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.399+0000 7f4d45200700 1 --2- 172.21.15.112:0/245732940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d4006b180 0x7f4d4019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.400+0000 7f4d45c00700 1 --2- 172.21.15.112:0/245732940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40109510 0x7f4d40199970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800b070 tx=0x7f4d38004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.400+0000 7f4d3d600700 1 -- 172.21.15.112:0/245732940 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38021070 con 0x7f4d40109510 2024-05-08T05:50:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.400+0000 7f4d47c00700 1 -- 172.21.15.112:0/245732940 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4019a360 con 0x7f4d40109510 2024-05-08T05:50:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.400+0000 7f4d3d600700 1 -- 172.21.15.112:0/245732940 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d38007e30 con 0x7f4d40109510 2024-05-08T05:50:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.400+0000 7f4d3d600700 1 -- 172.21.15.112:0/245732940 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d380045f0 con 0x7f4d40109510 2024-05-08T05:50:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.400+0000 7f4d47c00700 1 -- 172.21.15.112:0/245732940 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d401a6630 con 0x7f4d40109510 2024-05-08T05:50:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.401+0000 7f4d47c00700 1 -- 172.21.15.112:0/245732940 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d4006a190 con 0x7f4d40109510 2024-05-08T05:50:39.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.403+0000 7f4d3d600700 1 -- 172.21.15.112:0/245732940 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f4d38007980 con 0x7f4d40109510 2024-05-08T05:50:39.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.405+0000 7f4d3d600700 1 --2- 172.21.15.112:0/245732940 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4d280b2880 0x7f4d280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:39.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.405+0000 7f4d3d600700 1 -- 172.21.15.112:0/245732940 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f4d380e2c00 con 0x7f4d40109510 2024-05-08T05:50:39.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.405+0000 7f4d45200700 1 --2- 172.21.15.112:0/245732940 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4d280b2880 0x7f4d280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:39.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.406+0000 7f4d45200700 1 --2- 172.21.15.112:0/245732940 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4d280b2880 0x7f4d280b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4d40100a70 tx=0x7f4d34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:39.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.407+0000 7f4d3d600700 1 -- 172.21.15.112:0/245732940 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4d380aadb0 con 0x7f4d40109510 2024-05-08T05:50:39.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.689+0000 7f4d47c00700 1 -- 172.21.15.112:0/245732940 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d4002e510 con 0x7f4d40109510 2024-05-08T05:50:39.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.690+0000 7f4d3d600700 1 -- 172.21.15.112:0/245732940 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7f4d380aa6b0 con 0x7f4d40109510 2024-05-08T05:50:39.690 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:50:39.690 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-08T05:50:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.693+0000 7f4d47c00700 1 -- 172.21.15.112:0/245732940 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4d280b2880 msgr2=0x7f4d280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.693+0000 7f4d47c00700 1 --2- 172.21.15.112:0/245732940 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4d280b2880 0x7f4d280b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4d40100a70 tx=0x7f4d34009260 comp rx=0 tx=0).stop 2024-05-08T05:50:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.693+0000 7f4d47c00700 1 -- 172.21.15.112:0/245732940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40109510 msgr2=0x7f4d40199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.693+0000 7f4d47c00700 1 --2- 172.21.15.112:0/245732940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40109510 0x7f4d40199970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800b070 tx=0x7f4d38004c80 comp rx=0 tx=0).stop 2024-05-08T05:50:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.693+0000 7f4d47c00700 1 -- 172.21.15.112:0/245732940 shutdown_connections 2024-05-08T05:50:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.693+0000 7f4d47c00700 1 --2- 172.21.15.112:0/245732940 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4d280b2880 0x7f4d280b4d40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.693+0000 7f4d47c00700 1 --2- 172.21.15.112:0/245732940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d40109510 0x7f4d40199970 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.693+0000 7f4d47c00700 1 --2- 172.21.15.112:0/245732940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d4006b180 0x7f4d4019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.693+0000 7f4d47c00700 1 --2- 172.21.15.112:0/245732940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d4006bb40 0x7f4d4019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.693+0000 7f4d47c00700 1 -- 172.21.15.112:0/245732940 >> 172.21.15.112:0/245732940 conn(0x7f4d40079350 msgr2=0x7f4d401010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.693+0000 7f4d47c00700 1 -- 172.21.15.112:0/245732940 shutdown_connections 2024-05-08T05:50:39.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:39.693+0000 7f4d47c00700 1 -- 172.21.15.112:0/245732940 wait complete. 2024-05-08T05:50:39.695 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 10 2024-05-08T05:50:40.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:39 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:40.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:39 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:40.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:39 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:40.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:39 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:40.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:39 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:40.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:39 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:40.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:39 smithi112 ceph-mon[30003]: Deploying daemon mds.a on smithi112 2024-05-08T05:50:40.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:39 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/245732940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:40.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:39 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.112:6838/3014879788,v1:172.21.15.112:6839/3014879788] up:boot 2024-05-08T05:50:40.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:39 smithi112 ceph-mon[30003]: fsmap 10 up:standby 2024-05-08T05:50:40.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:39 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:50:40.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:39 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:40.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:39 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:40.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:39 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:40.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:39 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:40.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:39 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:40.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:39 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:40.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:39 smithi150 ceph-mon[34315]: Deploying daemon mds.a on smithi112 2024-05-08T05:50:40.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:39 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/245732940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:40.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:39 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.112:6838/3014879788,v1:172.21.15.112:6839/3014879788] up:boot 2024-05-08T05:50:40.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:39 smithi150 ceph-mon[34315]: fsmap 10 up:standby 2024-05-08T05:50:40.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:39 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:50:40.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:39 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:40.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:39 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:40.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:39 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:40.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:39 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:40.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:39 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:40.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:39 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:40.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:39 smithi162 ceph-mon[34800]: Deploying daemon mds.a on smithi112 2024-05-08T05:50:40.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:39 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/245732940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:40.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:39 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.112:6838/3014879788,v1:172.21.15.112:6839/3014879788] up:boot 2024-05-08T05:50:40.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:39 smithi162 ceph-mon[34800]: fsmap 10 up:standby 2024-05-08T05:50:40.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:39 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:50:41.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:40 smithi112 ceph-mon[30003]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:41.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:40 smithi162 ceph-mon[34800]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:41.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:40 smithi150 ceph-mon[34315]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:42.464 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:50:42.816 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 05:50:42 smithi112 systemd[1]: Starting Ceph mds.a for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:50:43.215 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:42 smithi112 ceph-mon[30003]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:43.216 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 05:50:42 smithi112 podman[69820]: 2024-05-08 05:50:42.864549602 +0000 UTC m=+0.042997047 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:50:43.216 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 05:50:43 smithi112 podman[69820]: 2024-05-08 05:50:43.003278828 +0000 UTC m=+0.181726347 container create b7a24dd92d926e507edb14783a33c4ddf6f15db875377ad0659cc98f04929c70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-05-08T05:50:43.216 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 05:50:43 smithi112 podman[69820]: 2024-05-08 05:50:43.215482333 +0000 UTC m=+0.393929822 container init b7a24dd92d926e507edb14783a33c4ddf6f15db875377ad0659cc98f04929c70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-a, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:50:43.301 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:43.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:42 smithi162 ceph-mon[34800]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:43.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:42 smithi150 ceph-mon[34315]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:43.515 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 05:50:43 smithi112 podman[69820]: 2024-05-08 05:50:43.223689483 +0000 UTC m=+0.402136928 container start b7a24dd92d926e507edb14783a33c4ddf6f15db875377ad0659cc98f04929c70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:50:43.515 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 05:50:43 smithi112 ceph-mds[69860]: -- 172.21.15.112:0/3504321592 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e5f73da40 con 0x556e5f710c00 2024-05-08T05:50:43.515 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 05:50:43 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-a[69856]: starting mds.a at 2024-05-08T05:50:43.515 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 05:50:43 smithi112 bash[69820]: b7a24dd92d926e507edb14783a33c4ddf6f15db875377ad0659cc98f04929c70 2024-05-08T05:50:43.515 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 05:50:43 smithi112 systemd[1]: Started Ceph mds.a for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:50:44.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:44.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:44.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:44.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:44.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:44.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:44.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:44 smithi150 ceph-mon[34315]: Deploying daemon mds.b on smithi150 2024-05-08T05:50:44.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:44 smithi150 ceph-mon[34315]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:44.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:44 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] up:boot 2024-05-08T05:50:44.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:44 smithi150 ceph-mon[34315]: fsmap 11 up:standby 2024-05-08T05:50:44.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:50:44.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:44.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:44.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:44.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:44.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:44.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:44.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:44 smithi112 ceph-mon[30003]: Deploying daemon mds.b on smithi150 2024-05-08T05:50:44.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:44 smithi112 ceph-mon[30003]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:44.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:44 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] up:boot 2024-05-08T05:50:44.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:44 smithi112 ceph-mon[30003]: fsmap 11 up:standby 2024-05-08T05:50:44.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:50:44.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:44.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:44.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:44.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:50:44.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:50:44.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:44.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:44 smithi162 ceph-mon[34800]: Deploying daemon mds.b on smithi150 2024-05-08T05:50:44.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:44 smithi162 ceph-mon[34800]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:44.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:44 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] up:boot 2024-05-08T05:50:44.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:44 smithi162 ceph-mon[34800]: fsmap 11 up:standby 2024-05-08T05:50:44.818 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:50:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.571+0000 7efd1ba00700 1 -- 172.21.15.112:0/625985052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd14114670 msgr2=0x7efd14116a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.571+0000 7efd1ba00700 1 --2- 172.21.15.112:0/625985052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd14114670 0x7efd14116a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009990 tx=0x7efd0c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.571+0000 7efd1ba00700 1 -- 172.21.15.112:0/625985052 shutdown_connections 2024-05-08T05:50:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.571+0000 7efd1ba00700 1 --2- 172.21.15.112:0/625985052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd14114670 0x7efd14116a60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.571+0000 7efd1ba00700 1 --2- 172.21.15.112:0/625985052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd140756d0 0x7efd14075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.571+0000 7efd1ba00700 1 --2- 172.21.15.112:0/625985052 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd14075ff0 0x7efd14114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.571+0000 7efd1ba00700 1 -- 172.21.15.112:0/625985052 >> 172.21.15.112:0/625985052 conn(0x7efd1407a600 msgr2=0x7efd1407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.571+0000 7efd1ba00700 1 -- 172.21.15.112:0/625985052 shutdown_connections 2024-05-08T05:50:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.572+0000 7efd1ba00700 1 -- 172.21.15.112:0/625985052 wait complete. 2024-05-08T05:50:45.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.572+0000 7efd1ba00700 1 Processor -- start 2024-05-08T05:50:45.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.573+0000 7efd1ba00700 1 -- start start 2024-05-08T05:50:45.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.573+0000 7efd1ba00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd140756d0 0x7efd141a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.573+0000 7efd1ba00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd14075ff0 0x7efd141a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.573+0000 7efd1ba00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd14114670 0x7efd141a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.573+0000 7efd1ba00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd14119c50 con 0x7efd14114670 2024-05-08T05:50:45.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.573+0000 7efd1ba00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efd14119ad0 con 0x7efd14075ff0 2024-05-08T05:50:45.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.573+0000 7efd1ba00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7efd14119dd0 con 0x7efd140756d0 2024-05-08T05:50:45.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.573+0000 7efd19000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd140756d0 0x7efd141a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.573+0000 7efd19a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd14114670 0x7efd141a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.573+0000 7efd19000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd140756d0 0x7efd141a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:57836/0 (socket says 172.21.15.112:57836) 2024-05-08T05:50:45.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.573+0000 7efd0be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd14075ff0 0x7efd141a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.573+0000 7efd19000700 1 -- 172.21.15.112:0/847233806 learned_addr learned my addr 172.21.15.112:0/847233806 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:50:45.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.574+0000 7efd19000700 1 -- 172.21.15.112:0/847233806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd14075ff0 msgr2=0x7efd141a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.574+0000 7efd19000700 1 --2- 172.21.15.112:0/847233806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd14075ff0 0x7efd141a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.574+0000 7efd19000700 1 -- 172.21.15.112:0/847233806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd14114670 msgr2=0x7efd141a49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.574+0000 7efd19000700 1 --2- 172.21.15.112:0/847233806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd14114670 0x7efd141a49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.574+0000 7efd19000700 1 -- 172.21.15.112:0/847233806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd141a5130 con 0x7efd140756d0 2024-05-08T05:50:45.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.574+0000 7efd19000700 1 --2- 172.21.15.112:0/847233806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd140756d0 0x7efd141a0120 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd0400ef10 tx=0x7efd0400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:45.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.574+0000 7efd12000700 1 -- 172.21.15.112:0/847233806 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0400c9c0 con 0x7efd140756d0 2024-05-08T05:50:45.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.575+0000 7efd1ba00700 1 -- 172.21.15.112:0/847233806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd141a53c0 con 0x7efd140756d0 2024-05-08T05:50:45.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.575+0000 7efd12000700 1 -- 172.21.15.112:0/847233806 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd04019070 con 0x7efd140756d0 2024-05-08T05:50:45.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.575+0000 7efd12000700 1 -- 172.21.15.112:0/847233806 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04014480 con 0x7efd140756d0 2024-05-08T05:50:45.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.575+0000 7efd1ba00700 1 -- 172.21.15.112:0/847233806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd141acca0 con 0x7efd140756d0 2024-05-08T05:50:45.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.576+0000 7efd1ba00700 1 -- 172.21.15.112:0/847233806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd1406a190 con 0x7efd140756d0 2024-05-08T05:50:45.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.578+0000 7efd12000700 1 -- 172.21.15.112:0/847233806 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7efd04012050 con 0x7efd140756d0 2024-05-08T05:50:45.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.580+0000 7efd12000700 1 --2- 172.21.15.112:0/847233806 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efcf80b2900 0x7efcf80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.580+0000 7efd12000700 1 -- 172.21.15.112:0/847233806 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7efd040e23a0 con 0x7efd140756d0 2024-05-08T05:50:45.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.580+0000 7efd0be00700 1 --2- 172.21.15.112:0/847233806 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efcf80b2900 0x7efcf80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.581+0000 7efd0be00700 1 --2- 172.21.15.112:0/847233806 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efcf80b2900 0x7efcf80b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efd14110f90 tx=0x7efd00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:45.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.582+0000 7efd12000700 1 -- 172.21.15.112:0/847233806 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd04010040 con 0x7efd140756d0 2024-05-08T05:50:45.683 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 05:50:45 smithi150 systemd[1]: Starting Ceph mds.b for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:50:45.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.860+0000 7efd1ba00700 1 -- 172.21.15.112:0/847233806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd141a1480 con 0x7efd140756d0 2024-05-08T05:50:45.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.861+0000 7efd12000700 1 -- 172.21.15.112:0/847233806 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9674 (secure 0 0 0) 0x7efd141a1480 con 0x7efd140756d0 2024-05-08T05:50:45.861 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:50:45.861 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-05-08T05:50:45.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.866+0000 7efd1ba00700 1 -- 172.21.15.112:0/847233806 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efcf80b2900 msgr2=0x7efcf80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.866+0000 7efd1ba00700 1 --2- 172.21.15.112:0/847233806 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efcf80b2900 0x7efcf80b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efd14110f90 tx=0x7efd00009260 comp rx=0 tx=0).stop 2024-05-08T05:50:45.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.866+0000 7efd1ba00700 1 -- 172.21.15.112:0/847233806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd140756d0 msgr2=0x7efd141a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.866+0000 7efd1ba00700 1 --2- 172.21.15.112:0/847233806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd140756d0 0x7efd141a0120 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd0400ef10 tx=0x7efd0400c3a0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.867+0000 7efd1ba00700 1 -- 172.21.15.112:0/847233806 shutdown_connections 2024-05-08T05:50:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.867+0000 7efd1ba00700 1 --2- 172.21.15.112:0/847233806 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efcf80b2900 0x7efcf80b4dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.867+0000 7efd1ba00700 1 --2- 172.21.15.112:0/847233806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd14114670 0x7efd141a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.867+0000 7efd1ba00700 1 --2- 172.21.15.112:0/847233806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd140756d0 0x7efd141a0120 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.867+0000 7efd1ba00700 1 --2- 172.21.15.112:0/847233806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd14075ff0 0x7efd141a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.867+0000 7efd1ba00700 1 -- 172.21.15.112:0/847233806 >> 172.21.15.112:0/847233806 conn(0x7efd1407a600 msgr2=0x7efd141134a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.867+0000 7efd1ba00700 1 -- 172.21.15.112:0/847233806 shutdown_connections 2024-05-08T05:50:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:45.867+0000 7efd1ba00700 1 -- 172.21.15.112:0/847233806 wait complete. 2024-05-08T05:50:45.868 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 12 2024-05-08T05:50:45.939 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 05:50:45 smithi150 podman[64150]: 2024-05-08 05:50:45.59210003 +0000 UTC m=+0.017833721 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:50:45.939 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 05:50:45 smithi150 podman[64150]: 2024-05-08 05:50:45.717887948 +0000 UTC m=+0.143621625 container create 7dc66aadba577ce6bf384b27537676732d371887822376268dc4ef039052680e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-b, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-08T05:50:45.939 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 05:50:45 smithi150 podman[64150]: 2024-05-08 05:50:45.937278903 +0000 UTC m=+0.363012580 container init 7dc66aadba577ce6bf384b27537676732d371887822376268dc4ef039052680e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-b, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-08T05:50:46.433 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 05:50:45 smithi150 podman[64150]: 2024-05-08 05:50:45.939780524 +0000 UTC m=+0.365514201 container start 7dc66aadba577ce6bf384b27537676732d371887822376268dc4ef039052680e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-08T05:50:46.433 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 05:50:45 smithi150 ceph-mds[64169]: -- 172.21.15.150:0/1767215248 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5574c3db5a40 con 0x5574c3d88c00 2024-05-08T05:50:46.433 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 05:50:45 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-b[64165]: starting mds.b at 2024-05-08T05:50:46.433 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 05:50:46 smithi150 bash[64150]: 7dc66aadba577ce6bf384b27537676732d371887822376268dc4ef039052680e 2024-05-08T05:50:46.433 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 05:50:46 smithi150 systemd[1]: Started Ceph mds.b for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:50:46.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:46 smithi112 ceph-mon[30003]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:46.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:46 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/847233806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:46.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:46 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:46.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:46 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:46.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:46 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:46.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:46 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:46.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:46 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:46.817 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:46 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:50:46.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:46 smithi162 ceph-mon[34800]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:46.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:46 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/847233806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:46.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:46 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:46.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:46 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:46.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:46 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:46.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:46 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:46.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:46 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:46.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:46 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:50:46.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:46 smithi150 ceph-mon[34315]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:46.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:46 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/847233806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:46.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:46 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:46.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:46 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:46.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:46 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:46.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:46 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:46.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:46 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:46.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:46 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:50:47.748 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:47 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] up:boot 2024-05-08T05:50:47.748 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:47 smithi150 ceph-mon[34315]: fsmap 12 up:standby 2024-05-08T05:50:47.748 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:47 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:50:47.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:47 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] up:boot 2024-05-08T05:50:47.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:47 smithi112 ceph-mon[30003]: fsmap 12 up:standby 2024-05-08T05:50:47.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:47 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:50:47.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:47 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] up:boot 2024-05-08T05:50:47.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:47 smithi162 ceph-mon[34800]: fsmap 12 up:standby 2024-05-08T05:50:47.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:47 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:50:48.750 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:50:49.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:49 smithi112 ceph-mon[30003]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:49.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:49 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:49.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:49 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:49.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:49 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:49.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:49 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:49.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:49 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:49.204 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:49.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:49 smithi162 ceph-mon[34800]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:49.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:49 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:49.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:49 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:49.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:49 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:49.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:49 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:49.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:49 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:49.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:49 smithi150 ceph-mon[34315]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:49.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:49 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:49.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:49 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:49.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:49 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:49.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:49 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:49.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:49 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:50.418 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:50.418 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:50.418 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:50.418 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:50:50.418 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:50.418 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:50 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:50:50.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:50.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:50.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:50.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:50:50.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:50.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:50 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:50:50.458 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:50.458 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:50.458 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:50.458 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:50:50.458 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:50.459 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:50 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:50:51.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.269+0000 7fb264e00700 1 -- 172.21.15.112:0/385981101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2600756d0 msgr2=0x7fb260075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.269+0000 7fb264e00700 1 --2- 172.21.15.112:0/385981101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2600756d0 0x7fb260075ab0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb2500098f0 tx=0x7fb250009c00 comp rx=0 tx=0).stop 2024-05-08T05:50:51.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.269+0000 7fb264e00700 1 -- 172.21.15.112:0/385981101 shutdown_connections 2024-05-08T05:50:51.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.270+0000 7fb264e00700 1 --2- 172.21.15.112:0/385981101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb26010fea0 0x7fb260112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.270+0000 7fb264e00700 1 --2- 172.21.15.112:0/385981101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2600756d0 0x7fb260075ab0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.270+0000 7fb264e00700 1 --2- 172.21.15.112:0/385981101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb260075ff0 0x7fb26010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.270+0000 7fb264e00700 1 -- 172.21.15.112:0/385981101 >> 172.21.15.112:0/385981101 conn(0x7fb26007a600 msgr2=0x7fb26007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:51.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.270+0000 7fb264e00700 1 -- 172.21.15.112:0/385981101 shutdown_connections 2024-05-08T05:50:51.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.270+0000 7fb264e00700 1 -- 172.21.15.112:0/385981101 wait complete. 2024-05-08T05:50:51.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.271+0000 7fb264e00700 1 Processor -- start 2024-05-08T05:50:51.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.271+0000 7fb264e00700 1 -- start start 2024-05-08T05:50:51.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb264e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2600756d0 0x7fb26019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb264e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb260075ff0 0x7fb26019c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb264e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26010fea0 0x7fb2601a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb264e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2601158a0 con 0x7fb260075ff0 2024-05-08T05:50:51.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb264e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb260115720 con 0x7fb2600756d0 2024-05-08T05:50:51.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb264e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb260115a20 con 0x7fb26010fea0 2024-05-08T05:50:51.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb25de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2600756d0 0x7fb26019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb25de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2600756d0 0x7fb26019bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:45736/0 (socket says 172.21.15.112:45736) 2024-05-08T05:50:51.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb25de00700 1 -- 172.21.15.112:0/916316978 learned_addr learned my addr 172.21.15.112:0/916316978 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:50:51.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb25d400700 1 --2- 172.21.15.112:0/916316978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb260075ff0 0x7fb26019c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb25e800700 1 --2- 172.21.15.112:0/916316978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26010fea0 0x7fb2601a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb25e800700 1 -- 172.21.15.112:0/916316978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2600756d0 msgr2=0x7fb26019bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb25e800700 1 --2- 172.21.15.112:0/916316978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2600756d0 0x7fb26019bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb25e800700 1 -- 172.21.15.112:0/916316978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb260075ff0 msgr2=0x7fb26019c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb25e800700 1 --2- 172.21.15.112:0/916316978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb260075ff0 0x7fb26019c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb25e800700 1 -- 172.21.15.112:0/916316978 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2601a0da0 con 0x7fb26010fea0 2024-05-08T05:50:51.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.272+0000 7fb25d400700 1 --2- 172.21.15.112:0/916316978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb260075ff0 0x7fb26019c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:51.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.273+0000 7fb25e800700 1 --2- 172.21.15.112:0/916316978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26010fea0 0x7fb2601a0660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb258007ee0 tx=0x7fb25800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:51.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.273+0000 7fb256a00700 1 -- 172.21.15.112:0/916316978 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb258004db0 con 0x7fb26010fea0 2024-05-08T05:50:51.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.273+0000 7fb256a00700 1 -- 172.21.15.112:0/916316978 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb258003a00 con 0x7fb26010fea0 2024-05-08T05:50:51.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.273+0000 7fb264e00700 1 -- 172.21.15.112:0/916316978 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2601a1090 con 0x7fb26010fea0 2024-05-08T05:50:51.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.273+0000 7fb256a00700 1 -- 172.21.15.112:0/916316978 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2580044e0 con 0x7fb26010fea0 2024-05-08T05:50:51.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.273+0000 7fb264e00700 1 -- 172.21.15.112:0/916316978 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2601a8910 con 0x7fb26010fea0 2024-05-08T05:50:51.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.274+0000 7fb264e00700 1 -- 172.21.15.112:0/916316978 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb26006a190 con 0x7fb26010fea0 2024-05-08T05:50:51.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.276+0000 7fb256a00700 1 -- 172.21.15.112:0/916316978 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fb25801c020 con 0x7fb26010fea0 2024-05-08T05:50:51.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.277+0000 7fb256a00700 1 --2- 172.21.15.112:0/916316978 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb2440b2900 0x7fb2440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.278+0000 7fb256a00700 1 -- 172.21.15.112:0/916316978 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fb2580e1490 con 0x7fb26010fea0 2024-05-08T05:50:51.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.278+0000 7fb25de00700 1 --2- 172.21.15.112:0/916316978 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb2440b2900 0x7fb2440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.278+0000 7fb25de00700 1 --2- 172.21.15.112:0/916316978 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb2440b2900 0x7fb2440b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb250009680 tx=0x7fb250019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:51.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.282+0000 7fb256a00700 1 -- 172.21.15.112:0/916316978 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb258051df0 con 0x7fb26010fea0 2024-05-08T05:50:51.459 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:51 smithi112 ceph-mon[30003]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:51.459 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:51 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:51.460 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:51 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:51.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.563+0000 7fb264e00700 1 -- 172.21.15.112:0/916316978 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb26002e510 con 0x7fb26010fea0 2024-05-08T05:50:51.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.564+0000 7fb256a00700 1 -- 172.21.15.112:0/916316978 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10503 (secure 0 0 0) 0x7fb258010070 con 0x7fb26010fea0 2024-05-08T05:50:51.564 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:50:51.564 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24518,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-05-08T05:50:51.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.567+0000 7fb264e00700 1 -- 172.21.15.112:0/916316978 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb2440b2900 msgr2=0x7fb2440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.567+0000 7fb264e00700 1 --2- 172.21.15.112:0/916316978 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb2440b2900 0x7fb2440b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb250009680 tx=0x7fb250019040 comp rx=0 tx=0).stop 2024-05-08T05:50:51.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.567+0000 7fb264e00700 1 -- 172.21.15.112:0/916316978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26010fea0 msgr2=0x7fb2601a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.567+0000 7fb264e00700 1 --2- 172.21.15.112:0/916316978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26010fea0 0x7fb2601a0660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb258007ee0 tx=0x7fb25800d3b0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.567+0000 7fb264e00700 1 -- 172.21.15.112:0/916316978 shutdown_connections 2024-05-08T05:50:51.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.567+0000 7fb264e00700 1 --2- 172.21.15.112:0/916316978 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb2440b2900 0x7fb2440b4dc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.567+0000 7fb264e00700 1 --2- 172.21.15.112:0/916316978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26010fea0 0x7fb2601a0660 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.567+0000 7fb264e00700 1 --2- 172.21.15.112:0/916316978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2600756d0 0x7fb26019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.567+0000 7fb264e00700 1 --2- 172.21.15.112:0/916316978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb260075ff0 0x7fb26019c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.567+0000 7fb264e00700 1 -- 172.21.15.112:0/916316978 >> 172.21.15.112:0/916316978 conn(0x7fb26007a600 msgr2=0x7fb2601006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:51.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.567+0000 7fb264e00700 1 -- 172.21.15.112:0/916316978 shutdown_connections 2024-05-08T05:50:51.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:51.567+0000 7fb264e00700 1 -- 172.21.15.112:0/916316978 wait complete. 2024-05-08T05:50:51.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:51 smithi162 ceph-mon[34800]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:51.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:51 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:51.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:51 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:51.568 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 13 2024-05-08T05:50:51.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:51 smithi150 ceph-mon[34315]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:51.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:51 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:51.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:51 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:52.399 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-08T05:50:52.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:52 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:52.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:52 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:52.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:52 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/916316978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:52.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:52 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:52.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:52 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:52.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:52 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/916316978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:52.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:52 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:52.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:52 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:52.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:52 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/916316978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:52.847 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:53.295 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:53 smithi112 ceph-mon[30003]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:53.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:53 smithi162 ceph-mon[34800]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:53.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:53 smithi150 ceph-mon[34315]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:54.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.836+0000 7f7f75e00700 1 -- 172.21.15.112:0/2424676248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f701056c0 msgr2=0x7f7f70105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:54.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.836+0000 7f7f75e00700 1 --2- 172.21.15.112:0/2424676248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f701056c0 0x7f7f70105b40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009990 tx=0x7f7f5c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:50:54.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.836+0000 7f7f75e00700 1 -- 172.21.15.112:0/2424676248 shutdown_connections 2024-05-08T05:50:54.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.836+0000 7f7f75e00700 1 --2- 172.21.15.112:0/2424676248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f70106080 0x7f7f70112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:54.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.836+0000 7f7f75e00700 1 --2- 172.21.15.112:0/2424676248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f7010b6c0 0x7f7f7010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:54.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.836+0000 7f7f75e00700 1 --2- 172.21.15.112:0/2424676248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f701056c0 0x7f7f70105b40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:54.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.836+0000 7f7f75e00700 1 -- 172.21.15.112:0/2424676248 >> 172.21.15.112:0/2424676248 conn(0x7f7f701011b0 msgr2=0x7f7f701035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:54.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.836+0000 7f7f75e00700 1 -- 172.21.15.112:0/2424676248 shutdown_connections 2024-05-08T05:50:54.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.836+0000 7f7f75e00700 1 -- 172.21.15.112:0/2424676248 wait complete. 2024-05-08T05:50:54.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.837+0000 7f7f75e00700 1 Processor -- start 2024-05-08T05:50:54.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.837+0000 7f7f75e00700 1 -- start start 2024-05-08T05:50:54.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.838+0000 7f7f75e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f701056c0 0x7f7f7019bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:54.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.838+0000 7f7f75e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f70106080 0x7f7f7019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:54.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.838+0000 7f7f75e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f7010b6c0 0x7f7f701a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:54.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.838+0000 7f7f75e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f70115950 con 0x7f7f701056c0 2024-05-08T05:50:54.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.838+0000 7f7f75e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7f701157d0 con 0x7f7f70106080 2024-05-08T05:50:54.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.838+0000 7f7f75e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7f70115ad0 con 0x7f7f7010b6c0 2024-05-08T05:50:54.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.838+0000 7f7f6ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f701056c0 0x7f7f7019bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:54.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.838+0000 7f7f6ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f701056c0 0x7f7f7019bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39820/0 (socket says 172.21.15.112:39820) 2024-05-08T05:50:54.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.838+0000 7f7f6ea00700 1 -- 172.21.15.112:0/2103006290 learned_addr learned my addr 172.21.15.112:0/2103006290 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:50:54.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.838+0000 7f7f6e000700 1 --2- 172.21.15.112:0/2103006290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f70106080 0x7f7f7019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:54.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.838+0000 7f7f6f400700 1 --2- 172.21.15.112:0/2103006290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f7010b6c0 0x7f7f701a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:54.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.839+0000 7f7f6ea00700 1 -- 172.21.15.112:0/2103006290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f7010b6c0 msgr2=0x7f7f701a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:54.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.839+0000 7f7f6ea00700 1 --2- 172.21.15.112:0/2103006290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f7010b6c0 0x7f7f701a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:54.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.839+0000 7f7f6ea00700 1 -- 172.21.15.112:0/2103006290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f70106080 msgr2=0x7f7f7019c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:54.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.839+0000 7f7f6ea00700 1 --2- 172.21.15.112:0/2103006290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f70106080 0x7f7f7019c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:54.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.839+0000 7f7f6ea00700 1 -- 172.21.15.112:0/2103006290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f701a0d90 con 0x7f7f701056c0 2024-05-08T05:50:54.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.839+0000 7f7f6e000700 1 --2- 172.21.15.112:0/2103006290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f70106080 0x7f7f7019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:54.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.839+0000 7f7f6ea00700 1 --2- 172.21.15.112:0/2103006290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f701056c0 0x7f7f7019bd80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000e9c0 tx=0x7f7f6000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:54.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.839+0000 7f7f74a00700 1 -- 172.21.15.112:0/2103006290 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6000cbd0 con 0x7f7f701056c0 2024-05-08T05:50:54.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.839+0000 7f7f74a00700 1 -- 172.21.15.112:0/2103006290 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f6000ee60 con 0x7f7f701056c0 2024-05-08T05:50:54.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.839+0000 7f7f74a00700 1 -- 172.21.15.112:0/2103006290 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f600055c0 con 0x7f7f701056c0 2024-05-08T05:50:54.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.839+0000 7f7f75e00700 1 -- 172.21.15.112:0/2103006290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f701a1080 con 0x7f7f701056c0 2024-05-08T05:50:54.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.840+0000 7f7f75e00700 1 -- 172.21.15.112:0/2103006290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f701a8900 con 0x7f7f701056c0 2024-05-08T05:50:54.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.841+0000 7f7f75e00700 1 -- 172.21.15.112:0/2103006290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f7010dc50 con 0x7f7f701056c0 2024-05-08T05:50:54.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.843+0000 7f7f74a00700 1 -- 172.21.15.112:0/2103006290 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f7f60004750 con 0x7f7f701056c0 2024-05-08T05:50:54.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.844+0000 7f7f74a00700 1 --2- 172.21.15.112:0/2103006290 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7f540b2980 0x7f7f540b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:54.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.845+0000 7f7f74a00700 1 -- 172.21.15.112:0/2103006290 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f7f600e21d0 con 0x7f7f701056c0 2024-05-08T05:50:54.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.845+0000 7f7f6e000700 1 --2- 172.21.15.112:0/2103006290 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7f540b2980 0x7f7f540b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:54.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.845+0000 7f7f6e000700 1 --2- 172.21.15.112:0/2103006290 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7f540b2980 0x7f7f540b4e40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7f70103820 tx=0x7f7f5c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:54.847+0000 7f7f74a00700 1 -- 172.21.15.112:0/2103006290 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f60054df0 con 0x7f7f701056c0 2024-05-08T05:50:55.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:55.100+0000 7f7f75e00700 1 -- 172.21.15.112:0/2103006290 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7f7006a190 con 0x7f7f701056c0 2024-05-08T05:50:55.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:55.100+0000 7f7f74a00700 1 -- 172.21.15.112:0/2103006290 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7f7f600aa2f0 con 0x7f7f701056c0 2024-05-08T05:50:55.101 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:50:55.101 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-05-08T05:50:55.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T05:50:55.102 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-05-08T05:50:55.102 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-05-08T05:50:55.102 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T05:50:55.102 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-05-08T05:50:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:55.103+0000 7f7f75e00700 1 -- 172.21.15.112:0/2103006290 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7f540b2980 msgr2=0x7f7f540b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:55.103+0000 7f7f75e00700 1 --2- 172.21.15.112:0/2103006290 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7f540b2980 0x7f7f540b4e40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7f70103820 tx=0x7f7f5c01a040 comp rx=0 tx=0).stop 2024-05-08T05:50:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:55.103+0000 7f7f75e00700 1 -- 172.21.15.112:0/2103006290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f701056c0 msgr2=0x7f7f7019bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:55.103+0000 7f7f75e00700 1 --2- 172.21.15.112:0/2103006290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f701056c0 0x7f7f7019bd80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000e9c0 tx=0x7f7f6000ecd0 comp rx=0 tx=0).stop 2024-05-08T05:50:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:55.103+0000 7f7f75e00700 1 -- 172.21.15.112:0/2103006290 shutdown_connections 2024-05-08T05:50:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:55.103+0000 7f7f75e00700 1 --2- 172.21.15.112:0/2103006290 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7f540b2980 0x7f7f540b4e40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:55.103+0000 7f7f75e00700 1 --2- 172.21.15.112:0/2103006290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f7010b6c0 0x7f7f701a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:55.103+0000 7f7f75e00700 1 --2- 172.21.15.112:0/2103006290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f701056c0 0x7f7f7019bd80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:55.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:55.103+0000 7f7f75e00700 1 --2- 172.21.15.112:0/2103006290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f70106080 0x7f7f7019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:55.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:55.104+0000 7f7f75e00700 1 -- 172.21.15.112:0/2103006290 >> 172.21.15.112:0/2103006290 conn(0x7f7f701011b0 msgr2=0x7f7f70101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:55.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:55.104+0000 7f7f75e00700 1 -- 172.21.15.112:0/2103006290 shutdown_connections 2024-05-08T05:50:55.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:55.104+0000 7f7f75e00700 1 -- 172.21.15.112:0/2103006290 wait complete. 2024-05-08T05:50:55.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:55 smithi150 ceph-mon[34315]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:55.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:55.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:55.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:55.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:50:55.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:55 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:55.486 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:55 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/2103006290' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T05:50:55.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:55 smithi112 ceph-mon[30003]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:55.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:55.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:55.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:55.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:50:55.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:55 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:55.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:55 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2103006290' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T05:50:55.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:55 smithi162 ceph-mon[34800]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:55.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:55.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:55.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:55.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:50:55.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:55 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:50:55.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:55 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/2103006290' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T05:50:56.034 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T05:50:56.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T05:50:56.480 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:57.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:57 smithi112 ceph-mon[30003]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:57.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:57 smithi162 ceph-mon[34800]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:57.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:57 smithi150 ceph-mon[34315]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.656+0000 7f0206800700 1 -- 172.21.15.112:0/1841171640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020006bb40 msgr2=0x7f0200110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.656+0000 7f0206800700 1 --2- 172.21.15.112:0/1841171640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020006bb40 0x7f0200110330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f01f800b0a0 tx=0x7f01f800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:50:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.656+0000 7f0206800700 1 -- 172.21.15.112:0/1841171640 shutdown_connections 2024-05-08T05:50:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.656+0000 7f0206800700 1 --2- 172.21.15.112:0/1841171640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020006bb40 0x7f0200110330 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.656+0000 7f0206800700 1 --2- 172.21.15.112:0/1841171640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0200109510 0x7f02001098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.656+0000 7f0206800700 1 --2- 172.21.15.112:0/1841171640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f020006b180 0x7f020006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.656+0000 7f0206800700 1 -- 172.21.15.112:0/1841171640 >> 172.21.15.112:0/1841171640 conn(0x7f0200079350 msgr2=0x7f0200079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.656+0000 7f0206800700 1 -- 172.21.15.112:0/1841171640 shutdown_connections 2024-05-08T05:50:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.656+0000 7f0206800700 1 -- 172.21.15.112:0/1841171640 wait complete. 2024-05-08T05:50:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.657+0000 7f0206800700 1 Processor -- start 2024-05-08T05:50:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.657+0000 7f0206800700 1 -- start start 2024-05-08T05:50:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.657+0000 7f0206800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f020006b180 0x7f020019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.657+0000 7f0206800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020006bb40 0x7f020019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.657+0000 7f0206800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0200109510 0x7f0200199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.658+0000 7f0206800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0200113690 con 0x7f020006bb40 2024-05-08T05:50:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.658+0000 7f0206800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0200113510 con 0x7f0200109510 2024-05-08T05:50:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.658+0000 7f0206800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0200113810 con 0x7f020006b180 2024-05-08T05:50:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.658+0000 7f01fea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020006bb40 0x7f020019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.658+0000 7f01ff400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f020006b180 0x7f020019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.658+0000 7f01ff400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f020006b180 0x7f020019f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:54972/0 (socket says 172.21.15.112:54972) 2024-05-08T05:50:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.658+0000 7f01ff400700 1 -- 172.21.15.112:0/3704249949 learned_addr learned my addr 172.21.15.112:0/3704249949 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:50:58.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.658+0000 7f01ffe00700 1 --2- 172.21.15.112:0/3704249949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0200109510 0x7f0200199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:58.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.658+0000 7f01ff400700 1 -- 172.21.15.112:0/3704249949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0200109510 msgr2=0x7f0200199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:58.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.658+0000 7f01ff400700 1 --2- 172.21.15.112:0/3704249949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0200109510 0x7f0200199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:58.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.658+0000 7f01ff400700 1 -- 172.21.15.112:0/3704249949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020006bb40 msgr2=0x7f020019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:58.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.658+0000 7f01ff400700 1 --2- 172.21.15.112:0/3704249949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020006bb40 0x7f020019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:58.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.658+0000 7f01ff400700 1 -- 172.21.15.112:0/3704249949 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020019a0f0 con 0x7f020006b180 2024-05-08T05:50:58.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.658+0000 7f01fea00700 1 --2- 172.21.15.112:0/3704249949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020006bb40 0x7f020019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:58.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.659+0000 7f01ff400700 1 --2- 172.21.15.112:0/3704249949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f020006b180 0x7f020019f780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f01f000e9a0 tx=0x7f01f000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.659+0000 7f01f7e00700 1 -- 172.21.15.112:0/3704249949 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f000ca10 con 0x7f020006b180 2024-05-08T05:50:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.659+0000 7f0206800700 1 -- 172.21.15.112:0/3704249949 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020019a3e0 con 0x7f020006b180 2024-05-08T05:50:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.659+0000 7f0206800700 1 -- 172.21.15.112:0/3704249949 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02001a6650 con 0x7f020006b180 2024-05-08T05:50:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.659+0000 7f01f7e00700 1 -- 172.21.15.112:0/3704249949 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01f000cbb0 con 0x7f020006b180 2024-05-08T05:50:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.659+0000 7f01f7e00700 1 -- 172.21.15.112:0/3704249949 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0018420 con 0x7f020006b180 2024-05-08T05:50:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.660+0000 7f0206800700 1 -- 172.21.15.112:0/3704249949 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020006a190 con 0x7f020006b180 2024-05-08T05:50:58.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.662+0000 7f01f7e00700 1 -- 172.21.15.112:0/3704249949 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f01f0010430 con 0x7f020006b180 2024-05-08T05:50:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.664+0000 7f01f7e00700 1 --2- 172.21.15.112:0/3704249949 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f01e40b2900 0x7f01e40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.664+0000 7f01f7e00700 1 -- 172.21.15.112:0/3704249949 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f01f00e1f10 con 0x7f020006b180 2024-05-08T05:50:58.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.664+0000 7f01fea00700 1 --2- 172.21.15.112:0/3704249949 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f01e40b2900 0x7f01e40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:58.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.665+0000 7f01fea00700 1 --2- 172.21.15.112:0/3704249949 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f01e40b2900 0x7f01e40b4dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0200079950 tx=0x7f01ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:58.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.668+0000 7f01f7e00700 1 -- 172.21.15.112:0/3704249949 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f01f00aa030 con 0x7f020006b180 2024-05-08T05:50:58.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:58.901+0000 7f0206800700 1 -- 172.21.15.112:0/3704249949 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f020002e510 con 0x7f020006b180 2024-05-08T05:50:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:59.242+0000 7f01f7e00700 1 -- 172.21.15.112:0/3704249949 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7f01f000cd60 con 0x7f020006b180 2024-05-08T05:50:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:59.249+0000 7f0206800700 1 -- 172.21.15.112:0/3704249949 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f01e40b2900 msgr2=0x7f01e40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:59.249+0000 7f0206800700 1 --2- 172.21.15.112:0/3704249949 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f01e40b2900 0x7f01e40b4dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0200079950 tx=0x7f01ec009260 comp rx=0 tx=0).stop 2024-05-08T05:50:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:59.249+0000 7f0206800700 1 -- 172.21.15.112:0/3704249949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f020006b180 msgr2=0x7f020019f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:59.249+0000 7f0206800700 1 --2- 172.21.15.112:0/3704249949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f020006b180 0x7f020019f780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f01f000e9a0 tx=0x7f01f000ecb0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:59.249+0000 7f0206800700 1 -- 172.21.15.112:0/3704249949 shutdown_connections 2024-05-08T05:50:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:59.249+0000 7f0206800700 1 --2- 172.21.15.112:0/3704249949 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f01e40b2900 0x7f01e40b4dc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:59.249+0000 7f0206800700 1 --2- 172.21.15.112:0/3704249949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0200109510 0x7f0200199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:59.249+0000 7f0206800700 1 --2- 172.21.15.112:0/3704249949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f020006b180 0x7f020019f780 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:59.249+0000 7f0206800700 1 --2- 172.21.15.112:0/3704249949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f020006bb40 0x7f020019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:59.250+0000 7f0206800700 1 -- 172.21.15.112:0/3704249949 >> 172.21.15.112:0/3704249949 conn(0x7f0200079350 msgr2=0x7f02001010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:59.250+0000 7f0206800700 1 -- 172.21.15.112:0/3704249949 shutdown_connections 2024-05-08T05:50:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:50:59.250+0000 7f0206800700 1 -- 172.21.15.112:0/3704249949 wait complete. 2024-05-08T05:50:59.252 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-05-08T05:50:59.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:59 smithi112 ceph-mon[30003]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:59.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:59 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3704249949' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T05:50:59.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:50:59 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T05:50:59.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:59 smithi162 ceph-mon[34800]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:59.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:59 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3704249949' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T05:50:59.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:50:59 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T05:50:59.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:59 smithi150 ceph-mon[34315]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:59.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:59 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3704249949' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T05:50:59.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:50:59 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T05:51:00.131 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T05:51:00.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:00 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T05:51:00.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:00 smithi162 ceph-mon[34800]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T05:51:00.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:00 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T05:51:00.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:00 smithi112 ceph-mon[30003]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T05:51:00.583 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:00.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:00 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T05:51:00.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:00 smithi150 ceph-mon[34315]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T05:51:01.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:01 smithi112 ceph-mon[30003]: pgmap v302: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:51:01.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:01 smithi112 ceph-mon[30003]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:51:01.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:01 smithi112 ceph-mon[30003]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T05:51:01.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:01 smithi162 ceph-mon[34800]: pgmap v302: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:51:01.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:01 smithi162 ceph-mon[34800]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:51:01.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:01 smithi162 ceph-mon[34800]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T05:51:01.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:01 smithi150 ceph-mon[34315]: pgmap v302: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:51:01.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:01 smithi150 ceph-mon[34315]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:51:01.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:01 smithi150 ceph-mon[34315]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T05:51:02.417 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:02 smithi112 ceph-mon[30003]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T05:51:02.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:02 smithi162 ceph-mon[34800]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T05:51:02.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:02 smithi150 ceph-mon[34315]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T05:51:02.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.914+0000 7fa567200700 1 -- 172.21.15.112:0/1762914123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa56810a3a0 msgr2=0x7fa568116840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:02.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.914+0000 7fa567200700 1 --2- 172.21.15.112:0/1762914123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa56810a3a0 0x7fa568116840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa56000b0a0 tx=0x7fa56000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.914+0000 7fa567200700 1 -- 172.21.15.112:0/1762914123 shutdown_connections 2024-05-08T05:51:02.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.914+0000 7fa567200700 1 --2- 172.21.15.112:0/1762914123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa56810a3a0 0x7fa568116840 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.914+0000 7fa567200700 1 --2- 172.21.15.112:0/1762914123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa56810e8b0 0x7fa56810ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.914+0000 7fa567200700 1 --2- 172.21.15.112:0/1762914123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5681099e0 0x7fa568109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.914+0000 7fa567200700 1 -- 172.21.15.112:0/1762914123 >> 172.21.15.112:0/1762914123 conn(0x7fa568079230 msgr2=0x7fa5681082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:02.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.914+0000 7fa567200700 1 -- 172.21.15.112:0/1762914123 shutdown_connections 2024-05-08T05:51:02.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.914+0000 7fa567200700 1 -- 172.21.15.112:0/1762914123 wait complete. 2024-05-08T05:51:02.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.915+0000 7fa567200700 1 Processor -- start 2024-05-08T05:51:02.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.915+0000 7fa567200700 1 -- start start 2024-05-08T05:51:02.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.916+0000 7fa567200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5681099e0 0x7fa5681a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:02.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.916+0000 7fa567200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa56810a3a0 0x7fa5681a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:02.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.916+0000 7fa567200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56810e8b0 0x7fa5681a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:02.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.916+0000 7fa567200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa568119be0 con 0x7fa56810e8b0 2024-05-08T05:51:02.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.916+0000 7fa567200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa568119a60 con 0x7fa56810a3a0 2024-05-08T05:51:02.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.916+0000 7fa567200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa568119d60 con 0x7fa5681099e0 2024-05-08T05:51:02.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.916+0000 7fa566800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56810e8b0 0x7fa5681a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:02.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.916+0000 7fa566800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56810e8b0 0x7fa5681a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39836/0 (socket says 172.21.15.112:39836) 2024-05-08T05:51:02.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.916+0000 7fa566800700 1 -- 172.21.15.112:0/3435825431 learned_addr learned my addr 172.21.15.112:0/3435825431 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:51:02.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.916+0000 7fa565400700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa56810a3a0 0x7fa5681a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:02.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.916+0000 7fa565e00700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5681099e0 0x7fa5681a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:02.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.917+0000 7fa565400700 1 -- 172.21.15.112:0/3435825431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5681099e0 msgr2=0x7fa5681a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:02.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.917+0000 7fa565400700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5681099e0 0x7fa5681a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.917+0000 7fa565400700 1 -- 172.21.15.112:0/3435825431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56810e8b0 msgr2=0x7fa5681a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:02.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.917+0000 7fa565400700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56810e8b0 0x7fa5681a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.917+0000 7fa565400700 1 -- 172.21.15.112:0/3435825431 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5681a5050 con 0x7fa56810a3a0 2024-05-08T05:51:02.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.917+0000 7fa565e00700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5681099e0 0x7fa5681a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:02.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.917+0000 7fa566800700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56810e8b0 0x7fa5681a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:02.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.917+0000 7fa565400700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa56810a3a0 0x7fa5681a05d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa55400e9f0 tx=0x7fa55400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:02.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.917+0000 7fa55ea00700 1 -- 172.21.15.112:0/3435825431 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55400c9c0 con 0x7fa56810a3a0 2024-05-08T05:51:02.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.917+0000 7fa567200700 1 -- 172.21.15.112:0/3435825431 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5681a5340 con 0x7fa56810a3a0 2024-05-08T05:51:02.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.917+0000 7fa55ea00700 1 -- 172.21.15.112:0/3435825431 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa55400cb60 con 0x7fa56810a3a0 2024-05-08T05:51:02.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.918+0000 7fa55ea00700 1 -- 172.21.15.112:0/3435825431 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554018680 con 0x7fa56810a3a0 2024-05-08T05:51:02.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.918+0000 7fa567200700 1 -- 172.21.15.112:0/3435825431 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5681acbc0 con 0x7fa56810a3a0 2024-05-08T05:51:02.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.919+0000 7fa567200700 1 -- 172.21.15.112:0/3435825431 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56806a190 con 0x7fa56810a3a0 2024-05-08T05:51:02.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.922+0000 7fa55ea00700 1 -- 172.21.15.112:0/3435825431 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fa554010430 con 0x7fa56810a3a0 2024-05-08T05:51:02.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.923+0000 7fa55ea00700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa54c0b2900 0x7fa54c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:02.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.923+0000 7fa55ea00700 1 -- 172.21.15.112:0/3435825431 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7fa5540e23d0 con 0x7fa56810a3a0 2024-05-08T05:51:02.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.924+0000 7fa565e00700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa54c0b2900 0x7fa54c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:02.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.924+0000 7fa565e00700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa54c0b2900 0x7fa54c0b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa558009fd0 tx=0x7fa558009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:02.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:02.924+0000 7fa55ea00700 1 -- 172.21.15.112:0/3435825431 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa5540aa380 con 0x7fa56810a3a0 2024-05-08T05:51:03.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:03.159+0000 7fa567200700 1 -- 172.21.15.112:0/3435825431 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa56802e510 con 0x7fa56810a3a0 2024-05-08T05:51:03.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:03.281+0000 7fa55ea00700 1 -- 172.21.15.112:0/3435825431 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7fa5540b2400 con 0x7fa56810a3a0 2024-05-08T05:51:03.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:03.290+0000 7fa567200700 1 -- 172.21.15.112:0/3435825431 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa54c0b2900 msgr2=0x7fa54c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:03.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:03.290+0000 7fa567200700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa54c0b2900 0x7fa54c0b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa558009fd0 tx=0x7fa558009260 comp rx=0 tx=0).stop 2024-05-08T05:51:03.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:03.290+0000 7fa567200700 1 -- 172.21.15.112:0/3435825431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa56810a3a0 msgr2=0x7fa5681a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:03.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:03.290+0000 7fa567200700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa56810a3a0 0x7fa5681a05d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa55400e9f0 tx=0x7fa55400ed00 comp rx=0 tx=0).stop 2024-05-08T05:51:03.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:03.291+0000 7fa567200700 1 -- 172.21.15.112:0/3435825431 shutdown_connections 2024-05-08T05:51:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:03.291+0000 7fa567200700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa54c0b2900 0x7fa54c0b4dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:03.291+0000 7fa567200700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa56810e8b0 0x7fa5681a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:03.291+0000 7fa567200700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5681099e0 0x7fa5681a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:03.291+0000 7fa567200700 1 --2- 172.21.15.112:0/3435825431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa56810a3a0 0x7fa5681a05d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:03.291+0000 7fa567200700 1 -- 172.21.15.112:0/3435825431 >> 172.21.15.112:0/3435825431 conn(0x7fa568079230 msgr2=0x7fa568079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:03.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:03.292+0000 7fa567200700 1 -- 172.21.15.112:0/3435825431 shutdown_connections 2024-05-08T05:51:03.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:03.292+0000 7fa567200700 1 -- 172.21.15.112:0/3435825431 wait complete. 2024-05-08T05:51:03.298 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-05-08T05:51:03.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:03 smithi112 ceph-mon[30003]: pgmap v305: 33 pgs: 10 active+clean, 3 creating+activating, 8 creating+peering, 12 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:51:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:03 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3435825431' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T05:51:03.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:03 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T05:51:03.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:03 smithi162 ceph-mon[34800]: pgmap v305: 33 pgs: 10 active+clean, 3 creating+activating, 8 creating+peering, 12 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:51:03.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:03 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3435825431' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T05:51:03.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:03 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T05:51:03.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:03 smithi150 ceph-mon[34315]: pgmap v305: 33 pgs: 10 active+clean, 3 creating+activating, 8 creating+peering, 12 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:51:03.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:03 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3435825431' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T05:51:03.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:03 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T05:51:04.222 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T05:51:04.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:04 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T05:51:04.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:04 smithi112 ceph-mon[30003]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T05:51:04.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:04 smithi112 ceph-mon[30003]: pgmap v307: 97 pgs: 16 active+clean, 3 creating+activating, 8 creating+peering, 70 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:51:04.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:04 smithi112 ceph-mon[30003]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T05:51:04.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:04 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T05:51:04.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:04 smithi162 ceph-mon[34800]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T05:51:04.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:04 smithi162 ceph-mon[34800]: pgmap v307: 97 pgs: 16 active+clean, 3 creating+activating, 8 creating+peering, 70 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:51:04.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:04 smithi162 ceph-mon[34800]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T05:51:04.677 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:04 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T05:51:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:04 smithi150 ceph-mon[34315]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T05:51:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:04 smithi150 ceph-mon[34315]: pgmap v307: 97 pgs: 16 active+clean, 3 creating+activating, 8 creating+peering, 70 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:51:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:04 smithi150 ceph-mon[34315]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T05:51:06.526 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:06 smithi112 ceph-mon[30003]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T05:51:06.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:06 smithi112 ceph-mon[30003]: pgmap v310: 97 pgs: 50 active+clean, 3 creating+activating, 20 creating+peering, 24 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:51:06.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:06 smithi162 ceph-mon[34800]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T05:51:06.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:06 smithi162 ceph-mon[34800]: pgmap v310: 97 pgs: 50 active+clean, 3 creating+activating, 20 creating+peering, 24 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:51:06.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:06 smithi150 ceph-mon[34315]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T05:51:06.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:06 smithi150 ceph-mon[34315]: pgmap v310: 97 pgs: 50 active+clean, 3 creating+activating, 20 creating+peering, 24 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:51:07.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.019+0000 7f49f3e00700 1 -- 172.21.15.112:0/3175930853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f410b680 msgr2=0x7f49f410ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:07.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.019+0000 7f49f3e00700 1 --2- 172.21.15.112:0/3175930853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f410b680 0x7f49f410ba60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f49e40099c0 tx=0x7f49e4009cd0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.020+0000 7f49f3e00700 1 -- 172.21.15.112:0/3175930853 shutdown_connections 2024-05-08T05:51:07.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.020+0000 7f49f3e00700 1 --2- 172.21.15.112:0/3175930853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f4105f90 0x7f49f4112450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.020+0000 7f49f3e00700 1 --2- 172.21.15.112:0/3175930853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f410b680 0x7f49f410ba60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.020+0000 7f49f3e00700 1 --2- 172.21.15.112:0/3175930853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f41055d0 0x7f49f4105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.020+0000 7f49f3e00700 1 -- 172.21.15.112:0/3175930853 >> 172.21.15.112:0/3175930853 conn(0x7f49f4101080 msgr2=0x7f49f41034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:07.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.020+0000 7f49f3e00700 1 -- 172.21.15.112:0/3175930853 shutdown_connections 2024-05-08T05:51:07.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.020+0000 7f49f3e00700 1 -- 172.21.15.112:0/3175930853 wait complete. 2024-05-08T05:51:07.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.021+0000 7f49f3e00700 1 Processor -- start 2024-05-08T05:51:07.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.021+0000 7f49f3e00700 1 -- start start 2024-05-08T05:51:07.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.021+0000 7f49f3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f41055d0 0x7f49f41a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:07.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.021+0000 7f49f3e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f4105f90 0x7f49f41a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:07.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.022+0000 7f49f3e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f410b680 0x7f49f419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:07.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.022+0000 7f49f3e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41157d0 con 0x7f49f410b680 2024-05-08T05:51:07.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.022+0000 7f49f3e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f49f4115650 con 0x7f49f41055d0 2024-05-08T05:51:07.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.022+0000 7f49f3e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f49f4115950 con 0x7f49f4105f90 2024-05-08T05:51:07.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.022+0000 7f49f3400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f410b680 0x7f49f419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:07.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.022+0000 7f49f3400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f410b680 0x7f49f419baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44380/0 (socket says 172.21.15.112:44380) 2024-05-08T05:51:07.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.022+0000 7f49f3400700 1 -- 172.21.15.112:0/4163973090 learned_addr learned my addr 172.21.15.112:0/4163973090 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:51:07.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.022+0000 7f49f2a00700 1 --2- 172.21.15.112:0/4163973090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f41055d0 0x7f49f41a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.022+0000 7f49f2000700 1 --2- 172.21.15.112:0/4163973090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f4105f90 0x7f49f41a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.022+0000 7f49f3400700 1 -- 172.21.15.112:0/4163973090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f4105f90 msgr2=0x7f49f41a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.022+0000 7f49f3400700 1 --2- 172.21.15.112:0/4163973090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f4105f90 0x7f49f41a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.022+0000 7f49f3400700 1 -- 172.21.15.112:0/4163973090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f41055d0 msgr2=0x7f49f41a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.022+0000 7f49f3400700 1 --2- 172.21.15.112:0/4163973090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f41055d0 0x7f49f41a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.022+0000 7f49f3400700 1 -- 172.21.15.112:0/4163973090 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f419c250 con 0x7f49f410b680 2024-05-08T05:51:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.022+0000 7f49f2a00700 1 --2- 172.21.15.112:0/4163973090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f41055d0 0x7f49f41a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.023+0000 7f49f3400700 1 --2- 172.21.15.112:0/4163973090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f410b680 0x7f49f419baf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00ef30 tx=0x7f49ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.023+0000 7f49eb400700 1 -- 172.21.15.112:0/4163973090 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0092e0 con 0x7f49f410b680 2024-05-08T05:51:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.023+0000 7f49eb400700 1 -- 172.21.15.112:0/4163973090 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49ec004bb0 con 0x7f49f410b680 2024-05-08T05:51:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.023+0000 7f49eb400700 1 -- 172.21.15.112:0/4163973090 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec004020 con 0x7f49f410b680 2024-05-08T05:51:07.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.023+0000 7f49f3e00700 1 -- 172.21.15.112:0/4163973090 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f419c4e0 con 0x7f49f410b680 2024-05-08T05:51:07.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.023+0000 7f49f3e00700 1 -- 172.21.15.112:0/4163973090 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f41a87b0 con 0x7f49f410b680 2024-05-08T05:51:07.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.024+0000 7f49f3e00700 1 -- 172.21.15.112:0/4163973090 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f410db10 con 0x7f49f410b680 2024-05-08T05:51:07.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.027+0000 7f49eb400700 1 -- 172.21.15.112:0/4163973090 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f49ec01b030 con 0x7f49f410b680 2024-05-08T05:51:07.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.028+0000 7f49eb400700 1 --2- 172.21.15.112:0/4163973090 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f49d80b2900 0x7f49d80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:07.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.028+0000 7f49eb400700 1 -- 172.21.15.112:0/4163973090 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7f49ec0e24a0 con 0x7f49f410b680 2024-05-08T05:51:07.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.028+0000 7f49f2a00700 1 --2- 172.21.15.112:0/4163973090 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f49d80b2900 0x7f49d80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:07.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.029+0000 7f49f2a00700 1 --2- 172.21.15.112:0/4163973090 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f49d80b2900 0x7f49d80b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f49e4000c00 tx=0x7f49e4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:07.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.031+0000 7f49eb400700 1 -- 172.21.15.112:0/4163973090 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49ec022020 con 0x7f49f410b680 2024-05-08T05:51:07.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.310+0000 7f49f3e00700 1 -- 172.21.15.112:0/4163973090 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f49f406a190 con 0x7f49f410b680 2024-05-08T05:51:07.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.321+0000 7f49eb400700 1 -- 172.21.15.112:0/4163973090 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f49ec010040 con 0x7f49f410b680 2024-05-08T05:51:07.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.324+0000 7f49f3e00700 1 -- 172.21.15.112:0/4163973090 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f49d80b2900 msgr2=0x7f49d80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:07.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.324+0000 7f49f3e00700 1 --2- 172.21.15.112:0/4163973090 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f49d80b2900 0x7f49d80b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f49e4000c00 tx=0x7f49e4005fb0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.324+0000 7f49f3e00700 1 -- 172.21.15.112:0/4163973090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f410b680 msgr2=0x7f49f419baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.324+0000 7f49f3e00700 1 --2- 172.21.15.112:0/4163973090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f410b680 0x7f49f419baf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00ef30 tx=0x7f49ec00c3a0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.325+0000 7f49f3e00700 1 -- 172.21.15.112:0/4163973090 shutdown_connections 2024-05-08T05:51:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.325+0000 7f49f3e00700 1 --2- 172.21.15.112:0/4163973090 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f49d80b2900 0x7f49d80b4dc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.325+0000 7f49f3e00700 1 --2- 172.21.15.112:0/4163973090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f410b680 0x7f49f419baf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.325+0000 7f49f3e00700 1 --2- 172.21.15.112:0/4163973090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f41055d0 0x7f49f41a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.325+0000 7f49f3e00700 1 --2- 172.21.15.112:0/4163973090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f4105f90 0x7f49f41a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.325+0000 7f49f3e00700 1 -- 172.21.15.112:0/4163973090 >> 172.21.15.112:0/4163973090 conn(0x7f49f4101080 msgr2=0x7f49f4102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.325+0000 7f49f3e00700 1 -- 172.21.15.112:0/4163973090 shutdown_connections 2024-05-08T05:51:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:07.326+0000 7f49f3e00700 1 -- 172.21.15.112:0/4163973090 wait complete. 2024-05-08T05:51:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T05:51:07.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:07 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/4163973090' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T05:51:07.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:07 smithi112 ceph-mon[30003]: daemon mds.a assigned to filesystem cephfs as rank 0 2024-05-08T05:51:07.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:07 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/4163973090' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T05:51:07.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:07 smithi162 ceph-mon[34800]: daemon mds.a assigned to filesystem cephfs as rank 0 2024-05-08T05:51:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:07 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/4163973090' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T05:51:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:07 smithi150 ceph-mon[34315]: daemon mds.a assigned to filesystem cephfs as rank 0 2024-05-08T05:51:08.288 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:51:08.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:08 smithi112 ceph-mon[30003]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T05:51:08.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:08 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/4163973090' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T05:51:08.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:08 smithi112 ceph-mon[30003]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2024-05-08T05:51:08.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:08 smithi112 ceph-mon[30003]: daemon mds.a is now active in filesystem cephfs as rank 0 2024-05-08T05:51:08.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:08 smithi112 ceph-mon[30003]: pgmap v312: 97 pgs: 75 active+clean, 20 creating+peering, 2 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:51:08.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:08 smithi112 ceph-mon[30003]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:51:08.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:08 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] up:active 2024-05-08T05:51:08.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:08 smithi112 ceph-mon[30003]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-08T05:51:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:08 smithi150 ceph-mon[34315]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T05:51:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:08 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/4163973090' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T05:51:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:08 smithi150 ceph-mon[34315]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2024-05-08T05:51:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:08 smithi150 ceph-mon[34315]: daemon mds.a is now active in filesystem cephfs as rank 0 2024-05-08T05:51:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:08 smithi150 ceph-mon[34315]: pgmap v312: 97 pgs: 75 active+clean, 20 creating+peering, 2 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:51:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:08 smithi150 ceph-mon[34315]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:51:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:08 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] up:active 2024-05-08T05:51:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:08 smithi150 ceph-mon[34315]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-08T05:51:08.740 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:08.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:08 smithi162 ceph-mon[34800]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T05:51:08.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:08 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/4163973090' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T05:51:08.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:08 smithi162 ceph-mon[34800]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2024-05-08T05:51:08.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:08 smithi162 ceph-mon[34800]: daemon mds.a is now active in filesystem cephfs as rank 0 2024-05-08T05:51:08.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:08 smithi162 ceph-mon[34800]: pgmap v312: 97 pgs: 75 active+clean, 20 creating+peering, 2 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:51:08.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:08 smithi162 ceph-mon[34800]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:51:08.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:08 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] up:active 2024-05-08T05:51:08.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:08 smithi162 ceph-mon[34800]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-08T05:51:09.219 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 05:51:09 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-a[69856]: 2024-05-08T05:51:09.018+0000 7fc4f0800700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:51:09.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:09 smithi112 ceph-mon[30003]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-08T05:51:09.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:09 smithi112 ceph-mon[30003]: Cluster is now healthy 2024-05-08T05:51:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:09 smithi150 ceph-mon[34315]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-08T05:51:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:09 smithi150 ceph-mon[34315]: Cluster is now healthy 2024-05-08T05:51:09.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:09 smithi162 ceph-mon[34800]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-08T05:51:09.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:09 smithi162 ceph-mon[34800]: Cluster is now healthy 2024-05-08T05:51:10.456 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:10 smithi112 ceph-mon[30003]: pgmap v313: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-05-08T05:51:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:10 smithi150 ceph-mon[34315]: pgmap v313: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-05-08T05:51:10.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:10 smithi162 ceph-mon[34800]: pgmap v313: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-05-08T05:51:10.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.979+0000 7f6a2d600700 1 -- 172.21.15.112:0/2223344782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a28101880 msgr2=0x7f6a28101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:10.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.979+0000 7f6a2d600700 1 --2- 172.21.15.112:0/2223344782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a28101880 0x7f6a28101c60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009990 tx=0x7f6a18009ca0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.979+0000 7f6a2d600700 1 -- 172.21.15.112:0/2223344782 shutdown_connections 2024-05-08T05:51:10.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.979+0000 7f6a2d600700 1 --2- 172.21.15.112:0/2223344782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a2810fc90 0x7f6a28112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.979+0000 7f6a2d600700 1 --2- 172.21.15.112:0/2223344782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a28101880 0x7f6a28101c60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.979+0000 7f6a2d600700 1 --2- 172.21.15.112:0/2223344782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a281021a0 0x7f6a2810f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.979+0000 7f6a2d600700 1 -- 172.21.15.112:0/2223344782 >> 172.21.15.112:0/2223344782 conn(0x7f6a280785a0 msgr2=0x7f6a280ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:10.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.980+0000 7f6a2d600700 1 -- 172.21.15.112:0/2223344782 shutdown_connections 2024-05-08T05:51:10.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.980+0000 7f6a2d600700 1 -- 172.21.15.112:0/2223344782 wait complete. 2024-05-08T05:51:10.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.981+0000 7f6a2d600700 1 Processor -- start 2024-05-08T05:51:10.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.981+0000 7f6a2d600700 1 -- start start 2024-05-08T05:51:10.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.981+0000 7f6a2d600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a28101880 0x7f6a2819bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:10.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.981+0000 7f6a2d600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a281021a0 0x7f6a2819c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:10.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.981+0000 7f6a2d600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a2810fc90 0x7f6a281a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:10.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.981+0000 7f6a2d600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a281158e0 con 0x7f6a2810fc90 2024-05-08T05:51:10.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.981+0000 7f6a2d600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6a28115760 con 0x7f6a28101880 2024-05-08T05:51:10.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.981+0000 7f6a2d600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6a28115a60 con 0x7f6a281021a0 2024-05-08T05:51:10.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.981+0000 7f6a1fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a281021a0 0x7f6a2819c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:10.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.981+0000 7f6a1fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a281021a0 0x7f6a2819c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38460/0 (socket says 172.21.15.112:38460) 2024-05-08T05:51:10.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.981+0000 7f6a1fe00700 1 -- 172.21.15.112:0/186702125 learned_addr learned my addr 172.21.15.112:0/186702125 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:51:10.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.981+0000 7f6a27400700 1 --2- 172.21.15.112:0/186702125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a2810fc90 0x7f6a281a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:10.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.982+0000 7f6a26a00700 1 --2- 172.21.15.112:0/186702125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a28101880 0x7f6a2819bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:10.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.982+0000 7f6a1fe00700 1 -- 172.21.15.112:0/186702125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a28101880 msgr2=0x7f6a2819bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:10.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.982+0000 7f6a1fe00700 1 --2- 172.21.15.112:0/186702125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a28101880 0x7f6a2819bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.982+0000 7f6a1fe00700 1 -- 172.21.15.112:0/186702125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a2810fc90 msgr2=0x7f6a281a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:10.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.982+0000 7f6a1fe00700 1 --2- 172.21.15.112:0/186702125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a2810fc90 0x7f6a281a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.982+0000 7f6a1fe00700 1 -- 172.21.15.112:0/186702125 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a281a0da0 con 0x7f6a281021a0 2024-05-08T05:51:10.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.982+0000 7f6a1fe00700 1 --2- 172.21.15.112:0/186702125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a281021a0 0x7f6a2819c2d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000e9f0 tx=0x7f6a1000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:10.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.982+0000 7f6a1f400700 1 -- 172.21.15.112:0/186702125 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10004230 con 0x7f6a281021a0 2024-05-08T05:51:10.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.983+0000 7f6a1f400700 1 -- 172.21.15.112:0/186702125 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a100043d0 con 0x7f6a281021a0 2024-05-08T05:51:10.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.983+0000 7f6a1f400700 1 -- 172.21.15.112:0/186702125 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10003cd0 con 0x7f6a281021a0 2024-05-08T05:51:10.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.983+0000 7f6a2d600700 1 -- 172.21.15.112:0/186702125 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a281a1090 con 0x7f6a281021a0 2024-05-08T05:51:10.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.983+0000 7f6a2d600700 1 -- 172.21.15.112:0/186702125 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a281a8910 con 0x7f6a281021a0 2024-05-08T05:51:10.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.984+0000 7f6a2d600700 1 -- 172.21.15.112:0/186702125 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a281029b0 con 0x7f6a281021a0 2024-05-08T05:51:10.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.986+0000 7f6a1f400700 1 -- 172.21.15.112:0/186702125 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f6a1000c7c0 con 0x7f6a281021a0 2024-05-08T05:51:10.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.987+0000 7f6a1f400700 1 --2- 172.21.15.112:0/186702125 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6a0c0b2900 0x7f6a0c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:10.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.988+0000 7f6a1f400700 1 -- 172.21.15.112:0/186702125 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f6a100e23a0 con 0x7f6a281021a0 2024-05-08T05:51:10.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.988+0000 7f6a26a00700 1 --2- 172.21.15.112:0/186702125 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6a0c0b2900 0x7f6a0c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:10.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.988+0000 7f6a26a00700 1 --2- 172.21.15.112:0/186702125 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6a0c0b2900 0x7f6a0c0b4dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6a18006010 tx=0x7f6a1800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:10.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:10.989+0000 7f6a1f400700 1 -- 172.21.15.112:0/186702125 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6a100aa170 con 0x7f6a281021a0 2024-05-08T05:51:11.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:11.241+0000 7f6a2d600700 1 -- 172.21.15.112:0/186702125 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a2802e510 con 0x7f6a281021a0 2024-05-08T05:51:11.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:11.243+0000 7f6a1f400700 1 -- 172.21.15.112:0/186702125 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22287 (secure 0 0 0) 0x7f6a100a9a70 con 0x7f6a281021a0 2024-05-08T05:51:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:51:11.245 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":80,"fsid":"7a07ff60-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:48.498871+0000","modified":"2024-05-08T05:51:07.312576+0000","last_up_change":"2024-05-08T05:49:54.039380+0000","last_in_change":"2024-05-08T05:49:34.443850+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:47.485105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:50:58.918909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.175022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b378542-1bbe-4614-8805-8a6b875ada55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6803","nonce":3419091820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6805","nonce":3419091820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6809","nonce":3419091820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6807","nonce":3419091820}]},"public_addr":"172.21.15.112:6803/3419091820","cluster_addr":"172.21.15.112:6805/3419091820","heartbeat_back_addr":"172.21.15.112:6809/3419091820","heartbeat_front_addr":"172.21.15.112:6807/3419091820","state":["exists","up"]},{"osd":1,"uuid":"826ef036-2db7-470e-9956-d9a58b9181e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6801","nonce":2728419797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6803","nonce":2728419797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6807","nonce":2728419797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6805","nonce":2728419797}]},"public_addr":"172.21.15.150:6801/2728419797","cluster_addr":"172.21.15.150:6803/2728419797","heartbeat_back_addr":"172.21.15.150:6807/2728419797","heartbeat_front_addr":"172.21.15.150:6805/2728419797","state":["exists","up"]},{"osd":2,"uuid":"32e710e7-2a87-4863-bc7a-906d31aa4ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6801","nonce":1556203215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6803","nonce":1556203215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6807","nonce":1556203215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6805","nonce":1556203215}]},"public_addr":"172.21.15.162:6801/1556203215","cluster_addr":"172.21.15.162:6803/1556203215","heartbeat_back_addr":"172.21.15.162:6807/1556203215","heartbeat_front_addr":"172.21.15.162:6805/1556203215","state":["exists","up"]},{"osd":3,"uuid":"ba1d4a5f-6a95-4b9e-8a48-8524a899b004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6811","nonce":3325718755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6813","nonce":3325718755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6817","nonce":3325718755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6815","nonce":3325718755}]},"public_addr":"172.21.15.112:6811/3325718755","cluster_addr":"172.21.15.112:6813/3325718755","heartbeat_back_addr":"172.21.15.112:6817/3325718755","heartbeat_front_addr":"172.21.15.112:6815/3325718755","state":["exists","up"]},{"osd":4,"uuid":"cb4f1a42-8de8-4b50-b3bc-602de9f4803a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6809","nonce":1548914683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6811","nonce":1548914683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6815","nonce":1548914683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6813","nonce":1548914683}]},"public_addr":"172.21.15.150:6809/1548914683","cluster_addr":"172.21.15.150:6811/1548914683","heartbeat_back_addr":"172.21.15.150:6815/1548914683","heartbeat_front_addr":"172.21.15.150:6813/1548914683","state":["exists","up"]},{"osd":5,"uuid":"a26c03b8-050c-4a2c-8f23-5801fffb7c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6809","nonce":409836750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6811","nonce":409836750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6815","nonce":409836750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6813","nonce":409836750}]},"public_addr":"172.21.15.162:6809/409836750","cluster_addr":"172.21.15.162:6811/409836750","heartbeat_back_addr":"172.21.15.162:6815/409836750","heartbeat_front_addr":"172.21.15.162:6813/409836750","state":["exists","up"]},{"osd":6,"uuid":"8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6819","nonce":1098984186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6821","nonce":1098984186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6825","nonce":1098984186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6823","nonce":1098984186}]},"public_addr":"172.21.15.112:6819/1098984186","cluster_addr":"172.21.15.112:6821/1098984186","heartbeat_back_addr":"172.21.15.112:6825/1098984186","heartbeat_front_addr":"172.21.15.112:6823/1098984186","state":["exists","up"]},{"osd":7,"uuid":"af3629b4-9bed-4669-97a6-48deba503a96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6817","nonce":4009836564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6819","nonce":4009836564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6823","nonce":4009836564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6821","nonce":4009836564}]},"public_addr":"172.21.15.150:6817/4009836564","cluster_addr":"172.21.15.150:6819/4009836564","heartbeat_back_addr":"172.21.15.150:6823/4009836564","heartbeat_front_addr":"172.21.15.150:6821/4009836564","state":["exists","up"]},{"osd":8,"uuid":"9333e87d-dd78-4467-bbde-c2b44f4ea07f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6817","nonce":852791606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6819","nonce":852791606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6823","nonce":852791606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6821","nonce":852791606}]},"public_addr":"172.21.15.162:6817/852791606","cluster_addr":"172.21.15.162:6819/852791606","heartbeat_back_addr":"172.21.15.162:6823/852791606","heartbeat_front_addr":"172.21.15.162:6821/852791606","state":["exists","up"]},{"osd":9,"uuid":"09723581-1995-42e0-8db7-776ea98a918c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6827","nonce":2186764674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6829","nonce":2186764674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6833","nonce":2186764674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6831","nonce":2186764674}]},"public_addr":"172.21.15.112:6827/2186764674","cluster_addr":"172.21.15.112:6829/2186764674","heartbeat_back_addr":"172.21.15.112:6833/2186764674","heartbeat_front_addr":"172.21.15.112:6831/2186764674","state":["exists","up"]},{"osd":10,"uuid":"34e08e3e-1231-4899-a5b3-76d449105bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6825","nonce":2838995474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6827","nonce":2838995474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6831","nonce":2838995474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6829","nonce":2838995474}]},"public_addr":"172.21.15.150:6825/2838995474","cluster_addr":"172.21.15.150:6827/2838995474","heartbeat_back_addr":"172.21.15.150:6831/2838995474","heartbeat_front_addr":"172.21.15.150:6829/2838995474","state":["exists","up"]},{"osd":11,"uuid":"6fc34ec0-78ca-403c-acb7-b29d83ff64e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6825","nonce":3702614331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6827","nonce":3702614331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6831","nonce":3702614331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6829","nonce":3702614331}]},"public_addr":"172.21.15.162:6825/3702614331","cluster_addr":"172.21.15.162:6827/3702614331","heartbeat_back_addr":"172.21.15.162:6831/3702614331","heartbeat_front_addr":"172.21.15.162:6829/3702614331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:54.518702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:19.939543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:44.425586+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:16.126471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:41.696071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:05.517405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:38.549318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:03.799649+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:28.007282+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:01.395459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:27.963637+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:52.181212+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3667148497":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6800/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3425327158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1404611205":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3765334043":"2024-05-09T05:42:29.819307+0000","172.21.15.112:6800/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/2814303790":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1091751583":"2024-05-09T05:42:29.819307+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:51:11.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:11.246+0000 7f6a2d600700 1 -- 172.21.15.112:0/186702125 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6a0c0b2900 msgr2=0x7f6a0c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:11.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:11.247+0000 7f6a2d600700 1 --2- 172.21.15.112:0/186702125 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6a0c0b2900 0x7f6a0c0b4dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6a18006010 tx=0x7f6a1800b260 comp rx=0 tx=0).stop 2024-05-08T05:51:11.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:11.247+0000 7f6a2d600700 1 -- 172.21.15.112:0/186702125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a281021a0 msgr2=0x7f6a2819c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:11.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:11.247+0000 7f6a2d600700 1 --2- 172.21.15.112:0/186702125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a281021a0 0x7f6a2819c2d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000e9f0 tx=0x7f6a1000ed00 comp rx=0 tx=0).stop 2024-05-08T05:51:11.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:11.247+0000 7f6a2d600700 1 -- 172.21.15.112:0/186702125 shutdown_connections 2024-05-08T05:51:11.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:11.247+0000 7f6a2d600700 1 --2- 172.21.15.112:0/186702125 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6a0c0b2900 0x7f6a0c0b4dc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:11.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:11.247+0000 7f6a2d600700 1 --2- 172.21.15.112:0/186702125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a2810fc90 0x7f6a281a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:11.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:11.247+0000 7f6a2d600700 1 --2- 172.21.15.112:0/186702125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a28101880 0x7f6a2819bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:11.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:11.247+0000 7f6a2d600700 1 --2- 172.21.15.112:0/186702125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a281021a0 0x7f6a2819c2d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:11.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:11.247+0000 7f6a2d600700 1 -- 172.21.15.112:0/186702125 >> 172.21.15.112:0/186702125 conn(0x7f6a280785a0 msgr2=0x7f6a280ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:11.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:11.247+0000 7f6a2d600700 1 -- 172.21.15.112:0/186702125 shutdown_connections 2024-05-08T05:51:11.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:11.247+0000 7f6a2d600700 1 -- 172.21.15.112:0/186702125 wait complete. 2024-05-08T05:51:11.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:11 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/186702125' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:11 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/186702125' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:11.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:11 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/186702125' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:12.036 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:51:12.471 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:12.620 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:12 smithi112 ceph-mon[30003]: pgmap v314: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 997 B/s wr, 5 op/s 2024-05-08T05:51:12.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:12 smithi150 ceph-mon[34315]: pgmap v314: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 997 B/s wr, 5 op/s 2024-05-08T05:51:12.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:12 smithi162 ceph-mon[34800]: pgmap v314: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 997 B/s wr, 5 op/s 2024-05-08T05:51:14.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.790+0000 7fb052a00700 1 -- 172.21.15.112:0/593538682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb04c10a490 msgr2=0x7fb04c1169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.790+0000 7fb052a00700 1 --2- 172.21.15.112:0/593538682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb04c10a490 0x7fb04c1169a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb04400b0a0 tx=0x7fb04400b3b0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.792 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:14 smithi112 ceph-mon[30003]: pgmap v315: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-08T05:51:14.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.790+0000 7fb052a00700 1 -- 172.21.15.112:0/593538682 shutdown_connections 2024-05-08T05:51:14.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.790+0000 7fb052a00700 1 --2- 172.21.15.112:0/593538682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb04c10a490 0x7fb04c1169a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.790+0000 7fb052a00700 1 --2- 172.21.15.112:0/593538682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c10fb60 0x7fb04c10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.790+0000 7fb052a00700 1 --2- 172.21.15.112:0/593538682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb04c109ad0 0x7fb04c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.790+0000 7fb052a00700 1 -- 172.21.15.112:0/593538682 >> 172.21.15.112:0/593538682 conn(0x7fb04c076c70 msgr2=0x7fb04c1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:14.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.790+0000 7fb052a00700 1 -- 172.21.15.112:0/593538682 shutdown_connections 2024-05-08T05:51:14.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.790+0000 7fb052a00700 1 -- 172.21.15.112:0/593538682 wait complete. 2024-05-08T05:51:14.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.792+0000 7fb052a00700 1 Processor -- start 2024-05-08T05:51:14.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.792+0000 7fb052a00700 1 -- start start 2024-05-08T05:51:14.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.793+0000 7fb052a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb04c109ad0 0x7fb04c1a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.793+0000 7fb052a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c10a490 0x7fb04c1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.793+0000 7fb052a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb04c10fb60 0x7fb04c1a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.793+0000 7fb052a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c119d20 con 0x7fb04c10a490 2024-05-08T05:51:14.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.793+0000 7fb052a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c119ba0 con 0x7fb04c10fb60 2024-05-08T05:51:14.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.793+0000 7fb052a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c119ea0 con 0x7fb04c109ad0 2024-05-08T05:51:14.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.793+0000 7fb050a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb04c10fb60 0x7fb04c1a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.793+0000 7fb04be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb04c109ad0 0x7fb04c1a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.793+0000 7fb04b400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c10a490 0x7fb04c1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.793+0000 7fb04be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb04c109ad0 0x7fb04c1a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:54732/0 (socket says 172.21.15.112:54732) 2024-05-08T05:51:14.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.793+0000 7fb050a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb04c10fb60 0x7fb04c1a4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:56690/0 (socket says 172.21.15.112:56690) 2024-05-08T05:51:14.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.793+0000 7fb04be00700 1 -- 172.21.15.112:0/483089733 learned_addr learned my addr 172.21.15.112:0/483089733 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:51:14.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.794+0000 7fb04be00700 1 -- 172.21.15.112:0/483089733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb04c10fb60 msgr2=0x7fb04c1a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.794+0000 7fb04be00700 1 --2- 172.21.15.112:0/483089733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb04c10fb60 0x7fb04c1a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.794+0000 7fb04be00700 1 -- 172.21.15.112:0/483089733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c10a490 msgr2=0x7fb04c1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.794+0000 7fb04be00700 1 --2- 172.21.15.112:0/483089733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c10a490 0x7fb04c1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.794+0000 7fb04be00700 1 -- 172.21.15.112:0/483089733 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04c1a51c0 con 0x7fb04c109ad0 2024-05-08T05:51:14.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.794+0000 7fb050a00700 1 --2- 172.21.15.112:0/483089733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb04c10fb60 0x7fb04c1a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:14.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.794+0000 7fb04b400700 1 --2- 172.21.15.112:0/483089733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c10a490 0x7fb04c1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:14.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.794+0000 7fb04be00700 1 --2- 172.21.15.112:0/483089733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb04c109ad0 0x7fb04c1a01b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00e9a0 tx=0x7fb03c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:14.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.794+0000 7fb048c00700 1 -- 172.21.15.112:0/483089733 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00ca10 con 0x7fb04c109ad0 2024-05-08T05:51:14.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.794+0000 7fb052a00700 1 -- 172.21.15.112:0/483089733 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb04c1a54b0 con 0x7fb04c109ad0 2024-05-08T05:51:14.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.794+0000 7fb048c00700 1 -- 172.21.15.112:0/483089733 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03c00cbb0 con 0x7fb04c109ad0 2024-05-08T05:51:14.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.795+0000 7fb048c00700 1 -- 172.21.15.112:0/483089733 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c018680 con 0x7fb04c109ad0 2024-05-08T05:51:14.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.795+0000 7fb052a00700 1 -- 172.21.15.112:0/483089733 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb04c1acd30 con 0x7fb04c109ad0 2024-05-08T05:51:14.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.796+0000 7fb052a00700 1 -- 172.21.15.112:0/483089733 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb04c06a190 con 0x7fb04c109ad0 2024-05-08T05:51:14.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.798+0000 7fb048c00700 1 -- 172.21.15.112:0/483089733 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fb03c04aab0 con 0x7fb04c109ad0 2024-05-08T05:51:14.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.800+0000 7fb048c00700 1 --2- 172.21.15.112:0/483089733 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb0300b2880 0x7fb0300b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.800+0000 7fb048c00700 1 -- 172.21.15.112:0/483089733 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb03c0e2350 con 0x7fb04c109ad0 2024-05-08T05:51:14.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.800+0000 7fb04b400700 1 --2- 172.21.15.112:0/483089733 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb0300b2880 0x7fb0300b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.801+0000 7fb04b400700 1 --2- 172.21.15.112:0/483089733 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb0300b2880 0x7fb0300b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb04c078bc0 tx=0x7fb038009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:14.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:14.802+0000 7fb048c00700 1 -- 172.21.15.112:0/483089733 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb03c0aa120 con 0x7fb04c109ad0 2024-05-08T05:51:14.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:14 smithi162 ceph-mon[34800]: pgmap v315: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-08T05:51:14.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:14 smithi150 ceph-mon[34315]: pgmap v315: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-08T05:51:15.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:15.033+0000 7fb052a00700 1 -- 172.21.15.112:0/483089733 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb04c02e510 con 0x7fb04c109ad0 2024-05-08T05:51:15.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:15.035+0000 7fb048c00700 1 -- 172.21.15.112:0/483089733 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22287 (secure 0 0 0) 0x7fb03c0a9a20 con 0x7fb04c109ad0 2024-05-08T05:51:15.036 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:51:15.036 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":80,"fsid":"7a07ff60-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:48.498871+0000","modified":"2024-05-08T05:51:07.312576+0000","last_up_change":"2024-05-08T05:49:54.039380+0000","last_in_change":"2024-05-08T05:49:34.443850+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:47.485105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:50:58.918909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.175022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b378542-1bbe-4614-8805-8a6b875ada55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6803","nonce":3419091820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6805","nonce":3419091820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6809","nonce":3419091820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6807","nonce":3419091820}]},"public_addr":"172.21.15.112:6803/3419091820","cluster_addr":"172.21.15.112:6805/3419091820","heartbeat_back_addr":"172.21.15.112:6809/3419091820","heartbeat_front_addr":"172.21.15.112:6807/3419091820","state":["exists","up"]},{"osd":1,"uuid":"826ef036-2db7-470e-9956-d9a58b9181e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6801","nonce":2728419797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6803","nonce":2728419797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6807","nonce":2728419797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6805","nonce":2728419797}]},"public_addr":"172.21.15.150:6801/2728419797","cluster_addr":"172.21.15.150:6803/2728419797","heartbeat_back_addr":"172.21.15.150:6807/2728419797","heartbeat_front_addr":"172.21.15.150:6805/2728419797","state":["exists","up"]},{"osd":2,"uuid":"32e710e7-2a87-4863-bc7a-906d31aa4ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6801","nonce":1556203215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6803","nonce":1556203215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6807","nonce":1556203215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6805","nonce":1556203215}]},"public_addr":"172.21.15.162:6801/1556203215","cluster_addr":"172.21.15.162:6803/1556203215","heartbeat_back_addr":"172.21.15.162:6807/1556203215","heartbeat_front_addr":"172.21.15.162:6805/1556203215","state":["exists","up"]},{"osd":3,"uuid":"ba1d4a5f-6a95-4b9e-8a48-8524a899b004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6811","nonce":3325718755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6813","nonce":3325718755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6817","nonce":3325718755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6815","nonce":3325718755}]},"public_addr":"172.21.15.112:6811/3325718755","cluster_addr":"172.21.15.112:6813/3325718755","heartbeat_back_addr":"172.21.15.112:6817/3325718755","heartbeat_front_addr":"172.21.15.112:6815/3325718755","state":["exists","up"]},{"osd":4,"uuid":"cb4f1a42-8de8-4b50-b3bc-602de9f4803a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6809","nonce":1548914683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6811","nonce":1548914683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6815","nonce":1548914683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6813","nonce":1548914683}]},"public_addr":"172.21.15.150:6809/1548914683","cluster_addr":"172.21.15.150:6811/1548914683","heartbeat_back_addr":"172.21.15.150:6815/1548914683","heartbeat_front_addr":"172.21.15.150:6813/1548914683","state":["exists","up"]},{"osd":5,"uuid":"a26c03b8-050c-4a2c-8f23-5801fffb7c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6809","nonce":409836750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6811","nonce":409836750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6815","nonce":409836750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6813","nonce":409836750}]},"public_addr":"172.21.15.162:6809/409836750","cluster_addr":"172.21.15.162:6811/409836750","heartbeat_back_addr":"172.21.15.162:6815/409836750","heartbeat_front_addr":"172.21.15.162:6813/409836750","state":["exists","up"]},{"osd":6,"uuid":"8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6819","nonce":1098984186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6821","nonce":1098984186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6825","nonce":1098984186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6823","nonce":1098984186}]},"public_addr":"172.21.15.112:6819/1098984186","cluster_addr":"172.21.15.112:6821/1098984186","heartbeat_back_addr":"172.21.15.112:6825/1098984186","heartbeat_front_addr":"172.21.15.112:6823/1098984186","state":["exists","up"]},{"osd":7,"uuid":"af3629b4-9bed-4669-97a6-48deba503a96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6817","nonce":4009836564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6819","nonce":4009836564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6823","nonce":4009836564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6821","nonce":4009836564}]},"public_addr":"172.21.15.150:6817/4009836564","cluster_addr":"172.21.15.150:6819/4009836564","heartbeat_back_addr":"172.21.15.150:6823/4009836564","heartbeat_front_addr":"172.21.15.150:6821/4009836564","state":["exists","up"]},{"osd":8,"uuid":"9333e87d-dd78-4467-bbde-c2b44f4ea07f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6817","nonce":852791606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6819","nonce":852791606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6823","nonce":852791606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6821","nonce":852791606}]},"public_addr":"172.21.15.162:6817/852791606","cluster_addr":"172.21.15.162:6819/852791606","heartbeat_back_addr":"172.21.15.162:6823/852791606","heartbeat_front_addr":"172.21.15.162:6821/852791606","state":["exists","up"]},{"osd":9,"uuid":"09723581-1995-42e0-8db7-776ea98a918c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6827","nonce":2186764674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6829","nonce":2186764674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6833","nonce":2186764674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6831","nonce":2186764674}]},"public_addr":"172.21.15.112:6827/2186764674","cluster_addr":"172.21.15.112:6829/2186764674","heartbeat_back_addr":"172.21.15.112:6833/2186764674","heartbeat_front_addr":"172.21.15.112:6831/2186764674","state":["exists","up"]},{"osd":10,"uuid":"34e08e3e-1231-4899-a5b3-76d449105bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6825","nonce":2838995474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6827","nonce":2838995474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6831","nonce":2838995474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6829","nonce":2838995474}]},"public_addr":"172.21.15.150:6825/2838995474","cluster_addr":"172.21.15.150:6827/2838995474","heartbeat_back_addr":"172.21.15.150:6831/2838995474","heartbeat_front_addr":"172.21.15.150:6829/2838995474","state":["exists","up"]},{"osd":11,"uuid":"6fc34ec0-78ca-403c-acb7-b29d83ff64e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6825","nonce":3702614331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6827","nonce":3702614331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6831","nonce":3702614331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6829","nonce":3702614331}]},"public_addr":"172.21.15.162:6825/3702614331","cluster_addr":"172.21.15.162:6827/3702614331","heartbeat_back_addr":"172.21.15.162:6831/3702614331","heartbeat_front_addr":"172.21.15.162:6829/3702614331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:54.518702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:19.939543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:44.425586+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:16.126471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:41.696071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:05.517405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:38.549318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:03.799649+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:28.007282+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:01.395459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:27.963637+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:52.181212+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3667148497":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6800/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3425327158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1404611205":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3765334043":"2024-05-09T05:42:29.819307+0000","172.21.15.112:6800/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/2814303790":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1091751583":"2024-05-09T05:42:29.819307+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:51:15.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:15.038+0000 7fb052a00700 1 -- 172.21.15.112:0/483089733 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb0300b2880 msgr2=0x7fb0300b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:15.038+0000 7fb052a00700 1 --2- 172.21.15.112:0/483089733 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb0300b2880 0x7fb0300b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb04c078bc0 tx=0x7fb038009260 comp rx=0 tx=0).stop 2024-05-08T05:51:15.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:15.038+0000 7fb052a00700 1 -- 172.21.15.112:0/483089733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb04c109ad0 msgr2=0x7fb04c1a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:15.038+0000 7fb052a00700 1 --2- 172.21.15.112:0/483089733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb04c109ad0 0x7fb04c1a01b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00e9a0 tx=0x7fb03c00ecb0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:15.038+0000 7fb052a00700 1 -- 172.21.15.112:0/483089733 shutdown_connections 2024-05-08T05:51:15.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:15.038+0000 7fb052a00700 1 --2- 172.21.15.112:0/483089733 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb0300b2880 0x7fb0300b4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:15.038+0000 7fb052a00700 1 --2- 172.21.15.112:0/483089733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb04c10fb60 0x7fb04c1a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:15.038+0000 7fb052a00700 1 --2- 172.21.15.112:0/483089733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb04c109ad0 0x7fb04c1a01b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:15.038+0000 7fb052a00700 1 --2- 172.21.15.112:0/483089733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c10a490 0x7fb04c1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:15.038+0000 7fb052a00700 1 -- 172.21.15.112:0/483089733 >> 172.21.15.112:0/483089733 conn(0x7fb04c076c70 msgr2=0x7fb04c077250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:15.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:15.039+0000 7fb052a00700 1 -- 172.21.15.112:0/483089733 shutdown_connections 2024-05-08T05:51:15.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:15.039+0000 7fb052a00700 1 -- 172.21.15.112:0/483089733 wait complete. 2024-05-08T05:51:15.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:15 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/483089733' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:15.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:15 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/483089733' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:15.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:15 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/483089733' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:15.986 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-08T05:51:16.435 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:16.814 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:16 smithi112 ceph-mon[30003]: pgmap v316: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-08T05:51:16.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:16 smithi162 ceph-mon[34800]: pgmap v316: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-08T05:51:16.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:16 smithi150 ceph-mon[34315]: pgmap v316: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-08T05:51:18.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.761+0000 7f0d73800700 1 -- 172.21.15.112:0/578020181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d6c1056c0 msgr2=0x7f0d6c105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.761+0000 7f0d73800700 1 --2- 172.21.15.112:0/578020181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d6c1056c0 0x7f0d6c105b40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009990 tx=0x7f0d58009ca0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.762+0000 7f0d73800700 1 -- 172.21.15.112:0/578020181 shutdown_connections 2024-05-08T05:51:18.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.762+0000 7f0d73800700 1 --2- 172.21.15.112:0/578020181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d6c106080 0x7f0d6c112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.762+0000 7f0d73800700 1 --2- 172.21.15.112:0/578020181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d6c10b6c0 0x7f0d6c10baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.762+0000 7f0d73800700 1 --2- 172.21.15.112:0/578020181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d6c1056c0 0x7f0d6c105b40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.762+0000 7f0d73800700 1 -- 172.21.15.112:0/578020181 >> 172.21.15.112:0/578020181 conn(0x7f0d6c1011b0 msgr2=0x7f0d6c1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:18.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.762+0000 7f0d73800700 1 -- 172.21.15.112:0/578020181 shutdown_connections 2024-05-08T05:51:18.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.762+0000 7f0d73800700 1 -- 172.21.15.112:0/578020181 wait complete. 2024-05-08T05:51:18.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.763+0000 7f0d73800700 1 Processor -- start 2024-05-08T05:51:18.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.763+0000 7f0d73800700 1 -- start start 2024-05-08T05:51:18.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.763+0000 7f0d73800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d6c1056c0 0x7f0d6c19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.763+0000 7f0d73800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d6c106080 0x7f0d6c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.763+0000 7f0d73800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d6c10b6c0 0x7f0d6c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.763+0000 7f0d73800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c115950 con 0x7f0d6c106080 2024-05-08T05:51:18.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.763+0000 7f0d73800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c1157d0 con 0x7f0d6c10b6c0 2024-05-08T05:51:18.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.763+0000 7f0d73800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c115ad0 con 0x7f0d6c1056c0 2024-05-08T05:51:18.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.764+0000 7f0d70e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d6c1056c0 0x7f0d6c19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.764+0000 7f0d6be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d6c106080 0x7f0d6c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.764+0000 7f0d70e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d6c1056c0 0x7f0d6c19bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:54756/0 (socket says 172.21.15.112:54756) 2024-05-08T05:51:18.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.764+0000 7f0d70e00700 1 -- 172.21.15.112:0/1167550794 learned_addr learned my addr 172.21.15.112:0/1167550794 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:51:18.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.764+0000 7f0d71800700 1 --2- 172.21.15.112:0/1167550794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d6c10b6c0 0x7f0d6c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.764+0000 7f0d70e00700 1 -- 172.21.15.112:0/1167550794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d6c10b6c0 msgr2=0x7f0d6c1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.764+0000 7f0d70e00700 1 --2- 172.21.15.112:0/1167550794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d6c10b6c0 0x7f0d6c1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.764+0000 7f0d70e00700 1 -- 172.21.15.112:0/1167550794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d6c106080 msgr2=0x7f0d6c19c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.764+0000 7f0d70e00700 1 --2- 172.21.15.112:0/1167550794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d6c106080 0x7f0d6c19c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.764+0000 7f0d70e00700 1 -- 172.21.15.112:0/1167550794 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6c1a0dd0 con 0x7f0d6c1056c0 2024-05-08T05:51:18.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.764+0000 7f0d6be00700 1 --2- 172.21.15.112:0/1167550794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d6c106080 0x7f0d6c19c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:18.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.764+0000 7f0d70e00700 1 --2- 172.21.15.112:0/1167550794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d6c1056c0 0x7f0d6c19bdc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000e9c0 tx=0x7f0d6000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:18.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.765+0000 7f0d69600700 1 -- 172.21.15.112:0/1167550794 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6000cbd0 con 0x7f0d6c1056c0 2024-05-08T05:51:18.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.765+0000 7f0d73800700 1 -- 172.21.15.112:0/1167550794 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6c1a10c0 con 0x7f0d6c1056c0 2024-05-08T05:51:18.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.765+0000 7f0d69600700 1 -- 172.21.15.112:0/1167550794 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d6000cd70 con 0x7f0d6c1056c0 2024-05-08T05:51:18.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.765+0000 7f0d69600700 1 -- 172.21.15.112:0/1167550794 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60005610 con 0x7f0d6c1056c0 2024-05-08T05:51:18.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.765+0000 7f0d73800700 1 -- 172.21.15.112:0/1167550794 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d6c1a8940 con 0x7f0d6c1056c0 2024-05-08T05:51:18.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.766+0000 7f0d73800700 1 -- 172.21.15.112:0/1167550794 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d6c10dc50 con 0x7f0d6c1056c0 2024-05-08T05:51:18.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.768+0000 7f0d69600700 1 -- 172.21.15.112:0/1167550794 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f0d60004750 con 0x7f0d6c1056c0 2024-05-08T05:51:18.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.769+0000 7f0d69600700 1 --2- 172.21.15.112:0/1167550794 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0d540b2880 0x7f0d540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.770+0000 7f0d69600700 1 -- 172.21.15.112:0/1167550794 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f0d600e2420 con 0x7f0d6c1056c0 2024-05-08T05:51:18.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.770+0000 7f0d6be00700 1 --2- 172.21.15.112:0/1167550794 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0d540b2880 0x7f0d540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.771+0000 7f0d6be00700 1 --2- 172.21.15.112:0/1167550794 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0d540b2880 0x7f0d540b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c103830 tx=0x7f0d5801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:18.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:18.774+0000 7f0d69600700 1 -- 172.21.15.112:0/1167550794 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0d600aa1f0 con 0x7f0d6c1056c0 2024-05-08T05:51:18.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:18 smithi112 ceph-mon[30003]: pgmap v317: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-08T05:51:18.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:18 smithi162 ceph-mon[34800]: pgmap v317: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-08T05:51:18.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:18 smithi150 ceph-mon[34315]: pgmap v317: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-08T05:51:19.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:19.056+0000 7f0d73800700 1 -- 172.21.15.112:0/1167550794 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0d6c06a190 con 0x7f0d6c1056c0 2024-05-08T05:51:19.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:19.560+0000 7f0d69600700 1 -- 172.21.15.112:0/1167550794 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f0d600a9af0 con 0x7f0d6c1056c0 2024-05-08T05:51:19.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:19.565+0000 7f0d73800700 1 -- 172.21.15.112:0/1167550794 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0d540b2880 msgr2=0x7f0d540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:19.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:19.565+0000 7f0d73800700 1 --2- 172.21.15.112:0/1167550794 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0d540b2880 0x7f0d540b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c103830 tx=0x7f0d5801a040 comp rx=0 tx=0).stop 2024-05-08T05:51:19.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:19.565+0000 7f0d73800700 1 -- 172.21.15.112:0/1167550794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d6c1056c0 msgr2=0x7f0d6c19bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:19.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:19.565+0000 7f0d73800700 1 --2- 172.21.15.112:0/1167550794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d6c1056c0 0x7f0d6c19bdc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000e9c0 tx=0x7f0d6000ecd0 comp rx=0 tx=0).stop 2024-05-08T05:51:19.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:19.565+0000 7f0d73800700 1 -- 172.21.15.112:0/1167550794 shutdown_connections 2024-05-08T05:51:19.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:19.565+0000 7f0d73800700 1 --2- 172.21.15.112:0/1167550794 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0d540b2880 0x7f0d540b4d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:19.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:19.565+0000 7f0d73800700 1 --2- 172.21.15.112:0/1167550794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d6c10b6c0 0x7f0d6c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:19.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:19.565+0000 7f0d73800700 1 --2- 172.21.15.112:0/1167550794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d6c1056c0 0x7f0d6c19bdc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:19.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:19.565+0000 7f0d73800700 1 --2- 172.21.15.112:0/1167550794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d6c106080 0x7f0d6c19c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:19.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:19.565+0000 7f0d73800700 1 -- 172.21.15.112:0/1167550794 >> 172.21.15.112:0/1167550794 conn(0x7f0d6c1011b0 msgr2=0x7f0d6c1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:19.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:19.565+0000 7f0d73800700 1 -- 172.21.15.112:0/1167550794 shutdown_connections 2024-05-08T05:51:19.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:19.565+0000 7f0d73800700 1 -- 172.21.15.112:0/1167550794 wait complete. 2024-05-08T05:51:19.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:19 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1167550794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T05:51:19.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:19 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T05:51:19.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:19 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1167550794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T05:51:19.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:19 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T05:51:19.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:19 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1167550794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T05:51:19.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:19 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T05:51:20.468 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-08T05:51:20.468 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-08T05:51:20.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:20 smithi112 ceph-mon[30003]: pgmap v318: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-08T05:51:20.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:20 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T05:51:20.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:20 smithi112 ceph-mon[30003]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-08T05:51:20.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:20 smithi162 ceph-mon[34800]: pgmap v318: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-08T05:51:20.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:20 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T05:51:20.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:20 smithi162 ceph-mon[34800]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-08T05:51:20.909 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:20.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:20 smithi150 ceph-mon[34315]: pgmap v318: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-08T05:51:20.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:20 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T05:51:20.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:20 smithi150 ceph-mon[34315]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-08T05:51:22.634 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:22 smithi112 ceph-mon[30003]: pgmap v319: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 2 op/s 2024-05-08T05:51:22.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:22 smithi162 ceph-mon[34800]: pgmap v319: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 2 op/s 2024-05-08T05:51:22.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:22 smithi150 ceph-mon[34315]: pgmap v319: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 2 op/s 2024-05-08T05:51:23.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.090+0000 7ffbd3200700 1 -- 172.21.15.112:0/1921900089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd4103fd0 msgr2=0x7ffbd4108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.090+0000 7ffbd3200700 1 --2- 172.21.15.112:0/1921900089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd4103fd0 0x7ffbd4108020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00098f0 tx=0x7ffbc0009c00 comp rx=0 tx=0).stop 2024-05-08T05:51:23.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.090+0000 7ffbd3200700 1 -- 172.21.15.112:0/1921900089 shutdown_connections 2024-05-08T05:51:23.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.090+0000 7ffbd3200700 1 --2- 172.21.15.112:0/1921900089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbd4108700 0x7ffbd41102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.090+0000 7ffbd3200700 1 --2- 172.21.15.112:0/1921900089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd4103620 0x7ffbd4103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.090+0000 7ffbd3200700 1 --2- 172.21.15.112:0/1921900089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd4103fd0 0x7ffbd4108020 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.090+0000 7ffbd3200700 1 -- 172.21.15.112:0/1921900089 >> 172.21.15.112:0/1921900089 conn(0x7ffbd40feeb0 msgr2=0x7ffbd41012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:23.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.092+0000 7ffbd3200700 1 -- 172.21.15.112:0/1921900089 shutdown_connections 2024-05-08T05:51:23.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.092+0000 7ffbd3200700 1 -- 172.21.15.112:0/1921900089 wait complete. 2024-05-08T05:51:23.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.092+0000 7ffbd3200700 1 Processor -- start 2024-05-08T05:51:23.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.092+0000 7ffbd3200700 1 -- start start 2024-05-08T05:51:23.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.093+0000 7ffbd3200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd4103620 0x7ffbd40757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:23.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.093+0000 7ffbd3200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbd4103fd0 0x7ffbd4075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:23.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.093+0000 7ffbd3200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd4108700 0x7ffbd4076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:23.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.093+0000 7ffbd3200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41135c0 con 0x7ffbd4108700 2024-05-08T05:51:23.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.093+0000 7ffbd3200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd4113440 con 0x7ffbd4103620 2024-05-08T05:51:23.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.093+0000 7ffbd3200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd4113740 con 0x7ffbd4103fd0 2024-05-08T05:51:23.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.093+0000 7ffbd2800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd4108700 0x7ffbd4076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:23.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.093+0000 7ffbd2800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd4108700 0x7ffbd4076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42814/0 (socket says 172.21.15.112:42814) 2024-05-08T05:51:23.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.093+0000 7ffbd1400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbd4103fd0 0x7ffbd4075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:23.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.093+0000 7ffbd2800700 1 -- 172.21.15.112:0/3144554716 learned_addr learned my addr 172.21.15.112:0/3144554716 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:51:23.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.093+0000 7ffbd1e00700 1 --2- 172.21.15.112:0/3144554716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd4103620 0x7ffbd40757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:23.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.094+0000 7ffbd1e00700 1 -- 172.21.15.112:0/3144554716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbd4103fd0 msgr2=0x7ffbd4075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.094+0000 7ffbd1e00700 1 --2- 172.21.15.112:0/3144554716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbd4103fd0 0x7ffbd4075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.094+0000 7ffbd1e00700 1 -- 172.21.15.112:0/3144554716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd4108700 msgr2=0x7ffbd4076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.094+0000 7ffbd1e00700 1 --2- 172.21.15.112:0/3144554716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd4108700 0x7ffbd4076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.094+0000 7ffbd1e00700 1 -- 172.21.15.112:0/3144554716 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd41a8070 con 0x7ffbd4103620 2024-05-08T05:51:23.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.094+0000 7ffbd1400700 1 --2- 172.21.15.112:0/3144554716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbd4103fd0 0x7ffbd4075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:23.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.094+0000 7ffbd2800700 1 --2- 172.21.15.112:0/3144554716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd4108700 0x7ffbd4076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:23.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.094+0000 7ffbd1e00700 1 --2- 172.21.15.112:0/3144554716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd4103620 0x7ffbd40757a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400e9c0 tx=0x7ffbc400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:23.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.095+0000 7ffbcaa00700 1 -- 172.21.15.112:0/3144554716 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc400c9c0 con 0x7ffbd4103620 2024-05-08T05:51:23.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.095+0000 7ffbcaa00700 1 -- 172.21.15.112:0/3144554716 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbc400cb60 con 0x7ffbd4103620 2024-05-08T05:51:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.095+0000 7ffbcaa00700 1 -- 172.21.15.112:0/3144554716 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4018420 con 0x7ffbd4103620 2024-05-08T05:51:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.095+0000 7ffbd3200700 1 -- 172.21.15.112:0/3144554716 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd41a8360 con 0x7ffbd4103620 2024-05-08T05:51:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.095+0000 7ffbd3200700 1 -- 172.21.15.112:0/3144554716 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd41a8820 con 0x7ffbd4103620 2024-05-08T05:51:23.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.096+0000 7ffbd3200700 1 -- 172.21.15.112:0/3144554716 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd406a190 con 0x7ffbd4103620 2024-05-08T05:51:23.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.098+0000 7ffbcaa00700 1 -- 172.21.15.112:0/3144554716 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ffbc4010430 con 0x7ffbd4103620 2024-05-08T05:51:23.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.100+0000 7ffbcaa00700 1 --2- 172.21.15.112:0/3144554716 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffbb80b2900 0x7ffbb80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:23.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.100+0000 7ffbcaa00700 1 -- 172.21.15.112:0/3144554716 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ffbc40e2700 con 0x7ffbd4103620 2024-05-08T05:51:23.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.100+0000 7ffbd1400700 1 --2- 172.21.15.112:0/3144554716 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffbb80b2900 0x7ffbb80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:23.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.101+0000 7ffbd1400700 1 --2- 172.21.15.112:0/3144554716 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffbb80b2900 0x7ffbb80b4dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009fd0 tx=0x7ffbc0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:23.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.102+0000 7ffbcaa00700 1 -- 172.21.15.112:0/3144554716 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffbc40aa4d0 con 0x7ffbd4103620 2024-05-08T05:51:23.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.386+0000 7ffbd3200700 1 -- 172.21.15.112:0/3144554716 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7ffbd402e510 con 0x7ffbd4103620 2024-05-08T05:51:23.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.582+0000 7ffbcaa00700 1 -- 172.21.15.112:0/3144554716 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7ffbc40a9dd0 con 0x7ffbd4103620 2024-05-08T05:51:23.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.586+0000 7ffbd3200700 1 -- 172.21.15.112:0/3144554716 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffbb80b2900 msgr2=0x7ffbb80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.586+0000 7ffbd3200700 1 --2- 172.21.15.112:0/3144554716 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffbb80b2900 0x7ffbb80b4dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009fd0 tx=0x7ffbc0019040 comp rx=0 tx=0).stop 2024-05-08T05:51:23.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.586+0000 7ffbd3200700 1 -- 172.21.15.112:0/3144554716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd4103620 msgr2=0x7ffbd40757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.586+0000 7ffbd3200700 1 --2- 172.21.15.112:0/3144554716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd4103620 0x7ffbd40757a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400e9c0 tx=0x7ffbc400ecd0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.587+0000 7ffbd3200700 1 -- 172.21.15.112:0/3144554716 shutdown_connections 2024-05-08T05:51:23.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.587+0000 7ffbd3200700 1 --2- 172.21.15.112:0/3144554716 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffbb80b2900 0x7ffbb80b4dc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.587+0000 7ffbd3200700 1 --2- 172.21.15.112:0/3144554716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd4108700 0x7ffbd4076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.587+0000 7ffbd3200700 1 --2- 172.21.15.112:0/3144554716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd4103620 0x7ffbd40757a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.587+0000 7ffbd3200700 1 --2- 172.21.15.112:0/3144554716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbd4103fd0 0x7ffbd4075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.587+0000 7ffbd3200700 1 -- 172.21.15.112:0/3144554716 >> 172.21.15.112:0/3144554716 conn(0x7ffbd40feeb0 msgr2=0x7ffbd4100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:23.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.587+0000 7ffbd3200700 1 -- 172.21.15.112:0/3144554716 shutdown_connections 2024-05-08T05:51:23.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:23.587+0000 7ffbd3200700 1 -- 172.21.15.112:0/3144554716 wait complete. 2024-05-08T05:51:23.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:23 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3144554716' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T05:51:23.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:23 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T05:51:23.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:23 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3144554716' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T05:51:23.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:23 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T05:51:24.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:23 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3144554716' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T05:51:24.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:23 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T05:51:24.424 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-08T05:51:24.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:24 smithi112 ceph-mon[30003]: pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 0 op/s 2024-05-08T05:51:24.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:24 smithi112 ceph-mon[30003]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T05:51:24.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:24 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-08T05:51:24.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:24 smithi112 ceph-mon[30003]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T05:51:24.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:24 smithi112 ceph-mon[30003]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-08T05:51:24.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:24 smithi112 ceph-mon[30003]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2024-05-08T05:51:24.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:24 smithi112 ceph-mon[30003]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-08T05:51:24.888 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:24 smithi150 ceph-mon[34315]: pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 0 op/s 2024-05-08T05:51:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:24 smithi150 ceph-mon[34315]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T05:51:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:24 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-08T05:51:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:24 smithi150 ceph-mon[34315]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T05:51:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:24 smithi150 ceph-mon[34315]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-08T05:51:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:24 smithi150 ceph-mon[34315]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2024-05-08T05:51:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:24 smithi150 ceph-mon[34315]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-08T05:51:25.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:24 smithi162 ceph-mon[34800]: pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 0 op/s 2024-05-08T05:51:25.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:24 smithi162 ceph-mon[34800]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T05:51:25.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:24 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-08T05:51:25.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:24 smithi162 ceph-mon[34800]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T05:51:25.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:24 smithi162 ceph-mon[34800]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2024-05-08T05:51:25.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:24 smithi162 ceph-mon[34800]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2024-05-08T05:51:25.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:24 smithi162 ceph-mon[34800]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-08T05:51:25.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:25 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] up:active 2024-05-08T05:51:25.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:25 smithi112 ceph-mon[30003]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T05:51:25.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:25 smithi112 ceph-mon[30003]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T05:51:25.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:25 smithi112 ceph-mon[30003]: Cluster is now healthy 2024-05-08T05:51:25.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:25 smithi112 ceph-mon[30003]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2024-05-08T05:51:25.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:25 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2024-05-08T05:51:25.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:25 smithi112 ceph-mon[30003]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-08T05:51:25.848 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:25 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] up:active 2024-05-08T05:51:25.848 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:25 smithi150 ceph-mon[34315]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T05:51:25.849 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:25 smithi150 ceph-mon[34315]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T05:51:25.849 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:25 smithi150 ceph-mon[34315]: Cluster is now healthy 2024-05-08T05:51:25.849 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:25 smithi150 ceph-mon[34315]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2024-05-08T05:51:25.849 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:25 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2024-05-08T05:51:25.849 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:25 smithi150 ceph-mon[34315]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-08T05:51:26.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:25 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] up:active 2024-05-08T05:51:26.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:25 smithi162 ceph-mon[34800]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T05:51:26.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:25 smithi162 ceph-mon[34800]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T05:51:26.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:25 smithi162 ceph-mon[34800]: Cluster is now healthy 2024-05-08T05:51:26.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:25 smithi162 ceph-mon[34800]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2024-05-08T05:51:26.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:25 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2024-05-08T05:51:26.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:25 smithi162 ceph-mon[34800]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-08T05:51:26.751 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:26 smithi112 ceph-mon[30003]: pgmap v321: 97 pgs: 97 active+clean; 580 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 2 op/s 2024-05-08T05:51:26.751 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:26 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] up:active 2024-05-08T05:51:26.751 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:26 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2024-05-08T05:51:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:26 smithi150 ceph-mon[34315]: pgmap v321: 97 pgs: 97 active+clean; 580 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 2 op/s 2024-05-08T05:51:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:26 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] up:active 2024-05-08T05:51:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:26 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2024-05-08T05:51:27.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.040+0000 7f00b4e00700 1 -- 172.21.15.112:0/3447535067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00b0104eb0 msgr2=0x7f00b010d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:27.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.040+0000 7f00b4e00700 1 --2- 172.21.15.112:0/3447535067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00b0104eb0 0x7f00b010d3a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0098009990 tx=0x7f0098009ca0 comp rx=0 tx=0).stop 2024-05-08T05:51:27.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.040+0000 7f00b4e00700 1 -- 172.21.15.112:0/3447535067 shutdown_connections 2024-05-08T05:51:27.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.040+0000 7f00b4e00700 1 --2- 172.21.15.112:0/3447535067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b010d9c0 0x7f00b010fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:27.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.040+0000 7f00b4e00700 1 --2- 172.21.15.112:0/3447535067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00b0104590 0x7f00b0104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:27.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.040+0000 7f00b4e00700 1 --2- 172.21.15.112:0/3447535067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00b0104eb0 0x7f00b010d3a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:27.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.040+0000 7f00b4e00700 1 -- 172.21.15.112:0/3447535067 >> 172.21.15.112:0/3447535067 conn(0x7f00b00fdda0 msgr2=0x7f00b01001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:27.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.040+0000 7f00b4e00700 1 -- 172.21.15.112:0/3447535067 shutdown_connections 2024-05-08T05:51:27.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.041+0000 7f00b4e00700 1 -- 172.21.15.112:0/3447535067 wait complete. 2024-05-08T05:51:27.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.042+0000 7f00b4e00700 1 Processor -- start 2024-05-08T05:51:27.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.042+0000 7f00b4e00700 1 -- start start 2024-05-08T05:51:27.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.043+0000 7f00b4e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b0104590 0x7f00b019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:27.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.043+0000 7f00b4e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00b0104eb0 0x7f00b019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:27.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.043+0000 7f00b4e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00b010d9c0 0x7f00b01a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:27.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.043+0000 7f00b4e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00b01135e0 con 0x7f00b0104eb0 2024-05-08T05:51:27.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.043+0000 7f00b4e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f00b0113460 con 0x7f00b0104590 2024-05-08T05:51:27.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.043+0000 7f00b4e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f00b0113760 con 0x7f00b010d9c0 2024-05-08T05:51:27.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.043+0000 7f00af400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b0104590 0x7f00b019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:27.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.043+0000 7f00a6a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00b0104eb0 0x7f00b019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:27.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.043+0000 7f00af400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b0104590 0x7f00b019bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:44998/0 (socket says 172.21.15.112:44998) 2024-05-08T05:51:27.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.043+0000 7f00af400700 1 -- 172.21.15.112:0/3892087549 learned_addr learned my addr 172.21.15.112:0/3892087549 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:51:27.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.043+0000 7f00afe00700 1 --2- 172.21.15.112:0/3892087549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00b010d9c0 0x7f00b01a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.044+0000 7f00af400700 1 -- 172.21.15.112:0/3892087549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00b010d9c0 msgr2=0x7f00b01a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.044+0000 7f00af400700 1 --2- 172.21.15.112:0/3892087549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00b010d9c0 0x7f00b01a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.044+0000 7f00af400700 1 -- 172.21.15.112:0/3892087549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00b0104eb0 msgr2=0x7f00b019c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.044+0000 7f00af400700 1 --2- 172.21.15.112:0/3892087549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00b0104eb0 0x7f00b019c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.044+0000 7f00af400700 1 -- 172.21.15.112:0/3892087549 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b01a0d60 con 0x7f00b0104590 2024-05-08T05:51:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.044+0000 7f00a6a00700 1 --2- 172.21.15.112:0/3892087549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00b0104eb0 0x7f00b019c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.044+0000 7f00af400700 1 --2- 172.21.15.112:0/3892087549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b0104590 0x7f00b019bd50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f00a000e9a0 tx=0x7f00a000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.044+0000 7f00acc00700 1 -- 172.21.15.112:0/3892087549 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a000c9c0 con 0x7f00b0104590 2024-05-08T05:51:27.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.044+0000 7f00acc00700 1 -- 172.21.15.112:0/3892087549 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00a000cb60 con 0x7f00b0104590 2024-05-08T05:51:27.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.044+0000 7f00b4e00700 1 -- 172.21.15.112:0/3892087549 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b01a1050 con 0x7f00b0104590 2024-05-08T05:51:27.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.044+0000 7f00acc00700 1 -- 172.21.15.112:0/3892087549 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0018680 con 0x7f00b0104590 2024-05-08T05:51:27.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.044+0000 7f00b4e00700 1 -- 172.21.15.112:0/3892087549 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b01a88d0 con 0x7f00b0104590 2024-05-08T05:51:27.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.046+0000 7f00b4e00700 1 -- 172.21.15.112:0/3892087549 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b006a190 con 0x7f00b0104590 2024-05-08T05:51:27.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.048+0000 7f00acc00700 1 -- 172.21.15.112:0/3892087549 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f00a0010430 con 0x7f00b0104590 2024-05-08T05:51:27.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.049+0000 7f00acc00700 1 --2- 172.21.15.112:0/3892087549 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f00940b2880 0x7f00940b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:27.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.050+0000 7f00acc00700 1 -- 172.21.15.112:0/3892087549 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f00a00e23f0 con 0x7f00b0104590 2024-05-08T05:51:27.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.050+0000 7f00a6a00700 1 --2- 172.21.15.112:0/3892087549 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f00940b2880 0x7f00940b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:27.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.050+0000 7f00a6a00700 1 --2- 172.21.15.112:0/3892087549 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f00940b2880 0x7f00940b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f00b01102e0 tx=0x7f009800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:27.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.053+0000 7f00acc00700 1 -- 172.21.15.112:0/3892087549 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00a00aa1c0 con 0x7f00b0104590 2024-05-08T05:51:27.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:26 smithi162 ceph-mon[34800]: pgmap v321: 97 pgs: 97 active+clean; 580 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 2 op/s 2024-05-08T05:51:27.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:26 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] up:active 2024-05-08T05:51:27.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:26 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2024-05-08T05:51:27.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.335+0000 7f00b4e00700 1 -- 172.21.15.112:0/3892087549 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f00b006cec0 con 0x7f00b0104590 2024-05-08T05:51:27.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.622+0000 7f00acc00700 1 -- 172.21.15.112:0/3892087549 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f00a00b2240 con 0x7f00b0104590 2024-05-08T05:51:27.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.626+0000 7f00b4e00700 1 -- 172.21.15.112:0/3892087549 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f00940b2880 msgr2=0x7f00940b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:27.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.626+0000 7f00b4e00700 1 --2- 172.21.15.112:0/3892087549 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f00940b2880 0x7f00940b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f00b01102e0 tx=0x7f009800b260 comp rx=0 tx=0).stop 2024-05-08T05:51:27.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.626+0000 7f00b4e00700 1 -- 172.21.15.112:0/3892087549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b0104590 msgr2=0x7f00b019bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:27.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.626+0000 7f00b4e00700 1 --2- 172.21.15.112:0/3892087549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b0104590 0x7f00b019bd50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f00a000e9a0 tx=0x7f00a000ecb0 comp rx=0 tx=0).stop 2024-05-08T05:51:27.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.627+0000 7f00b4e00700 1 -- 172.21.15.112:0/3892087549 shutdown_connections 2024-05-08T05:51:27.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.627+0000 7f00b4e00700 1 --2- 172.21.15.112:0/3892087549 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f00940b2880 0x7f00940b4d40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:27.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.627+0000 7f00b4e00700 1 --2- 172.21.15.112:0/3892087549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00b010d9c0 0x7f00b01a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:27.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.627+0000 7f00b4e00700 1 --2- 172.21.15.112:0/3892087549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b0104590 0x7f00b019bd50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:27.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.627+0000 7f00b4e00700 1 --2- 172.21.15.112:0/3892087549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00b0104eb0 0x7f00b019c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:27.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.627+0000 7f00b4e00700 1 -- 172.21.15.112:0/3892087549 >> 172.21.15.112:0/3892087549 conn(0x7f00b00fdda0 msgr2=0x7f00b0100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:27.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.627+0000 7f00b4e00700 1 -- 172.21.15.112:0/3892087549 shutdown_connections 2024-05-08T05:51:27.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:27.627+0000 7f00b4e00700 1 -- 172.21.15.112:0/3892087549 wait complete. 2024-05-08T05:51:27.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:27 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3892087549' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T05:51:27.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:27 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T05:51:27.989 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:27 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3892087549' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T05:51:27.989 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:27 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T05:51:28.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:27 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3892087549' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T05:51:28.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:27 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T05:51:28.428 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-08T05:51:28.829 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:28.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:28 smithi150 ceph-mon[34315]: pgmap v322: 97 pgs: 97 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-08T05:51:28.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:28 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T05:51:28.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:28 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2024-05-08T05:51:28.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:28 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:51:28.977 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:28 smithi112 ceph-mon[30003]: pgmap v322: 97 pgs: 97 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-08T05:51:28.977 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:28 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T05:51:28.977 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:28 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2024-05-08T05:51:28.977 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:28 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:51:29.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:28 smithi162 ceph-mon[34800]: pgmap v322: 97 pgs: 97 active+clean; 581 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-08T05:51:29.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:28 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T05:51:29.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:28 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2024-05-08T05:51:29.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:28 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:51:30.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:30 smithi112 ceph-mon[30003]: pgmap v323: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-08T05:51:30.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:30 smithi150 ceph-mon[34315]: pgmap v323: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-08T05:51:31.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:30 smithi162 ceph-mon[34800]: pgmap v323: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-08T05:51:31.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.068+0000 7fa43ba00700 1 -- 172.21.15.112:0/2523594296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa43406c5a0 msgr2=0x7fa4341103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:31.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.068+0000 7fa43ba00700 1 --2- 172.21.15.112:0/2523594296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa43406c5a0 0x7fa4341103d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00b0a0 tx=0x7fa42c00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:51:31.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.068+0000 7fa43ba00700 1 -- 172.21.15.112:0/2523594296 shutdown_connections 2024-05-08T05:51:31.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.068+0000 7fa43ba00700 1 --2- 172.21.15.112:0/2523594296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa43406c5a0 0x7fa4341103d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:31.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.068+0000 7fa43ba00700 1 --2- 172.21.15.112:0/2523594296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa434109640 0x7fa434109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:31.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.068+0000 7fa43ba00700 1 --2- 172.21.15.112:0/2523594296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa43406bbe0 0x7fa43406c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:31.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.068+0000 7fa43ba00700 1 -- 172.21.15.112:0/2523594296 >> 172.21.15.112:0/2523594296 conn(0x7fa4340791a0 msgr2=0x7fa4340795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:31.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.069+0000 7fa43ba00700 1 -- 172.21.15.112:0/2523594296 shutdown_connections 2024-05-08T05:51:31.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.069+0000 7fa43ba00700 1 -- 172.21.15.112:0/2523594296 wait complete. 2024-05-08T05:51:31.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.070+0000 7fa43ba00700 1 Processor -- start 2024-05-08T05:51:31.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.070+0000 7fa43ba00700 1 -- start start 2024-05-08T05:51:31.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.071+0000 7fa43ba00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa43406bbe0 0x7fa43419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:31.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.071+0000 7fa43ba00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa43406c5a0 0x7fa43419c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:31.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.071+0000 7fa43ba00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa434109640 0x7fa4341a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:31.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.071+0000 7fa43ba00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa434113720 con 0x7fa43406bbe0 2024-05-08T05:51:31.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.071+0000 7fa43ba00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa4341135a0 con 0x7fa434109640 2024-05-08T05:51:31.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.071+0000 7fa43ba00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa4341138a0 con 0x7fa43406c5a0 2024-05-08T05:51:31.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.071+0000 7fa439000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa43406bbe0 0x7fa43419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:31.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.071+0000 7fa433e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa43406c5a0 0x7fa43419c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:31.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.071+0000 7fa433e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa43406c5a0 0x7fa43419c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:36140/0 (socket says 172.21.15.112:36140) 2024-05-08T05:51:31.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.071+0000 7fa439000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa43406bbe0 0x7fa43419bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42302/0 (socket says 172.21.15.112:42302) 2024-05-08T05:51:31.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.071+0000 7fa433e00700 1 -- 172.21.15.112:0/3368518014 learned_addr learned my addr 172.21.15.112:0/3368518014 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:51:31.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.071+0000 7fa439a00700 1 --2- 172.21.15.112:0/3368518014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa434109640 0x7fa4341a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:31.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.072+0000 7fa433e00700 1 -- 172.21.15.112:0/3368518014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa434109640 msgr2=0x7fa4341a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:31.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.072+0000 7fa433e00700 1 --2- 172.21.15.112:0/3368518014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa434109640 0x7fa4341a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:31.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.072+0000 7fa433e00700 1 -- 172.21.15.112:0/3368518014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa43406bbe0 msgr2=0x7fa43419bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:31.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.072+0000 7fa433e00700 1 --2- 172.21.15.112:0/3368518014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa43406bbe0 0x7fa43419bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:31.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.072+0000 7fa433e00700 1 -- 172.21.15.112:0/3368518014 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4341a0d40 con 0x7fa43406c5a0 2024-05-08T05:51:31.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.072+0000 7fa433e00700 1 --2- 172.21.15.112:0/3368518014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa43406c5a0 0x7fa43419c270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa42000e9c0 tx=0x7fa42000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:31.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.072+0000 7fa431600700 1 -- 172.21.15.112:0/3368518014 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42000cbd0 con 0x7fa43406c5a0 2024-05-08T05:51:31.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.072+0000 7fa431600700 1 -- 172.21.15.112:0/3368518014 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa42000ee60 con 0x7fa43406c5a0 2024-05-08T05:51:31.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.072+0000 7fa43ba00700 1 -- 172.21.15.112:0/3368518014 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4341a1030 con 0x7fa43406c5a0 2024-05-08T05:51:31.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.072+0000 7fa431600700 1 -- 172.21.15.112:0/3368518014 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4200055c0 con 0x7fa43406c5a0 2024-05-08T05:51:31.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.072+0000 7fa43ba00700 1 -- 172.21.15.112:0/3368518014 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4341a88b0 con 0x7fa43406c5a0 2024-05-08T05:51:31.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.073+0000 7fa43ba00700 1 -- 172.21.15.112:0/3368518014 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa43406a190 con 0x7fa43406c5a0 2024-05-08T05:51:31.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.076+0000 7fa431600700 1 -- 172.21.15.112:0/3368518014 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fa420047960 con 0x7fa43406c5a0 2024-05-08T05:51:31.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.077+0000 7fa431600700 1 --2- 172.21.15.112:0/3368518014 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa41c0b2880 0x7fa41c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:31.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.077+0000 7fa431600700 1 -- 172.21.15.112:0/3368518014 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fa4200e23b0 con 0x7fa43406c5a0 2024-05-08T05:51:31.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.077+0000 7fa439000700 1 --2- 172.21.15.112:0/3368518014 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa41c0b2880 0x7fa41c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:31.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.078+0000 7fa439000700 1 --2- 172.21.15.112:0/3368518014 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa41c0b2880 0x7fa41c0b4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa428009c60 tx=0x7fa428009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:31.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.080+0000 7fa431600700 1 -- 172.21.15.112:0/3368518014 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa4200aa180 con 0x7fa43406c5a0 2024-05-08T05:51:31.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.366+0000 7fa43ba00700 1 -- 172.21.15.112:0/3368518014 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa4341a8a80 con 0x7fa43406c5a0 2024-05-08T05:51:31.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.643+0000 7fa431600700 1 -- 172.21.15.112:0/3368518014 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7fa4200b2030 con 0x7fa43406c5a0 2024-05-08T05:51:31.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.647+0000 7fa43ba00700 1 -- 172.21.15.112:0/3368518014 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa41c0b2880 msgr2=0x7fa41c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:31.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.647+0000 7fa43ba00700 1 --2- 172.21.15.112:0/3368518014 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa41c0b2880 0x7fa41c0b4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa428009c60 tx=0x7fa428009260 comp rx=0 tx=0).stop 2024-05-08T05:51:31.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.647+0000 7fa43ba00700 1 -- 172.21.15.112:0/3368518014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa43406c5a0 msgr2=0x7fa43419c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:31.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.647+0000 7fa43ba00700 1 --2- 172.21.15.112:0/3368518014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa43406c5a0 0x7fa43419c270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa42000e9c0 tx=0x7fa42000ecd0 comp rx=0 tx=0).stop 2024-05-08T05:51:31.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.647+0000 7fa43ba00700 1 -- 172.21.15.112:0/3368518014 shutdown_connections 2024-05-08T05:51:31.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.647+0000 7fa43ba00700 1 --2- 172.21.15.112:0/3368518014 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa41c0b2880 0x7fa41c0b4d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:31.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.647+0000 7fa43ba00700 1 --2- 172.21.15.112:0/3368518014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa434109640 0x7fa4341a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:31.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.647+0000 7fa43ba00700 1 --2- 172.21.15.112:0/3368518014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa43406bbe0 0x7fa43419bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:31.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.647+0000 7fa43ba00700 1 --2- 172.21.15.112:0/3368518014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa43406c5a0 0x7fa43419c270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:31.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.647+0000 7fa43ba00700 1 -- 172.21.15.112:0/3368518014 >> 172.21.15.112:0/3368518014 conn(0x7fa4340791a0 msgr2=0x7fa43410e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:31.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.648+0000 7fa43ba00700 1 -- 172.21.15.112:0/3368518014 shutdown_connections 2024-05-08T05:51:31.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:31.648+0000 7fa43ba00700 1 -- 172.21.15.112:0/3368518014 wait complete. 2024-05-08T05:51:31.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:31 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3368518014' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T05:51:31.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:31 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T05:51:31.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:31 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3368518014' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T05:51:31.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:31 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T05:51:31.938 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:31 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3368518014' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T05:51:31.938 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:31 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T05:51:32.359 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-05-08T05:51:32.359 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-08T05:51:32.804 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:32.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:32 smithi150 ceph-mon[34315]: pgmap v324: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-08T05:51:32.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:32 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T05:51:32.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:32 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:51:32.949 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:32 smithi112 ceph-mon[30003]: pgmap v324: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-08T05:51:32.949 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:32 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T05:51:32.949 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:32 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:51:33.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:32 smithi162 ceph-mon[34800]: pgmap v324: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-08T05:51:33.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:32 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T05:51:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:32 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:51:34.871 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:34 smithi112 ceph-mon[30003]: pgmap v325: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-08T05:51:34.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:34 smithi150 ceph-mon[34315]: pgmap v325: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-08T05:51:35.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.039+0000 7fcd29800700 1 -- 172.21.15.112:0/479883484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd2410b680 msgr2=0x7fcd2410ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.039+0000 7fcd29800700 1 --2- 172.21.15.112:0/479883484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd2410b680 0x7fcd2410ba60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009990 tx=0x7fcd14009ca0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.039+0000 7fcd29800700 1 -- 172.21.15.112:0/479883484 shutdown_connections 2024-05-08T05:51:35.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.039+0000 7fcd29800700 1 --2- 172.21.15.112:0/479883484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd24106040 0x7fcd24112550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.039+0000 7fcd29800700 1 --2- 172.21.15.112:0/479883484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd2410b680 0x7fcd2410ba60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.039+0000 7fcd29800700 1 --2- 172.21.15.112:0/479883484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd24105680 0x7fcd24105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.039+0000 7fcd29800700 1 -- 172.21.15.112:0/479883484 >> 172.21.15.112:0/479883484 conn(0x7fcd24101150 msgr2=0x7fcd24103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:35.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.040+0000 7fcd29800700 1 -- 172.21.15.112:0/479883484 shutdown_connections 2024-05-08T05:51:35.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.040+0000 7fcd29800700 1 -- 172.21.15.112:0/479883484 wait complete. 2024-05-08T05:51:35.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.041+0000 7fcd29800700 1 Processor -- start 2024-05-08T05:51:35.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.041+0000 7fcd29800700 1 -- start start 2024-05-08T05:51:35.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.041+0000 7fcd29800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd24105680 0x7fcd2419bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.041+0000 7fcd29800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd24106040 0x7fcd2419c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.041+0000 7fcd29800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd2410b680 0x7fcd241a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.041+0000 7fcd29800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcd241158d0 con 0x7fcd24105680 2024-05-08T05:51:35.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.041+0000 7fcd29800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcd24115750 con 0x7fcd2410b680 2024-05-08T05:51:35.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.041+0000 7fcd29800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fcd24115a50 con 0x7fcd24106040 2024-05-08T05:51:35.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.042+0000 7fcd22a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd24105680 0x7fcd2419bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.042+0000 7fcd1be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd24106040 0x7fcd2419c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.042+0000 7fcd1be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd24106040 0x7fcd2419c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:57078/0 (socket says 172.21.15.112:57078) 2024-05-08T05:51:35.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.042+0000 7fcd1be00700 1 -- 172.21.15.112:0/4268857874 learned_addr learned my addr 172.21.15.112:0/4268857874 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:51:35.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.042+0000 7fcd23400700 1 --2- 172.21.15.112:0/4268857874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd2410b680 0x7fcd241a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.042+0000 7fcd1be00700 1 -- 172.21.15.112:0/4268857874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd2410b680 msgr2=0x7fcd241a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.042+0000 7fcd1be00700 1 --2- 172.21.15.112:0/4268857874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd2410b680 0x7fcd241a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.042+0000 7fcd1be00700 1 -- 172.21.15.112:0/4268857874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd24105680 msgr2=0x7fcd2419bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.042+0000 7fcd1be00700 1 --2- 172.21.15.112:0/4268857874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd24105680 0x7fcd2419bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.042+0000 7fcd1be00700 1 -- 172.21.15.112:0/4268857874 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd241a0df0 con 0x7fcd24106040 2024-05-08T05:51:35.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.042+0000 7fcd1be00700 1 --2- 172.21.15.112:0/4268857874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd24106040 0x7fcd2419c320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00e9c0 tx=0x7fcd0c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:35.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.043+0000 7fcd1b400700 1 -- 172.21.15.112:0/4268857874 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c00cbd0 con 0x7fcd24106040 2024-05-08T05:51:35.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.043+0000 7fcd1b400700 1 -- 172.21.15.112:0/4268857874 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd0c00ee50 con 0x7fcd24106040 2024-05-08T05:51:35.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.043+0000 7fcd1b400700 1 -- 172.21.15.112:0/4268857874 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c005550 con 0x7fcd24106040 2024-05-08T05:51:35.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.043+0000 7fcd29800700 1 -- 172.21.15.112:0/4268857874 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd241a10e0 con 0x7fcd24106040 2024-05-08T05:51:35.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.043+0000 7fcd29800700 1 -- 172.21.15.112:0/4268857874 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd241a8960 con 0x7fcd24106040 2024-05-08T05:51:35.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.044+0000 7fcd29800700 1 -- 172.21.15.112:0/4268857874 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd2410dc10 con 0x7fcd24106040 2024-05-08T05:51:35.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.047+0000 7fcd1b400700 1 -- 172.21.15.112:0/4268857874 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fcd0c047960 con 0x7fcd24106040 2024-05-08T05:51:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.048+0000 7fcd1b400700 1 --2- 172.21.15.112:0/4268857874 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcd080b2900 0x7fcd080b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.048+0000 7fcd1b400700 1 -- 172.21.15.112:0/4268857874 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fcd0c0e2330 con 0x7fcd24106040 2024-05-08T05:51:35.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.048+0000 7fcd22a00700 1 --2- 172.21.15.112:0/4268857874 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcd080b2900 0x7fcd080b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.049+0000 7fcd22a00700 1 --2- 172.21.15.112:0/4268857874 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcd080b2900 0x7fcd080b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcd14006010 tx=0x7fcd1400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:35.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.050+0000 7fcd1b400700 1 -- 172.21.15.112:0/4268857874 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcd0c0aa100 con 0x7fcd24106040 2024-05-08T05:51:35.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:34 smithi162 ceph-mon[34800]: pgmap v325: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-08T05:51:35.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.314+0000 7fcd29800700 1 -- 172.21.15.112:0/4268857874 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcd240638e0 con 0x7fcd080b2900 2024-05-08T05:51:35.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.354+0000 7fcd1b400700 1 -- 172.21.15.112:0/4268857874 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcd240638e0 con 0x7fcd080b2900 2024-05-08T05:51:35.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.357+0000 7fcd29800700 1 -- 172.21.15.112:0/4268857874 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcd080b2900 msgr2=0x7fcd080b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.357+0000 7fcd29800700 1 --2- 172.21.15.112:0/4268857874 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcd080b2900 0x7fcd080b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcd14006010 tx=0x7fcd1400b260 comp rx=0 tx=0).stop 2024-05-08T05:51:35.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.357+0000 7fcd29800700 1 -- 172.21.15.112:0/4268857874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd24106040 msgr2=0x7fcd2419c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.357+0000 7fcd29800700 1 --2- 172.21.15.112:0/4268857874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd24106040 0x7fcd2419c320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00e9c0 tx=0x7fcd0c00ecd0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.357+0000 7fcd29800700 1 -- 172.21.15.112:0/4268857874 shutdown_connections 2024-05-08T05:51:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.357+0000 7fcd29800700 1 --2- 172.21.15.112:0/4268857874 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcd080b2900 0x7fcd080b4dc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.357+0000 7fcd29800700 1 --2- 172.21.15.112:0/4268857874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd2410b680 0x7fcd241a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.357+0000 7fcd29800700 1 --2- 172.21.15.112:0/4268857874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd24105680 0x7fcd2419bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.357+0000 7fcd29800700 1 --2- 172.21.15.112:0/4268857874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd24106040 0x7fcd2419c320 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.357+0000 7fcd29800700 1 -- 172.21.15.112:0/4268857874 >> 172.21.15.112:0/4268857874 conn(0x7fcd24101150 msgr2=0x7fcd24101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:35.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.357+0000 7fcd29800700 1 -- 172.21.15.112:0/4268857874 shutdown_connections 2024-05-08T05:51:35.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:35.357+0000 7fcd29800700 1 -- 172.21.15.112:0/4268857874 wait complete. 2024-05-08T05:51:35.566 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:51:35 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:51:35.324+0000 7fa0ce200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:51:35.566 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:51:35 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:51:35.324+0000 7fa0ce200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:51:35.567 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:51:35 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:51:35.324+0000 7fa0ce200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:51:35.567 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:51:35 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:51:35.324+0000 7fa0ce200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:51:35.567 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 05:51:35 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T05:51:35.324+0000 7fa0ce200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:51:36.174 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-08T05:51:36.622 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:36.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:36 smithi150 ceph-mon[34315]: from='client.24610 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:36.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:36 smithi150 ceph-mon[34315]: pgmap v326: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-08T05:51:36.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:36 smithi150 ceph-mon[34315]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-08T05:51:36.953 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:36 smithi112 ceph-mon[30003]: from='client.24610 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:36.953 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:36 smithi112 ceph-mon[30003]: pgmap v326: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-08T05:51:36.953 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:36 smithi112 ceph-mon[30003]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-08T05:51:37.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:36 smithi162 ceph-mon[34800]: from='client.24610 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:37.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:36 smithi162 ceph-mon[34800]: pgmap v326: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-08T05:51:37.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:36 smithi162 ceph-mon[34800]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-08T05:51:38.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.826+0000 7f92bfe00700 1 -- 172.21.15.112:0/3253336826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92c0103620 msgr2=0x7f92c0103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:38.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.826+0000 7f92bfe00700 1 --2- 172.21.15.112:0/3253336826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92c0103620 0x7f92c0103a00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009990 tx=0x7f92b0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.827+0000 7f92bfe00700 1 -- 172.21.15.112:0/3253336826 shutdown_connections 2024-05-08T05:51:38.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.827+0000 7f92bfe00700 1 --2- 172.21.15.112:0/3253336826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c0108700 0x7f92c01102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.827+0000 7f92bfe00700 1 --2- 172.21.15.112:0/3253336826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92c0103620 0x7f92c0103a00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.827+0000 7f92bfe00700 1 --2- 172.21.15.112:0/3253336826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c0103fd0 0x7f92c0108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.827+0000 7f92bfe00700 1 -- 172.21.15.112:0/3253336826 >> 172.21.15.112:0/3253336826 conn(0x7f92c00feeb0 msgr2=0x7f92c01012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:38.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.827+0000 7f92bfe00700 1 -- 172.21.15.112:0/3253336826 shutdown_connections 2024-05-08T05:51:38.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.827+0000 7f92bfe00700 1 -- 172.21.15.112:0/3253336826 wait complete. 2024-05-08T05:51:38.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.828+0000 7f92bfe00700 1 Processor -- start 2024-05-08T05:51:38.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.828+0000 7f92bfe00700 1 -- start start 2024-05-08T05:51:38.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.829+0000 7f92bfe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c0103620 0x7f92c01a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.829+0000 7f92bfe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c0103fd0 0x7f92c01a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.829+0000 7f92bfe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92c0108700 0x7f92c019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.829+0000 7f92bfe00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92c01135e0 con 0x7f92c0103fd0 2024-05-08T05:51:38.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.829+0000 7f92bfe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f92c0113460 con 0x7f92c0108700 2024-05-08T05:51:38.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.829+0000 7f92bfe00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f92c0113760 con 0x7f92c0103620 2024-05-08T05:51:38.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.829+0000 7f92b6000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c0103fd0 0x7f92c01a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.829+0000 7f92b6000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c0103fd0 0x7f92c01a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36588/0 (socket says 172.21.15.112:36588) 2024-05-08T05:51:38.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.829+0000 7f92b6000700 1 -- 172.21.15.112:0/2854992544 learned_addr learned my addr 172.21.15.112:0/2854992544 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:51:38.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.829+0000 7f92bea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c0103620 0x7f92c01a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.829+0000 7f92bf400700 1 --2- 172.21.15.112:0/2854992544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92c0108700 0x7f92c019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.830+0000 7f92b6000700 1 -- 172.21.15.112:0/2854992544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c0103620 msgr2=0x7f92c01a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:38.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.830+0000 7f92b6000700 1 --2- 172.21.15.112:0/2854992544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c0103620 0x7f92c01a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.830+0000 7f92b6000700 1 -- 172.21.15.112:0/2854992544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92c0108700 msgr2=0x7f92c019bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:38.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.830+0000 7f92b6000700 1 --2- 172.21.15.112:0/2854992544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92c0108700 0x7f92c019bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.830+0000 7f92b6000700 1 -- 172.21.15.112:0/2854992544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c019c400 con 0x7f92c0103fd0 2024-05-08T05:51:38.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.830+0000 7f92bf400700 1 --2- 172.21.15.112:0/2854992544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92c0108700 0x7f92c019bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:38.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.830+0000 7f92b6000700 1 --2- 172.21.15.112:0/2854992544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c0103fd0 0x7f92c01a1f90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f92a800e970 tx=0x7f92a800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:38.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.830+0000 7f92b7e00700 1 -- 172.21.15.112:0/2854992544 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8004a30 con 0x7f92c0103fd0 2024-05-08T05:51:38.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.830+0000 7f92bfe00700 1 -- 172.21.15.112:0/2854992544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c019c6f0 con 0x7f92c0103fd0 2024-05-08T05:51:38.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.830+0000 7f92b7e00700 1 -- 172.21.15.112:0/2854992544 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92a800cc90 con 0x7f92c0103fd0 2024-05-08T05:51:38.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.830+0000 7f92b7e00700 1 -- 172.21.15.112:0/2854992544 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8005610 con 0x7f92c0103fd0 2024-05-08T05:51:38.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.830+0000 7f92bfe00700 1 -- 172.21.15.112:0/2854992544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c01a89d0 con 0x7f92c0103fd0 2024-05-08T05:51:38.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.831+0000 7f92bfe00700 1 -- 172.21.15.112:0/2854992544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c0104b00 con 0x7f92c0103fd0 2024-05-08T05:51:38.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.834+0000 7f92b7e00700 1 -- 172.21.15.112:0/2854992544 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f92a800c7e0 con 0x7f92c0103fd0 2024-05-08T05:51:38.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.835+0000 7f92b7e00700 1 --2- 172.21.15.112:0/2854992544 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f92a40b2910 0x7f92a40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.836+0000 7f92b7e00700 1 -- 172.21.15.112:0/2854992544 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f92a80e26d0 con 0x7f92c0103fd0 2024-05-08T05:51:38.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.836+0000 7f92bea00700 1 --2- 172.21.15.112:0/2854992544 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f92a40b2910 0x7f92a40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.836+0000 7f92bea00700 1 --2- 172.21.15.112:0/2854992544 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f92a40b2910 0x7f92a40b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f92b000b790 tx=0x7f92b000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:38.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:38.838+0000 7f92b7e00700 1 -- 172.21.15.112:0/2854992544 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f92a8054df0 con 0x7f92c0103fd0 2024-05-08T05:51:38.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:38 smithi150 ceph-mon[34315]: pgmap v327: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-08T05:51:39.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:38 smithi112 ceph-mon[30003]: pgmap v327: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-08T05:51:39.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:38 smithi162 ceph-mon[34800]: pgmap v327: 97 pgs: 97 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-08T05:51:39.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:39.104+0000 7f92bfe00700 1 -- 172.21.15.112:0/2854992544 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f92c0076f70 con 0x7f92a40b2910 2024-05-08T05:51:39.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:39.151+0000 7f92b7e00700 1 -- 172.21.15.112:0/2854992544 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f92c0076f70 con 0x7f92a40b2910 2024-05-08T05:51:39.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:39.154+0000 7f92bfe00700 1 -- 172.21.15.112:0/2854992544 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f92a40b2910 msgr2=0x7f92a40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:39.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:39.154+0000 7f92bfe00700 1 --2- 172.21.15.112:0/2854992544 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f92a40b2910 0x7f92a40b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f92b000b790 tx=0x7f92b000b260 comp rx=0 tx=0).stop 2024-05-08T05:51:39.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:39.154+0000 7f92bfe00700 1 -- 172.21.15.112:0/2854992544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c0103fd0 msgr2=0x7f92c01a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:39.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:39.154+0000 7f92bfe00700 1 --2- 172.21.15.112:0/2854992544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c0103fd0 0x7f92c01a1f90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f92a800e970 tx=0x7f92a800ec80 comp rx=0 tx=0).stop 2024-05-08T05:51:39.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:39.154+0000 7f92bfe00700 1 -- 172.21.15.112:0/2854992544 shutdown_connections 2024-05-08T05:51:39.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:39.154+0000 7f92bfe00700 1 --2- 172.21.15.112:0/2854992544 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f92a40b2910 0x7f92a40b4dd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:39.154+0000 7f92bfe00700 1 --2- 172.21.15.112:0/2854992544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92c0108700 0x7f92c019bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:39.154+0000 7f92bfe00700 1 --2- 172.21.15.112:0/2854992544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c0103620 0x7f92c01a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:39.154+0000 7f92bfe00700 1 --2- 172.21.15.112:0/2854992544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92c0103fd0 0x7f92c01a1f90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:39.154+0000 7f92bfe00700 1 -- 172.21.15.112:0/2854992544 >> 172.21.15.112:0/2854992544 conn(0x7f92c00feeb0 msgr2=0x7f92c010f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:39.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:39.155+0000 7f92bfe00700 1 -- 172.21.15.112:0/2854992544 shutdown_connections 2024-05-08T05:51:39.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:39.155+0000 7f92bfe00700 1 -- 172.21.15.112:0/2854992544 wait complete. 2024-05-08T05:51:39.956 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:39 smithi112 ceph-mon[30003]: from='client.14493 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:40.005 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-08T05:51:40.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:39 smithi162 ceph-mon[34800]: from='client.14493 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:40.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:39 smithi150 ceph-mon[34315]: from='client.14493 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:40.451 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:40.960 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:40 smithi112 ceph-mon[30003]: pgmap v328: 97 pgs: 97 active+clean; 602 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-05-08T05:51:41.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:40 smithi162 ceph-mon[34800]: pgmap v328: 97 pgs: 97 active+clean; 602 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-05-08T05:51:41.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:40 smithi150 ceph-mon[34315]: pgmap v328: 97 pgs: 97 active+clean; 602 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-05-08T05:51:42.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.581+0000 7f4bfd800700 1 -- 172.21.15.112:0/2530443063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf8105f90 msgr2=0x7f4bf81124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.581+0000 7f4bfd800700 1 --2- 172.21.15.112:0/2530443063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf8105f90 0x7f4bf81124a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000b0a0 tx=0x7f4bf000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.581+0000 7f4bfd800700 1 -- 172.21.15.112:0/2530443063 shutdown_connections 2024-05-08T05:51:42.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.581+0000 7f4bfd800700 1 --2- 172.21.15.112:0/2530443063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf8105f90 0x7f4bf81124a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.581+0000 7f4bfd800700 1 --2- 172.21.15.112:0/2530443063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf810b660 0x7f4bf810ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.581+0000 7f4bfd800700 1 --2- 172.21.15.112:0/2530443063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bf81055d0 0x7f4bf8105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.581+0000 7f4bfd800700 1 -- 172.21.15.112:0/2530443063 >> 172.21.15.112:0/2530443063 conn(0x7f4bf81010c0 msgr2=0x7f4bf81034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:42.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.581+0000 7f4bfd800700 1 -- 172.21.15.112:0/2530443063 shutdown_connections 2024-05-08T05:51:42.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.581+0000 7f4bfd800700 1 -- 172.21.15.112:0/2530443063 wait complete. 2024-05-08T05:51:42.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.582+0000 7f4bfd800700 1 Processor -- start 2024-05-08T05:51:42.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.583+0000 7f4bfd800700 1 -- start start 2024-05-08T05:51:42.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.583+0000 7f4bfd800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bf81055d0 0x7f4bf81a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:42.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.583+0000 7f4bfd800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf8105f90 0x7f4bf81a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:42.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.583+0000 7f4bfd800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf810b660 0x7f4bf819b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:42.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.583+0000 7f4bfd800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf81157d0 con 0x7f4bf810b660 2024-05-08T05:51:42.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.583+0000 7f4bfd800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf8115650 con 0x7f4bf8105f90 2024-05-08T05:51:42.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.583+0000 7f4bfd800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf8115950 con 0x7f4bf81055d0 2024-05-08T05:51:42.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.583+0000 7f4bfce00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf810b660 0x7f4bf819b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:42.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.583+0000 7f4bf7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bf81055d0 0x7f4bf81a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:42.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.583+0000 7f4bfce00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf810b660 0x7f4bf819b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36598/0 (socket says 172.21.15.112:36598) 2024-05-08T05:51:42.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.583+0000 7f4bfce00700 1 -- 172.21.15.112:0/2964965897 learned_addr learned my addr 172.21.15.112:0/2964965897 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:51:42.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.584+0000 7f4bf7400700 1 --2- 172.21.15.112:0/2964965897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf8105f90 0x7f4bf81a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:42.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.584+0000 7f4bfce00700 1 -- 172.21.15.112:0/2964965897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bf81055d0 msgr2=0x7f4bf81a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.584+0000 7f4bfce00700 1 --2- 172.21.15.112:0/2964965897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bf81055d0 0x7f4bf81a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.584+0000 7f4bfce00700 1 -- 172.21.15.112:0/2964965897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf8105f90 msgr2=0x7f4bf81a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.584+0000 7f4bfce00700 1 --2- 172.21.15.112:0/2964965897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf8105f90 0x7f4bf81a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.584+0000 7f4bfce00700 1 -- 172.21.15.112:0/2964965897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf819c150 con 0x7f4bf810b660 2024-05-08T05:51:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.584+0000 7f4bf7e00700 1 --2- 172.21.15.112:0/2964965897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bf81055d0 0x7f4bf81a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.584+0000 7f4bfce00700 1 --2- 172.21.15.112:0/2964965897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf810b660 0x7f4bf819b9f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00062a0 tx=0x7f4bf0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.584+0000 7f4bf4c00700 1 -- 172.21.15.112:0/2964965897 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0021070 con 0x7f4bf810b660 2024-05-08T05:51:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.585+0000 7f4bf4c00700 1 -- 172.21.15.112:0/2964965897 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bf0007e30 con 0x7f4bf810b660 2024-05-08T05:51:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.585+0000 7f4bfd800700 1 -- 172.21.15.112:0/2964965897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf819c3e0 con 0x7f4bf810b660 2024-05-08T05:51:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.585+0000 7f4bf4c00700 1 -- 172.21.15.112:0/2964965897 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf00046d0 con 0x7f4bf810b660 2024-05-08T05:51:42.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.585+0000 7f4bfd800700 1 -- 172.21.15.112:0/2964965897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf81a86b0 con 0x7f4bf810b660 2024-05-08T05:51:42.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.586+0000 7f4bfd800700 1 -- 172.21.15.112:0/2964965897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf806a190 con 0x7f4bf810b660 2024-05-08T05:51:42.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.588+0000 7f4bf4c00700 1 -- 172.21.15.112:0/2964965897 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f4bf0007980 con 0x7f4bf810b660 2024-05-08T05:51:42.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.589+0000 7f4bf4c00700 1 --2- 172.21.15.112:0/2964965897 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4bdc0b2910 0x7f4bdc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:42.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.590+0000 7f4bf4c00700 1 -- 172.21.15.112:0/2964965897 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f4bf000f020 con 0x7f4bf810b660 2024-05-08T05:51:42.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.590+0000 7f4bf7e00700 1 --2- 172.21.15.112:0/2964965897 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4bdc0b2910 0x7f4bdc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:42.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.591+0000 7f4bf7e00700 1 --2- 172.21.15.112:0/2964965897 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4bdc0b2910 0x7f4bdc0b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4bf810a6c0 tx=0x7f4be8007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:42.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.594+0000 7f4bf4c00700 1 -- 172.21.15.112:0/2964965897 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4bf002a020 con 0x7f4bf810b660 2024-05-08T05:51:42.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.869+0000 7f4bfd800700 1 -- 172.21.15.112:0/2964965897 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4bf80638e0 con 0x7f4bdc0b2910 2024-05-08T05:51:42.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.880+0000 7f4bf4c00700 1 -- 172.21.15.112:0/2964965897 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4bf80638e0 con 0x7f4bdc0b2910 2024-05-08T05:51:42.881 INFO:teuthology.orchestra.run.smithi112.stdout:/volumes/qa/sv_0/3c451e6f-1cdb-41d0-8e23-fdc5c7332b52 2024-05-08T05:51:42.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.884+0000 7f4bfd800700 1 -- 172.21.15.112:0/2964965897 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4bdc0b2910 msgr2=0x7f4bdc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.884+0000 7f4bfd800700 1 --2- 172.21.15.112:0/2964965897 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4bdc0b2910 0x7f4bdc0b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4bf810a6c0 tx=0x7f4be8007400 comp rx=0 tx=0).stop 2024-05-08T05:51:42.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.884+0000 7f4bfd800700 1 -- 172.21.15.112:0/2964965897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf810b660 msgr2=0x7f4bf819b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.884+0000 7f4bfd800700 1 --2- 172.21.15.112:0/2964965897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf810b660 0x7f4bf819b9f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00062a0 tx=0x7f4bf0004c80 comp rx=0 tx=0).stop 2024-05-08T05:51:42.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.884+0000 7f4bfd800700 1 -- 172.21.15.112:0/2964965897 shutdown_connections 2024-05-08T05:51:42.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.884+0000 7f4bfd800700 1 --2- 172.21.15.112:0/2964965897 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4bdc0b2910 0x7f4bdc0b4dd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.884+0000 7f4bfd800700 1 --2- 172.21.15.112:0/2964965897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bf810b660 0x7f4bf819b9f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.884+0000 7f4bfd800700 1 --2- 172.21.15.112:0/2964965897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bf81055d0 0x7f4bf81a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.884+0000 7f4bfd800700 1 --2- 172.21.15.112:0/2964965897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf8105f90 0x7f4bf81a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.884+0000 7f4bfd800700 1 -- 172.21.15.112:0/2964965897 >> 172.21.15.112:0/2964965897 conn(0x7f4bf81010c0 msgr2=0x7f4bf8102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:42.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.885+0000 7f4bfd800700 1 -- 172.21.15.112:0/2964965897 shutdown_connections 2024-05-08T05:51:42.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:42.885+0000 7f4bfd800700 1 -- 172.21.15.112:0/2964965897 wait complete. 2024-05-08T05:51:43.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:42 smithi112 ceph-mon[30003]: pgmap v329: 97 pgs: 97 active+clean; 602 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-08T05:51:43.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:42 smithi162 ceph-mon[34800]: pgmap v329: 97 pgs: 97 active+clean; 602 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-08T05:51:43.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:42 smithi150 ceph-mon[34315]: pgmap v329: 97 pgs: 97 active+clean; 602 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-08T05:51:43.692 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-08T05:51:44.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:43 smithi112 ceph-mon[30003]: from='client.14499 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:44.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:43 smithi162 ceph-mon[34800]: from='client.14499 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:44.146 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:44.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:43 smithi150 ceph-mon[34315]: from='client.14499 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:45.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:44 smithi112 ceph-mon[30003]: pgmap v330: 97 pgs: 97 active+clean; 602 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-05-08T05:51:45.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:44 smithi162 ceph-mon[34800]: pgmap v330: 97 pgs: 97 active+clean; 602 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-05-08T05:51:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:44 smithi150 ceph-mon[34315]: pgmap v330: 97 pgs: 97 active+clean; 602 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-05-08T05:51:46.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.400+0000 7fc5ba000700 1 -- 172.21.15.112:0/699943282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109510 msgr2=0x7fc5b41098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.400+0000 7fc5ba000700 1 --2- 172.21.15.112:0/699943282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109510 0x7fc5b41098f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009990 tx=0x7fc5a4009ca0 comp rx=0 tx=0).stop 2024-05-08T05:51:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.400+0000 7fc5ba000700 1 -- 172.21.15.112:0/699943282 shutdown_connections 2024-05-08T05:51:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.400+0000 7fc5ba000700 1 --2- 172.21.15.112:0/699943282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5b406bb40 0x7fc5b4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.400+0000 7fc5ba000700 1 --2- 172.21.15.112:0/699943282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b4109510 0x7fc5b41098f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.400+0000 7fc5ba000700 1 --2- 172.21.15.112:0/699943282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5b406b180 0x7fc5b406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.401+0000 7fc5ba000700 1 -- 172.21.15.112:0/699943282 >> 172.21.15.112:0/699943282 conn(0x7fc5b4079350 msgr2=0x7fc5b4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.401+0000 7fc5ba000700 1 -- 172.21.15.112:0/699943282 shutdown_connections 2024-05-08T05:51:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.401+0000 7fc5ba000700 1 -- 172.21.15.112:0/699943282 wait complete. 2024-05-08T05:51:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.402+0000 7fc5ba000700 1 Processor -- start 2024-05-08T05:51:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.402+0000 7fc5ba000700 1 -- start start 2024-05-08T05:51:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.402+0000 7fc5ba000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5b406b180 0x7fc5b419be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.402+0000 7fc5ba000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b406bb40 0x7fc5b419c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.402+0000 7fc5ba000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5b4109510 0x7fc5b41a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.402+0000 7fc5ba000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b4113640 con 0x7fc5b406bb40 2024-05-08T05:51:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.402+0000 7fc5ba000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41134c0 con 0x7fc5b4109510 2024-05-08T05:51:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.402+0000 7fc5ba000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41137c0 con 0x7fc5b406b180 2024-05-08T05:51:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.403+0000 7fc5b2a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b406bb40 0x7fc5b419c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.403+0000 7fc5b3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5b4109510 0x7fc5b41a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.403+0000 7fc5b3400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5b406b180 0x7fc5b419be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.403+0000 7fc5b3400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5b406b180 0x7fc5b419be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:58698/0 (socket says 172.21.15.112:58698) 2024-05-08T05:51:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.403+0000 7fc5b3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5b4109510 0x7fc5b41a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:37272/0 (socket says 172.21.15.112:37272) 2024-05-08T05:51:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.403+0000 7fc5b3400700 1 -- 172.21.15.112:0/2657737775 learned_addr learned my addr 172.21.15.112:0/2657737775 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:51:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.403+0000 7fc5b3e00700 1 -- 172.21.15.112:0/2657737775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5b406b180 msgr2=0x7fc5b419be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.403+0000 7fc5b3e00700 1 --2- 172.21.15.112:0/2657737775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5b406b180 0x7fc5b419be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.403+0000 7fc5b3e00700 1 -- 172.21.15.112:0/2657737775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b406bb40 msgr2=0x7fc5b419c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.403+0000 7fc5b3e00700 1 --2- 172.21.15.112:0/2657737775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b406bb40 0x7fc5b419c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.403+0000 7fc5b3e00700 1 -- 172.21.15.112:0/2657737775 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b41a0e80 con 0x7fc5b4109510 2024-05-08T05:51:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.404+0000 7fc5b3e00700 1 --2- 172.21.15.112:0/2657737775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5b4109510 0x7fc5b41a0740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00efb0 tx=0x7fc5ac00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:46.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.404+0000 7fc5abe00700 1 -- 172.21.15.112:0/2657737775 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac003a00 con 0x7fc5b4109510 2024-05-08T05:51:46.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.404+0000 7fc5abe00700 1 -- 172.21.15.112:0/2657737775 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5ac00ce50 con 0x7fc5b4109510 2024-05-08T05:51:46.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.404+0000 7fc5ba000700 1 -- 172.21.15.112:0/2657737775 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b41a1170 con 0x7fc5b4109510 2024-05-08T05:51:46.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.404+0000 7fc5abe00700 1 -- 172.21.15.112:0/2657737775 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac0047c0 con 0x7fc5b4109510 2024-05-08T05:51:46.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.404+0000 7fc5ba000700 1 -- 172.21.15.112:0/2657737775 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b41a89f0 con 0x7fc5b4109510 2024-05-08T05:51:46.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.405+0000 7fc5ba000700 1 -- 172.21.15.112:0/2657737775 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b406a190 con 0x7fc5b4109510 2024-05-08T05:51:46.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.407+0000 7fc5abe00700 1 -- 172.21.15.112:0/2657737775 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc5ac012050 con 0x7fc5b4109510 2024-05-08T05:51:46.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.409+0000 7fc5abe00700 1 --2- 172.21.15.112:0/2657737775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc5980b2910 0x7fc5980b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:46.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.409+0000 7fc5abe00700 1 -- 172.21.15.112:0/2657737775 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc5ac0e1d50 con 0x7fc5b4109510 2024-05-08T05:51:46.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.409+0000 7fc5b3400700 1 --2- 172.21.15.112:0/2657737775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc5980b2910 0x7fc5980b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:46.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.410+0000 7fc5b3400700 1 --2- 172.21.15.112:0/2657737775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc5980b2910 0x7fc5980b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4005cd0 tx=0x7fc5a4005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:46.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.411+0000 7fc5abe00700 1 -- 172.21.15.112:0/2657737775 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc5ac022060 con 0x7fc5b4109510 2024-05-08T05:51:46.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.677+0000 7fc5ba000700 1 -- 172.21.15.112:0/2657737775 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc5b410ed90 con 0x7fc5980b2910 2024-05-08T05:51:46.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.737+0000 7fc5abe00700 1 -- 172.21.15.112:0/2657737775 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc5b410ed90 con 0x7fc5980b2910 2024-05-08T05:51:46.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.740+0000 7fc5ba000700 1 -- 172.21.15.112:0/2657737775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc5980b2910 msgr2=0x7fc5980b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:46.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.740+0000 7fc5ba000700 1 --2- 172.21.15.112:0/2657737775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc5980b2910 0x7fc5980b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4005cd0 tx=0x7fc5a4005c60 comp rx=0 tx=0).stop 2024-05-08T05:51:46.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.740+0000 7fc5ba000700 1 -- 172.21.15.112:0/2657737775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5b4109510 msgr2=0x7fc5b41a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:46.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.740+0000 7fc5ba000700 1 --2- 172.21.15.112:0/2657737775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5b4109510 0x7fc5b41a0740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00efb0 tx=0x7fc5ac00c3a0 comp rx=0 tx=0).stop 2024-05-08T05:51:46.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.741+0000 7fc5ba000700 1 -- 172.21.15.112:0/2657737775 shutdown_connections 2024-05-08T05:51:46.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.741+0000 7fc5ba000700 1 --2- 172.21.15.112:0/2657737775 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc5980b2910 0x7fc5980b4dd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:46.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.741+0000 7fc5ba000700 1 --2- 172.21.15.112:0/2657737775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5b4109510 0x7fc5b41a0740 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:46.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.741+0000 7fc5ba000700 1 --2- 172.21.15.112:0/2657737775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5b406b180 0x7fc5b419be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:46.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.741+0000 7fc5ba000700 1 --2- 172.21.15.112:0/2657737775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5b406bb40 0x7fc5b419c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:46.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.741+0000 7fc5ba000700 1 -- 172.21.15.112:0/2657737775 >> 172.21.15.112:0/2657737775 conn(0x7fc5b4079350 msgr2=0x7fc5b410e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:46.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.741+0000 7fc5ba000700 1 -- 172.21.15.112:0/2657737775 shutdown_connections 2024-05-08T05:51:46.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:46.741+0000 7fc5ba000700 1 -- 172.21.15.112:0/2657737775 wait complete. 2024-05-08T05:51:47.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:46 smithi112 ceph-mon[30003]: pgmap v331: 97 pgs: 97 active+clean; 608 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-05-08T05:51:47.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:46 smithi162 ceph-mon[34800]: pgmap v331: 97 pgs: 97 active+clean; 608 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-05-08T05:51:47.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:46 smithi150 ceph-mon[34315]: pgmap v331: 97 pgs: 97 active+clean; 608 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-05-08T05:51:47.581 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-08T05:51:47.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:47 smithi112 ceph-mon[30003]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:48.029 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:48.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:47 smithi162 ceph-mon[34800]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:48.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:47 smithi150 ceph-mon[34315]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:49.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:48 smithi112 ceph-mon[30003]: pgmap v332: 97 pgs: 97 active+clean; 608 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-05-08T05:51:49.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:48 smithi162 ceph-mon[34800]: pgmap v332: 97 pgs: 97 active+clean; 608 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-05-08T05:51:49.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:48 smithi150 ceph-mon[34315]: pgmap v332: 97 pgs: 97 active+clean; 608 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-05-08T05:51:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.361+0000 7f4e2fc00700 1 -- 172.21.15.112:0/328800041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e28108410 msgr2=0x7f4e281087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.361+0000 7f4e2fc00700 1 --2- 172.21.15.112:0/328800041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e28108410 0x7f4e281087f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0098f0 tx=0x7f4e1c009c00 comp rx=0 tx=0).stop 2024-05-08T05:51:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.361+0000 7f4e2fc00700 1 -- 172.21.15.112:0/328800041 shutdown_connections 2024-05-08T05:51:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.361+0000 7f4e2fc00700 1 --2- 172.21.15.112:0/328800041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e28102d60 0x7f4e2810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.361+0000 7f4e2fc00700 1 --2- 172.21.15.112:0/328800041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e28108410 0x7f4e281087f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.361+0000 7f4e2fc00700 1 --2- 172.21.15.112:0/328800041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e281023a0 0x7f4e28102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.361+0000 7f4e2fc00700 1 -- 172.21.15.112:0/328800041 >> 172.21.15.112:0/328800041 conn(0x7f4e28079350 msgr2=0x7f4e28079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.362+0000 7f4e2fc00700 1 -- 172.21.15.112:0/328800041 shutdown_connections 2024-05-08T05:51:50.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.362+0000 7f4e2fc00700 1 -- 172.21.15.112:0/328800041 wait complete. 2024-05-08T05:51:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.363+0000 7f4e2fc00700 1 Processor -- start 2024-05-08T05:51:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.363+0000 7f4e2fc00700 1 -- start start 2024-05-08T05:51:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.363+0000 7f4e2fc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e281023a0 0x7f4e2819bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.363+0000 7f4e2fc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e28102d60 0x7f4e2819c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.364+0000 7f4e2fc00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e28108410 0x7f4e281a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.364+0000 7f4e2fc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e28112560 con 0x7f4e28102d60 2024-05-08T05:51:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.364+0000 7f4e2fc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281123e0 con 0x7f4e281023a0 2024-05-08T05:51:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.364+0000 7f4e2fc00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281126e0 con 0x7f4e28108410 2024-05-08T05:51:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.364+0000 7f4e27e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e28102d60 0x7f4e2819c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.364+0000 7f4e2d200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e281023a0 0x7f4e2819bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.364+0000 7f4e27e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e28102d60 0x7f4e2819c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42554/0 (socket says 172.21.15.112:42554) 2024-05-08T05:51:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.364+0000 7f4e27e00700 1 -- 172.21.15.112:0/3020080328 learned_addr learned my addr 172.21.15.112:0/3020080328 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:51:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.364+0000 7f4e2d200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e281023a0 0x7f4e2819bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:37290/0 (socket says 172.21.15.112:37290) 2024-05-08T05:51:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.364+0000 7f4e2dc00700 1 --2- 172.21.15.112:0/3020080328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e28108410 0x7f4e281a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.364+0000 7f4e2d200700 1 -- 172.21.15.112:0/3020080328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e28108410 msgr2=0x7f4e281a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.364+0000 7f4e2d200700 1 --2- 172.21.15.112:0/3020080328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e28108410 0x7f4e281a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.364+0000 7f4e2d200700 1 -- 172.21.15.112:0/3020080328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e28102d60 msgr2=0x7f4e2819c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.364+0000 7f4e2d200700 1 --2- 172.21.15.112:0/3020080328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e28102d60 0x7f4e2819c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.364+0000 7f4e2d200700 1 -- 172.21.15.112:0/3020080328 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e281a0dd0 con 0x7f4e281023a0 2024-05-08T05:51:50.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.365+0000 7f4e2d200700 1 --2- 172.21.15.112:0/3020080328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e281023a0 0x7f4e2819bdc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009230 tx=0x7f4e1c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.365+0000 7f4e25600700 1 -- 172.21.15.112:0/3020080328 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c005040 con 0x7f4e281023a0 2024-05-08T05:51:50.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.365+0000 7f4e25600700 1 -- 172.21.15.112:0/3020080328 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e1c005690 con 0x7f4e281023a0 2024-05-08T05:51:50.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.365+0000 7f4e2fc00700 1 -- 172.21.15.112:0/3020080328 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e281a1060 con 0x7f4e281023a0 2024-05-08T05:51:50.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.365+0000 7f4e25600700 1 -- 172.21.15.112:0/3020080328 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0038e0 con 0x7f4e281023a0 2024-05-08T05:51:50.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.365+0000 7f4e2fc00700 1 -- 172.21.15.112:0/3020080328 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e281a8940 con 0x7f4e281023a0 2024-05-08T05:51:50.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.366+0000 7f4e2fc00700 1 -- 172.21.15.112:0/3020080328 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e28103ac0 con 0x7f4e281023a0 2024-05-08T05:51:50.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.368+0000 7f4e25600700 1 -- 172.21.15.112:0/3020080328 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f4e1c00b760 con 0x7f4e281023a0 2024-05-08T05:51:50.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.370+0000 7f4e25600700 1 --2- 172.21.15.112:0/3020080328 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4e100b2910 0x7f4e100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.370+0000 7f4e25600700 1 -- 172.21.15.112:0/3020080328 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f4e1c0e30e0 con 0x7f4e281023a0 2024-05-08T05:51:50.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.370+0000 7f4e27e00700 1 --2- 172.21.15.112:0/3020080328 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4e100b2910 0x7f4e100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.371+0000 7f4e27e00700 1 --2- 172.21.15.112:0/3020080328 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4e100b2910 0x7f4e100b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4e28079770 tx=0x7f4e14009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.374+0000 7f4e25600700 1 -- 172.21.15.112:0/3020080328 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4e1c013030 con 0x7f4e281023a0 2024-05-08T05:51:50.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.642+0000 7f4e2fc00700 1 -- 172.21.15.112:0/3020080328 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f4e2806b8f0 con 0x7f4e100b2910 2024-05-08T05:51:50.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.651+0000 7f4e25600700 1 -- 172.21.15.112:0/3020080328 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4e2806b8f0 con 0x7f4e100b2910 2024-05-08T05:51:50.652 INFO:teuthology.orchestra.run.smithi112.stdout:/volumes/qa/sv_1/f10a0d2e-83c8-40e2-b411-c0dc32989241 2024-05-08T05:51:50.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.654+0000 7f4e2fc00700 1 -- 172.21.15.112:0/3020080328 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4e100b2910 msgr2=0x7f4e100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.654+0000 7f4e2fc00700 1 --2- 172.21.15.112:0/3020080328 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4e100b2910 0x7f4e100b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4e28079770 tx=0x7f4e14009310 comp rx=0 tx=0).stop 2024-05-08T05:51:50.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.654+0000 7f4e2fc00700 1 -- 172.21.15.112:0/3020080328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e281023a0 msgr2=0x7f4e2819bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.654+0000 7f4e2fc00700 1 --2- 172.21.15.112:0/3020080328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e281023a0 0x7f4e2819bdc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009230 tx=0x7f4e1c00fa00 comp rx=0 tx=0).stop 2024-05-08T05:51:50.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.655+0000 7f4e2fc00700 1 -- 172.21.15.112:0/3020080328 shutdown_connections 2024-05-08T05:51:50.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.655+0000 7f4e2fc00700 1 --2- 172.21.15.112:0/3020080328 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4e100b2910 0x7f4e100b4dd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.655+0000 7f4e2fc00700 1 --2- 172.21.15.112:0/3020080328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e28108410 0x7f4e281a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.655+0000 7f4e2fc00700 1 --2- 172.21.15.112:0/3020080328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e281023a0 0x7f4e2819bdc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.655+0000 7f4e2fc00700 1 --2- 172.21.15.112:0/3020080328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e28102d60 0x7f4e2819c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.655+0000 7f4e2fc00700 1 -- 172.21.15.112:0/3020080328 >> 172.21.15.112:0/3020080328 conn(0x7f4e28079350 msgr2=0x7f4e2806b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:50.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.655+0000 7f4e2fc00700 1 -- 172.21.15.112:0/3020080328 shutdown_connections 2024-05-08T05:51:50.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:50.655+0000 7f4e2fc00700 1 -- 172.21.15.112:0/3020080328 wait complete. 2024-05-08T05:51:51.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:50 smithi112 ceph-mon[30003]: pgmap v333: 97 pgs: 97 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-05-08T05:51:51.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:50 smithi162 ceph-mon[34800]: pgmap v333: 97 pgs: 97 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-05-08T05:51:51.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:50 smithi150 ceph-mon[34315]: pgmap v333: 97 pgs: 97 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-05-08T05:51:51.580 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:51:51.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:51 smithi112 ceph-mon[30003]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:52.019 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:52.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:51 smithi162 ceph-mon[34800]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:52.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:51 smithi150 ceph-mon[34315]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:52.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:52 smithi112 ceph-mon[30003]: pgmap v334: 97 pgs: 97 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-08T05:51:53.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:52 smithi162 ceph-mon[34800]: pgmap v334: 97 pgs: 97 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-08T05:51:53.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:52 smithi150 ceph-mon[34315]: pgmap v334: 97 pgs: 97 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-08T05:51:54.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.172+0000 7f1852200700 1 -- 172.21.15.112:0/2050243110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f184c109ad0 msgr2=0x7f184c109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:54.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.172+0000 7f1852200700 1 --2- 172.21.15.112:0/2050243110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f184c109ad0 0x7f184c109f50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f18380098f0 tx=0x7f1838009c00 comp rx=0 tx=0).stop 2024-05-08T05:51:54.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.173+0000 7f1852200700 1 -- 172.21.15.112:0/2050243110 shutdown_connections 2024-05-08T05:51:54.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.173+0000 7f1852200700 1 --2- 172.21.15.112:0/2050243110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f184c10a490 0x7f184c1169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:54.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.173+0000 7f1852200700 1 --2- 172.21.15.112:0/2050243110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f184c10fb60 0x7f184c10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:54.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.173+0000 7f1852200700 1 --2- 172.21.15.112:0/2050243110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f184c109ad0 0x7f184c109f50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:54.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.173+0000 7f1852200700 1 -- 172.21.15.112:0/2050243110 >> 172.21.15.112:0/2050243110 conn(0x7f184c076c70 msgr2=0x7f184c1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:54.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.173+0000 7f1852200700 1 -- 172.21.15.112:0/2050243110 shutdown_connections 2024-05-08T05:51:54.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.173+0000 7f1852200700 1 -- 172.21.15.112:0/2050243110 wait complete. 2024-05-08T05:51:54.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.175+0000 7f1852200700 1 Processor -- start 2024-05-08T05:51:54.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.175+0000 7f1852200700 1 -- start start 2024-05-08T05:51:54.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.175+0000 7f1852200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f184c109ad0 0x7f184c1a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:54.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.175+0000 7f1852200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f184c10a490 0x7f184c1a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:54.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.175+0000 7f1852200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f184c10fb60 0x7f184c1a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:54.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.175+0000 7f1852200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f184c119d00 con 0x7f184c109ad0 2024-05-08T05:51:54.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.175+0000 7f1852200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f184c119b80 con 0x7f184c10a490 2024-05-08T05:51:54.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.175+0000 7f1852200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f184c119e80 con 0x7f184c10fb60 2024-05-08T05:51:54.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.176+0000 7f184b400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f184c109ad0 0x7f184c1a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:54.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.176+0000 7f184aa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f184c10a490 0x7f184c1a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:54.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.176+0000 7f184aa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f184c10a490 0x7f184c1a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:41278/0 (socket says 172.21.15.112:41278) 2024-05-08T05:51:54.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.176+0000 7f184aa00700 1 -- 172.21.15.112:0/902930325 learned_addr learned my addr 172.21.15.112:0/902930325 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:51:54.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.176+0000 7f184be00700 1 --2- 172.21.15.112:0/902930325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f184c10fb60 0x7f184c1a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:54.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.176+0000 7f184aa00700 1 -- 172.21.15.112:0/902930325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f184c10fb60 msgr2=0x7f184c1a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:54.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.176+0000 7f184aa00700 1 --2- 172.21.15.112:0/902930325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f184c10fb60 0x7f184c1a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:54.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.176+0000 7f184aa00700 1 -- 172.21.15.112:0/902930325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f184c109ad0 msgr2=0x7f184c1a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:54.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.176+0000 7f184aa00700 1 --2- 172.21.15.112:0/902930325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f184c109ad0 0x7f184c1a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:54.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.176+0000 7f184aa00700 1 -- 172.21.15.112:0/902930325 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c1a5230 con 0x7f184c10a490 2024-05-08T05:51:54.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.177+0000 7f184aa00700 1 --2- 172.21.15.112:0/902930325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f184c10a490 0x7f184c1a07b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1838009230 tx=0x7f183800fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:54.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.177+0000 7f1843e00700 1 -- 172.21.15.112:0/902930325 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18380050c0 con 0x7f184c10a490 2024-05-08T05:51:54.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.177+0000 7f1843e00700 1 -- 172.21.15.112:0/902930325 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f183800b560 con 0x7f184c10a490 2024-05-08T05:51:54.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.177+0000 7f1852200700 1 -- 172.21.15.112:0/902930325 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184c1a54c0 con 0x7f184c10a490 2024-05-08T05:51:54.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.177+0000 7f1843e00700 1 -- 172.21.15.112:0/902930325 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838003a00 con 0x7f184c10a490 2024-05-08T05:51:54.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.177+0000 7f1852200700 1 -- 172.21.15.112:0/902930325 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184c1acda0 con 0x7f184c10a490 2024-05-08T05:51:54.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.178+0000 7f1852200700 1 -- 172.21.15.112:0/902930325 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f184c06a190 con 0x7f184c10a490 2024-05-08T05:51:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.180+0000 7f1843e00700 1 -- 172.21.15.112:0/902930325 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f183800b710 con 0x7f184c10a490 2024-05-08T05:51:54.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.182+0000 7f1843e00700 1 --2- 172.21.15.112:0/902930325 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f18300b2990 0x7f18300b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:54.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.182+0000 7f1843e00700 1 -- 172.21.15.112:0/902930325 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f18380e3140 con 0x7f184c10a490 2024-05-08T05:51:54.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.182+0000 7f184b400700 1 --2- 172.21.15.112:0/902930325 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f18300b2990 0x7f18300b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:54.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.183+0000 7f184b400700 1 --2- 172.21.15.112:0/902930325 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f18300b2990 0x7f18300b4e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f183c009be0 tx=0x7f183c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:54.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.186+0000 7f1843e00700 1 -- 172.21.15.112:0/902930325 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1838013030 con 0x7f184c10a490 2024-05-08T05:51:54.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.468+0000 7f1852200700 1 -- 172.21.15.112:0/902930325 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f184c1a1580 con 0x7f184c10a490 2024-05-08T05:51:54.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.469+0000 7f1843e00700 1 -- 172.21.15.112:0/902930325 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11891 (secure 0 0 0) 0x7f18380aae40 con 0x7f184c10a490 2024-05-08T05:51:54.470 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:51:54.470 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T05:51:31.368725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24511,24518]},"id":1}]} 2024-05-08T05:51:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.472+0000 7f1852200700 1 -- 172.21.15.112:0/902930325 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f18300b2990 msgr2=0x7f18300b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.472+0000 7f1852200700 1 --2- 172.21.15.112:0/902930325 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f18300b2990 0x7f18300b4e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f183c009be0 tx=0x7f183c009260 comp rx=0 tx=0).stop 2024-05-08T05:51:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.472+0000 7f1852200700 1 -- 172.21.15.112:0/902930325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f184c10a490 msgr2=0x7f184c1a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.472+0000 7f1852200700 1 --2- 172.21.15.112:0/902930325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f184c10a490 0x7f184c1a07b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1838009230 tx=0x7f183800fa70 comp rx=0 tx=0).stop 2024-05-08T05:51:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.472+0000 7f1852200700 1 -- 172.21.15.112:0/902930325 shutdown_connections 2024-05-08T05:51:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.472+0000 7f1852200700 1 --2- 172.21.15.112:0/902930325 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f18300b2990 0x7f18300b4e50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.472+0000 7f1852200700 1 --2- 172.21.15.112:0/902930325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f184c10fb60 0x7f184c1a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.472+0000 7f1852200700 1 --2- 172.21.15.112:0/902930325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f184c109ad0 0x7f184c1a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.472+0000 7f1852200700 1 --2- 172.21.15.112:0/902930325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f184c10a490 0x7f184c1a07b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.472+0000 7f1852200700 1 -- 172.21.15.112:0/902930325 >> 172.21.15.112:0/902930325 conn(0x7f184c076c70 msgr2=0x7f184c0772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.473+0000 7f1852200700 1 -- 172.21.15.112:0/902930325 shutdown_connections 2024-05-08T05:51:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:54.473+0000 7f1852200700 1 -- 172.21.15.112:0/902930325 wait complete. 2024-05-08T05:51:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 24 2024-05-08T05:51:55.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:55 smithi112 ceph-mon[30003]: pgmap v335: 97 pgs: 97 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-05-08T05:51:55.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:55 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/902930325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:51:55.220 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:51:55.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:55 smithi162 ceph-mon[34800]: pgmap v335: 97 pgs: 97 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-05-08T05:51:55.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:55 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/902930325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:51:55.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:55 smithi150 ceph-mon[34315]: pgmap v335: 97 pgs: 97 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-05-08T05:51:55.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:55 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/902930325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:51:55.689 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:56.161 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:56 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:51:56.298 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:56 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:51:56.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:56 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:51:57.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:57 smithi112 ceph-mon[30003]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-05-08T05:51:57.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:57 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:51:57.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:57 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:51:57.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:57 smithi162 ceph-mon[34800]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-05-08T05:51:57.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:57 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:51:57.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:57 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:51:57.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:57 smithi150 ceph-mon[34315]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-05-08T05:51:57.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:57 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:51:57.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:57 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:51:58.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.011+0000 7fac77800700 1 -- 172.21.15.112:0/2700861097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac70075ff0 msgr2=0x7fac7010f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:58.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.011+0000 7fac77800700 1 --2- 172.21.15.112:0/2700861097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac70075ff0 0x7fac7010f960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fac580098f0 tx=0x7fac58009c00 comp rx=0 tx=0).stop 2024-05-08T05:51:58.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.011+0000 7fac77800700 1 -- 172.21.15.112:0/2700861097 shutdown_connections 2024-05-08T05:51:58.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.011+0000 7fac77800700 1 --2- 172.21.15.112:0/2700861097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac7010fea0 0x7fac70112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:58.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.011+0000 7fac77800700 1 --2- 172.21.15.112:0/2700861097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac700756d0 0x7fac70075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:58.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.011+0000 7fac77800700 1 --2- 172.21.15.112:0/2700861097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac70075ff0 0x7fac7010f960 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:58.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.011+0000 7fac77800700 1 -- 172.21.15.112:0/2700861097 >> 172.21.15.112:0/2700861097 conn(0x7fac7007a600 msgr2=0x7fac7007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:58.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.011+0000 7fac77800700 1 -- 172.21.15.112:0/2700861097 shutdown_connections 2024-05-08T05:51:58.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.011+0000 7fac77800700 1 -- 172.21.15.112:0/2700861097 wait complete. 2024-05-08T05:51:58.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.012+0000 7fac77800700 1 Processor -- start 2024-05-08T05:51:58.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.013+0000 7fac77800700 1 -- start start 2024-05-08T05:51:58.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.013+0000 7fac77800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac700756d0 0x7fac7019bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:58.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.013+0000 7fac77800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac70075ff0 0x7fac7019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:58.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.013+0000 7fac77800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac7010fea0 0x7fac701a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:58.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.013+0000 7fac77800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac70115860 con 0x7fac700756d0 2024-05-08T05:51:58.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.013+0000 7fac77800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fac701156e0 con 0x7fac7010fea0 2024-05-08T05:51:58.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.013+0000 7fac77800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fac701159e0 con 0x7fac70075ff0 2024-05-08T05:51:58.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.013+0000 7fac67e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac70075ff0 0x7fac7019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:58.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.013+0000 7fac74e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac700756d0 0x7fac7019bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:58.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.013+0000 7fac67e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac70075ff0 0x7fac7019c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:53894/0 (socket says 172.21.15.112:53894) 2024-05-08T05:51:58.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.013+0000 7fac67e00700 1 -- 172.21.15.112:0/2439326521 learned_addr learned my addr 172.21.15.112:0/2439326521 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:51:58.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.014+0000 7fac75800700 1 --2- 172.21.15.112:0/2439326521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac7010fea0 0x7fac701a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:58.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.014+0000 7fac67e00700 1 -- 172.21.15.112:0/2439326521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac7010fea0 msgr2=0x7fac701a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:58.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.014+0000 7fac67e00700 1 --2- 172.21.15.112:0/2439326521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac7010fea0 0x7fac701a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:58.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.014+0000 7fac67e00700 1 -- 172.21.15.112:0/2439326521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac700756d0 msgr2=0x7fac7019bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:58.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.014+0000 7fac67e00700 1 --2- 172.21.15.112:0/2439326521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac700756d0 0x7fac7019bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:58.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.014+0000 7fac67e00700 1 -- 172.21.15.112:0/2439326521 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac701a0de0 con 0x7fac70075ff0 2024-05-08T05:51:58.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.014+0000 7fac67e00700 1 --2- 172.21.15.112:0/2439326521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac70075ff0 0x7fac7019c310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fac58009230 tx=0x7fac5800f990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:58.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.015+0000 7fac6e000700 1 -- 172.21.15.112:0/2439326521 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58005140 con 0x7fac70075ff0 2024-05-08T05:51:58.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.015+0000 7fac6e000700 1 -- 172.21.15.112:0/2439326521 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac5800b560 con 0x7fac70075ff0 2024-05-08T05:51:58.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.015+0000 7fac6e000700 1 -- 172.21.15.112:0/2439326521 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58003aa0 con 0x7fac70075ff0 2024-05-08T05:51:58.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.015+0000 7fac77800700 1 -- 172.21.15.112:0/2439326521 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac701a1070 con 0x7fac70075ff0 2024-05-08T05:51:58.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.015+0000 7fac77800700 1 -- 172.21.15.112:0/2439326521 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac701a8950 con 0x7fac70075ff0 2024-05-08T05:51:58.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.016+0000 7fac77800700 1 -- 172.21.15.112:0/2439326521 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac70076dc0 con 0x7fac70075ff0 2024-05-08T05:51:58.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.018+0000 7fac6e000700 1 -- 172.21.15.112:0/2439326521 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fac5800ba50 con 0x7fac70075ff0 2024-05-08T05:51:58.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.019+0000 7fac6e000700 1 --2- 172.21.15.112:0/2439326521 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fac540b2990 0x7fac540b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:58.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.020+0000 7fac6e000700 1 -- 172.21.15.112:0/2439326521 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fac580e31e0 con 0x7fac70075ff0 2024-05-08T05:51:58.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.020+0000 7fac74e00700 1 --2- 172.21.15.112:0/2439326521 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fac540b2990 0x7fac540b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:58.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.020+0000 7fac74e00700 1 --2- 172.21.15.112:0/2439326521 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fac540b2990 0x7fac540b4e50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fac6000f540 tx=0x7fac6000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:58.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.024+0000 7fac6e000700 1 -- 172.21.15.112:0/2439326521 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac58013030 con 0x7fac70075ff0 2024-05-08T05:51:58.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.260+0000 7fac77800700 1 -- 172.21.15.112:0/2439326521 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac7002e510 con 0x7fac70075ff0 2024-05-08T05:51:58.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.262+0000 7fac6e000700 1 -- 172.21.15.112:0/2439326521 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22287 (secure 0 0 0) 0x7fac580aaee0 con 0x7fac70075ff0 2024-05-08T05:51:58.263 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:51:58.264 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":80,"fsid":"7a07ff60-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:48.498871+0000","modified":"2024-05-08T05:51:07.312576+0000","last_up_change":"2024-05-08T05:49:54.039380+0000","last_in_change":"2024-05-08T05:49:34.443850+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:47.485105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:50:58.918909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.175022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b378542-1bbe-4614-8805-8a6b875ada55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6803","nonce":3419091820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6805","nonce":3419091820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6809","nonce":3419091820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6807","nonce":3419091820}]},"public_addr":"172.21.15.112:6803/3419091820","cluster_addr":"172.21.15.112:6805/3419091820","heartbeat_back_addr":"172.21.15.112:6809/3419091820","heartbeat_front_addr":"172.21.15.112:6807/3419091820","state":["exists","up"]},{"osd":1,"uuid":"826ef036-2db7-470e-9956-d9a58b9181e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6801","nonce":2728419797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6803","nonce":2728419797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6807","nonce":2728419797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6805","nonce":2728419797}]},"public_addr":"172.21.15.150:6801/2728419797","cluster_addr":"172.21.15.150:6803/2728419797","heartbeat_back_addr":"172.21.15.150:6807/2728419797","heartbeat_front_addr":"172.21.15.150:6805/2728419797","state":["exists","up"]},{"osd":2,"uuid":"32e710e7-2a87-4863-bc7a-906d31aa4ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6801","nonce":1556203215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6803","nonce":1556203215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6807","nonce":1556203215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6805","nonce":1556203215}]},"public_addr":"172.21.15.162:6801/1556203215","cluster_addr":"172.21.15.162:6803/1556203215","heartbeat_back_addr":"172.21.15.162:6807/1556203215","heartbeat_front_addr":"172.21.15.162:6805/1556203215","state":["exists","up"]},{"osd":3,"uuid":"ba1d4a5f-6a95-4b9e-8a48-8524a899b004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6811","nonce":3325718755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6813","nonce":3325718755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6817","nonce":3325718755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6815","nonce":3325718755}]},"public_addr":"172.21.15.112:6811/3325718755","cluster_addr":"172.21.15.112:6813/3325718755","heartbeat_back_addr":"172.21.15.112:6817/3325718755","heartbeat_front_addr":"172.21.15.112:6815/3325718755","state":["exists","up"]},{"osd":4,"uuid":"cb4f1a42-8de8-4b50-b3bc-602de9f4803a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6809","nonce":1548914683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6811","nonce":1548914683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6815","nonce":1548914683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6813","nonce":1548914683}]},"public_addr":"172.21.15.150:6809/1548914683","cluster_addr":"172.21.15.150:6811/1548914683","heartbeat_back_addr":"172.21.15.150:6815/1548914683","heartbeat_front_addr":"172.21.15.150:6813/1548914683","state":["exists","up"]},{"osd":5,"uuid":"a26c03b8-050c-4a2c-8f23-5801fffb7c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6809","nonce":409836750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6811","nonce":409836750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6815","nonce":409836750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6813","nonce":409836750}]},"public_addr":"172.21.15.162:6809/409836750","cluster_addr":"172.21.15.162:6811/409836750","heartbeat_back_addr":"172.21.15.162:6815/409836750","heartbeat_front_addr":"172.21.15.162:6813/409836750","state":["exists","up"]},{"osd":6,"uuid":"8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6819","nonce":1098984186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6821","nonce":1098984186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6825","nonce":1098984186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6823","nonce":1098984186}]},"public_addr":"172.21.15.112:6819/1098984186","cluster_addr":"172.21.15.112:6821/1098984186","heartbeat_back_addr":"172.21.15.112:6825/1098984186","heartbeat_front_addr":"172.21.15.112:6823/1098984186","state":["exists","up"]},{"osd":7,"uuid":"af3629b4-9bed-4669-97a6-48deba503a96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6817","nonce":4009836564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6819","nonce":4009836564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6823","nonce":4009836564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6821","nonce":4009836564}]},"public_addr":"172.21.15.150:6817/4009836564","cluster_addr":"172.21.15.150:6819/4009836564","heartbeat_back_addr":"172.21.15.150:6823/4009836564","heartbeat_front_addr":"172.21.15.150:6821/4009836564","state":["exists","up"]},{"osd":8,"uuid":"9333e87d-dd78-4467-bbde-c2b44f4ea07f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6817","nonce":852791606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6819","nonce":852791606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6823","nonce":852791606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6821","nonce":852791606}]},"public_addr":"172.21.15.162:6817/852791606","cluster_addr":"172.21.15.162:6819/852791606","heartbeat_back_addr":"172.21.15.162:6823/852791606","heartbeat_front_addr":"172.21.15.162:6821/852791606","state":["exists","up"]},{"osd":9,"uuid":"09723581-1995-42e0-8db7-776ea98a918c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6827","nonce":2186764674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6829","nonce":2186764674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6833","nonce":2186764674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6831","nonce":2186764674}]},"public_addr":"172.21.15.112:6827/2186764674","cluster_addr":"172.21.15.112:6829/2186764674","heartbeat_back_addr":"172.21.15.112:6833/2186764674","heartbeat_front_addr":"172.21.15.112:6831/2186764674","state":["exists","up"]},{"osd":10,"uuid":"34e08e3e-1231-4899-a5b3-76d449105bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6825","nonce":2838995474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6827","nonce":2838995474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6831","nonce":2838995474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6829","nonce":2838995474}]},"public_addr":"172.21.15.150:6825/2838995474","cluster_addr":"172.21.15.150:6827/2838995474","heartbeat_back_addr":"172.21.15.150:6831/2838995474","heartbeat_front_addr":"172.21.15.150:6829/2838995474","state":["exists","up"]},{"osd":11,"uuid":"6fc34ec0-78ca-403c-acb7-b29d83ff64e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6825","nonce":3702614331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6827","nonce":3702614331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6831","nonce":3702614331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6829","nonce":3702614331}]},"public_addr":"172.21.15.162:6825/3702614331","cluster_addr":"172.21.15.162:6827/3702614331","heartbeat_back_addr":"172.21.15.162:6831/3702614331","heartbeat_front_addr":"172.21.15.162:6829/3702614331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:54.518702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:19.939543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:44.425586+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:16.126471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:41.696071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:05.517405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:38.549318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:03.799649+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:28.007282+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:01.395459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:27.963637+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:52.181212+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3667148497":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6800/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3425327158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1404611205":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3765334043":"2024-05-09T05:42:29.819307+0000","172.21.15.112:6800/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/2814303790":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1091751583":"2024-05-09T05:42:29.819307+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:51:58.264 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:58 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:51:58.264 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:58 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:51:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.266+0000 7fac77800700 1 -- 172.21.15.112:0/2439326521 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fac540b2990 msgr2=0x7fac540b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.266+0000 7fac77800700 1 --2- 172.21.15.112:0/2439326521 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fac540b2990 0x7fac540b4e50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fac6000f540 tx=0x7fac6000b040 comp rx=0 tx=0).stop 2024-05-08T05:51:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.266+0000 7fac77800700 1 -- 172.21.15.112:0/2439326521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac70075ff0 msgr2=0x7fac7019c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.266+0000 7fac77800700 1 --2- 172.21.15.112:0/2439326521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac70075ff0 0x7fac7019c310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fac58009230 tx=0x7fac5800f990 comp rx=0 tx=0).stop 2024-05-08T05:51:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.266+0000 7fac77800700 1 -- 172.21.15.112:0/2439326521 shutdown_connections 2024-05-08T05:51:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.266+0000 7fac77800700 1 --2- 172.21.15.112:0/2439326521 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fac540b2990 0x7fac540b4e50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.266+0000 7fac77800700 1 --2- 172.21.15.112:0/2439326521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac7010fea0 0x7fac701a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.266+0000 7fac77800700 1 --2- 172.21.15.112:0/2439326521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac700756d0 0x7fac7019bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:58.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.266+0000 7fac77800700 1 --2- 172.21.15.112:0/2439326521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac70075ff0 0x7fac7019c310 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:58.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.266+0000 7fac77800700 1 -- 172.21.15.112:0/2439326521 >> 172.21.15.112:0/2439326521 conn(0x7fac7007a600 msgr2=0x7fac70100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:58.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.267+0000 7fac77800700 1 -- 172.21.15.112:0/2439326521 shutdown_connections 2024-05-08T05:51:58.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:51:58.267+0000 7fac77800700 1 -- 172.21.15.112:0/2439326521 wait complete. 2024-05-08T05:51:58.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:58 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:51:58.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:58 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:51:58.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:58 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:51:58.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:58 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:51:59.186 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T05:51:59.186 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:51:59.243 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:59 smithi112 ceph-mon[30003]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-08T05:51:59.243 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:51:59 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2439326521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:59.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:59 smithi162 ceph-mon[34800]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-08T05:51:59.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:51:59 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/2439326521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:59.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:59 smithi150 ceph-mon[34315]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-08T05:51:59.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:51:59 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/2439326521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:59.632 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:01.252 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:00 smithi112 ceph-mon[30003]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-08T05:52:01.252 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:00 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:52:01.252 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:00 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:52:01.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:00 smithi162 ceph-mon[34800]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-08T05:52:01.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:00 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:52:01.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:00 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:52:01.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:00 smithi150 ceph-mon[34315]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-08T05:52:01.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:00 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:52:01.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:00 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:52:01.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.746+0000 7fd421600700 1 -- 172.21.15.112:0/3504834517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c109ad0 msgr2=0x7fd41c109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:01.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.746+0000 7fd421600700 1 --2- 172.21.15.112:0/3504834517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c109ad0 0x7fd41c109f50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd408009990 tx=0x7fd408009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.746+0000 7fd421600700 1 -- 172.21.15.112:0/3504834517 shutdown_connections 2024-05-08T05:52:01.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.746+0000 7fd421600700 1 --2- 172.21.15.112:0/3504834517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd41c10a490 0x7fd41c1169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.746+0000 7fd421600700 1 --2- 172.21.15.112:0/3504834517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd41c10fb60 0x7fd41c10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.746+0000 7fd421600700 1 --2- 172.21.15.112:0/3504834517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c109ad0 0x7fd41c109f50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.746+0000 7fd421600700 1 -- 172.21.15.112:0/3504834517 >> 172.21.15.112:0/3504834517 conn(0x7fd41c076c70 msgr2=0x7fd41c1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:01.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.746+0000 7fd421600700 1 -- 172.21.15.112:0/3504834517 shutdown_connections 2024-05-08T05:52:01.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.746+0000 7fd421600700 1 -- 172.21.15.112:0/3504834517 wait complete. 2024-05-08T05:52:01.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.747+0000 7fd421600700 1 Processor -- start 2024-05-08T05:52:01.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.747+0000 7fd421600700 1 -- start start 2024-05-08T05:52:01.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.748+0000 7fd421600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c109ad0 0x7fd41c1a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:01.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.748+0000 7fd421600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd41c10a490 0x7fd41c1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:01.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.748+0000 7fd421600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd41c10fb60 0x7fd41c1a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:01.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.748+0000 7fd421600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c119d00 con 0x7fd41c109ad0 2024-05-08T05:52:01.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.748+0000 7fd421600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c119b80 con 0x7fd41c10fb60 2024-05-08T05:52:01.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.748+0000 7fd421600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c119e80 con 0x7fd41c10a490 2024-05-08T05:52:01.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.748+0000 7fd41aa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c109ad0 0x7fd41c1a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:01.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.748+0000 7fd41a000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd41c10a490 0x7fd41c1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:01.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.748+0000 7fd41a000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd41c10a490 0x7fd41c1a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:53914/0 (socket says 172.21.15.112:53914) 2024-05-08T05:52:01.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.748+0000 7fd41aa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c109ad0 0x7fd41c1a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58380/0 (socket says 172.21.15.112:58380) 2024-05-08T05:52:01.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.748+0000 7fd41a000700 1 -- 172.21.15.112:0/2413726154 learned_addr learned my addr 172.21.15.112:0/2413726154 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:01.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.748+0000 7fd41b400700 1 --2- 172.21.15.112:0/2413726154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd41c10fb60 0x7fd41c1a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:01.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.749+0000 7fd41a000700 1 -- 172.21.15.112:0/2413726154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd41c10fb60 msgr2=0x7fd41c1a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:01.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.749+0000 7fd41a000700 1 --2- 172.21.15.112:0/2413726154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd41c10fb60 0x7fd41c1a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.749+0000 7fd41a000700 1 -- 172.21.15.112:0/2413726154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c109ad0 msgr2=0x7fd41c1a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:01.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.749+0000 7fd41a000700 1 --2- 172.21.15.112:0/2413726154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c109ad0 0x7fd41c1a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.749+0000 7fd41a000700 1 -- 172.21.15.112:0/2413726154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41c1a5190 con 0x7fd41c10a490 2024-05-08T05:52:01.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.749+0000 7fd41a000700 1 --2- 172.21.15.112:0/2413726154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd41c10a490 0x7fd41c1a0710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd408005910 tx=0x7fd4080047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:01.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.749+0000 7fd413400700 1 -- 172.21.15.112:0/2413726154 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408027070 con 0x7fd41c10a490 2024-05-08T05:52:01.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.749+0000 7fd421600700 1 -- 172.21.15.112:0/2413726154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41c1a5420 con 0x7fd41c10a490 2024-05-08T05:52:01.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.749+0000 7fd413400700 1 -- 172.21.15.112:0/2413726154 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd408003f20 con 0x7fd41c10a490 2024-05-08T05:52:01.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.749+0000 7fd413400700 1 -- 172.21.15.112:0/2413726154 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408022c60 con 0x7fd41c10a490 2024-05-08T05:52:01.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.749+0000 7fd421600700 1 -- 172.21.15.112:0/2413726154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41c1acd00 con 0x7fd41c10a490 2024-05-08T05:52:01.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.750+0000 7fd421600700 1 -- 172.21.15.112:0/2413726154 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd41c06a190 con 0x7fd41c10a490 2024-05-08T05:52:01.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.753+0000 7fd413400700 1 -- 172.21.15.112:0/2413726154 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fd40802a070 con 0x7fd41c10a490 2024-05-08T05:52:01.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.754+0000 7fd413400700 1 --2- 172.21.15.112:0/2413726154 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4000b2910 0x7fd4000b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:01.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.754+0000 7fd413400700 1 -- 172.21.15.112:0/2413726154 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fd408022e00 con 0x7fd41c10a490 2024-05-08T05:52:01.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.754+0000 7fd41aa00700 1 --2- 172.21.15.112:0/2413726154 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4000b2910 0x7fd4000b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:01.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.755+0000 7fd41aa00700 1 --2- 172.21.15.112:0/2413726154 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4000b2910 0x7fd4000b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd40c007b60 tx=0x7fd40c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:01.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.757+0000 7fd413400700 1 -- 172.21.15.112:0/2413726154 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd408031070 con 0x7fd41c10a490 2024-05-08T05:52:01.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.983+0000 7fd421600700 1 -- 172.21.15.112:0/2413726154 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd41c1a14e0 con 0x7fd4000b2910 2024-05-08T05:52:01.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:01.998+0000 7fd413400700 1 -- 172.21.15.112:0/2413726154 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258698 (secure 0 0 0) 0x7fd41c1a14e0 con 0x7fd4000b2910 2024-05-08T05:52:02.005 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:52:02.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:02.003+0000 7fd421600700 1 -- 172.21.15.112:0/2413726154 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4000b2910 msgr2=0x7fd4000b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:02.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:02.003+0000 7fd421600700 1 --2- 172.21.15.112:0/2413726154 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4000b2910 0x7fd4000b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd40c007b60 tx=0x7fd40c0073d0 comp rx=0 tx=0).stop 2024-05-08T05:52:02.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:02.003+0000 7fd421600700 1 -- 172.21.15.112:0/2413726154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd41c10a490 msgr2=0x7fd41c1a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:02.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:02.004+0000 7fd421600700 1 --2- 172.21.15.112:0/2413726154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd41c10a490 0x7fd41c1a0710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd408005910 tx=0x7fd4080047b0 comp rx=0 tx=0).stop 2024-05-08T05:52:02.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:02.004+0000 7fd421600700 1 -- 172.21.15.112:0/2413726154 shutdown_connections 2024-05-08T05:52:02.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:02.004+0000 7fd421600700 1 --2- 172.21.15.112:0/2413726154 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4000b2910 0x7fd4000b4dd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:02.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:02.004+0000 7fd421600700 1 --2- 172.21.15.112:0/2413726154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd41c10fb60 0x7fd41c1a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:02.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:02.004+0000 7fd421600700 1 --2- 172.21.15.112:0/2413726154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd41c109ad0 0x7fd41c1a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:02.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:02.004+0000 7fd421600700 1 --2- 172.21.15.112:0/2413726154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd41c10a490 0x7fd41c1a0710 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:02.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:02.004+0000 7fd421600700 1 -- 172.21.15.112:0/2413726154 >> 172.21.15.112:0/2413726154 conn(0x7fd41c076c70 msgr2=0x7fd41c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:02.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:02.004+0000 7fd421600700 1 -- 172.21.15.112:0/2413726154 shutdown_connections 2024-05-08T05:52:02.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:02.004+0000 7fd421600700 1 -- 172.21.15.112:0/2413726154 wait complete. 2024-05-08T05:52:02.007 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-05-08T05:52:02.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:01 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:52:02.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:01 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:52:02.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:01 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:52:02.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:01 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:52:02.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:01 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:52:02.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:01 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:52:02.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:01 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:52:02.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:01 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:52:02.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:01 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:52:02.845 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-05-08T05:52:01.489287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656353,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":298,"num_read_kb":245,"num_write":290,"num_write_kb":2264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361400,"kb_used_data":12248,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580424,"statfs":{"total":1151940427776,"available":1151570354176,"internally_reserved":0,"allocated":12541952,"data_stored":6977562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9184,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":67,"num_read_kb":45,"num_write":12,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004387"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.318789+0000","last_change":"2024-05-08T05:51:04.320319+0000","last_active":"2024-05-08T05:51:07.318789+0000","last_peered":"2024-05-08T05:51:07.318789+0000","last_clean":"2024-05-08T05:51:07.318789+0000","last_became_active":"2024-05-08T05:51:04.319961+0000","last_became_peered":"2024-05-08T05:51:04.319961+0000","last_unstale":"2024-05-08T05:51:07.318789+0000","last_undegraded":"2024-05-08T05:51:07.318789+0000","last_fullsized":"2024-05-08T05:51:07.318789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:32.463261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335395+0000","last_change":"2024-05-08T05:51:04.315531+0000","last_active":"2024-05-08T05:51:07.335395+0000","last_peered":"2024-05-08T05:51:07.335395+0000","last_clean":"2024-05-08T05:51:07.335395+0000","last_became_active":"2024-05-08T05:51:04.314116+0000","last_became_peered":"2024-05-08T05:51:04.314116+0000","last_unstale":"2024-05-08T05:51:07.335395+0000","last_undegraded":"2024-05-08T05:51:07.335395+0000","last_fullsized":"2024-05-08T05:51:07.335395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:12:03.505888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335448+0000","last_change":"2024-05-08T05:51:04.315826+0000","last_active":"2024-05-08T05:51:07.335448+0000","last_peered":"2024-05-08T05:51:07.335448+0000","last_clean":"2024-05-08T05:51:07.335448+0000","last_became_active":"2024-05-08T05:51:04.315460+0000","last_became_peered":"2024-05-08T05:51:04.315460+0000","last_unstale":"2024-05-08T05:51:07.335448+0000","last_undegraded":"2024-05-08T05:51:07.335448+0000","last_fullsized":"2024-05-08T05:51:07.335448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:22:20.412368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333562+0000","last_change":"2024-05-08T05:51:04.297401+0000","last_active":"2024-05-08T05:51:07.333562+0000","last_peered":"2024-05-08T05:51:07.333562+0000","last_clean":"2024-05-08T05:51:07.333562+0000","last_became_active":"2024-05-08T05:51:04.297061+0000","last_became_peered":"2024-05-08T05:51:04.297061+0000","last_unstale":"2024-05-08T05:51:07.333562+0000","last_undegraded":"2024-05-08T05:51:07.333562+0000","last_fullsized":"2024-05-08T05:51:07.333562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:27.967517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334225+0000","last_change":"2024-05-08T05:51:04.315069+0000","last_active":"2024-05-08T05:51:07.334225+0000","last_peered":"2024-05-08T05:51:07.334225+0000","last_clean":"2024-05-08T05:51:07.334225+0000","last_became_active":"2024-05-08T05:51:04.314513+0000","last_became_peered":"2024-05-08T05:51:04.314513+0000","last_unstale":"2024-05-08T05:51:07.334225+0000","last_undegraded":"2024-05-08T05:51:07.334225+0000","last_fullsized":"2024-05-08T05:51:07.334225+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:02.520130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.318398+0000","last_change":"2024-05-08T05:51:04.302851+0000","last_active":"2024-05-08T05:51:07.318398+0000","last_peered":"2024-05-08T05:51:07.318398+0000","last_clean":"2024-05-08T05:51:07.318398+0000","last_became_active":"2024-05-08T05:51:04.302256+0000","last_became_peered":"2024-05-08T05:51:04.302256+0000","last_unstale":"2024-05-08T05:51:07.318398+0000","last_undegraded":"2024-05-08T05:51:07.318398+0000","last_fullsized":"2024-05-08T05:51:07.318398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:45.587319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.337820+0000","last_change":"2024-05-08T05:51:04.308674+0000","last_active":"2024-05-08T05:51:07.337820+0000","last_peered":"2024-05-08T05:51:07.337820+0000","last_clean":"2024-05-08T05:51:07.337820+0000","last_became_active":"2024-05-08T05:51:04.308295+0000","last_became_peered":"2024-05-08T05:51:04.308295+0000","last_unstale":"2024-05-08T05:51:07.337820+0000","last_undegraded":"2024-05-08T05:51:07.337820+0000","last_fullsized":"2024-05-08T05:51:07.337820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:59:17.561655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.319096+0000","last_change":"2024-05-08T05:51:04.314511+0000","last_active":"2024-05-08T05:51:07.319096+0000","last_peered":"2024-05-08T05:51:07.319096+0000","last_clean":"2024-05-08T05:51:07.319096+0000","last_became_active":"2024-05-08T05:51:04.314082+0000","last_became_peered":"2024-05-08T05:51:04.314082+0000","last_unstale":"2024-05-08T05:51:07.319096+0000","last_undegraded":"2024-05-08T05:51:07.319096+0000","last_fullsized":"2024-05-08T05:51:07.319096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:24.182391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335715+0000","last_change":"2024-05-08T05:51:04.293745+0000","last_active":"2024-05-08T05:51:07.335715+0000","last_peered":"2024-05-08T05:51:07.335715+0000","last_clean":"2024-05-08T05:51:07.335715+0000","last_became_active":"2024-05-08T05:51:04.293334+0000","last_became_peered":"2024-05-08T05:51:04.293334+0000","last_unstale":"2024-05-08T05:51:07.335715+0000","last_undegraded":"2024-05-08T05:51:07.335715+0000","last_fullsized":"2024-05-08T05:51:07.335715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:16.576967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.318406+0000","last_change":"2024-05-08T05:51:04.297800+0000","last_active":"2024-05-08T05:51:07.318406+0000","last_peered":"2024-05-08T05:51:07.318406+0000","last_clean":"2024-05-08T05:51:07.318406+0000","last_became_active":"2024-05-08T05:51:04.297349+0000","last_became_peered":"2024-05-08T05:51:04.297349+0000","last_unstale":"2024-05-08T05:51:07.318406+0000","last_undegraded":"2024-05-08T05:51:07.318406+0000","last_fullsized":"2024-05-08T05:51:07.318406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:55.784512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.338828+0000","last_change":"2024-05-08T05:51:04.297997+0000","last_active":"2024-05-08T05:51:07.338828+0000","last_peered":"2024-05-08T05:51:07.338828+0000","last_clean":"2024-05-08T05:51:07.338828+0000","last_became_active":"2024-05-08T05:51:04.297516+0000","last_became_peered":"2024-05-08T05:51:04.297516+0000","last_unstale":"2024-05-08T05:51:07.338828+0000","last_undegraded":"2024-05-08T05:51:07.338828+0000","last_fullsized":"2024-05-08T05:51:07.338828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:25.991555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333377+0000","last_change":"2024-05-08T05:51:04.295610+0000","last_active":"2024-05-08T05:51:07.333377+0000","last_peered":"2024-05-08T05:51:07.333377+0000","last_clean":"2024-05-08T05:51:07.333377+0000","last_became_active":"2024-05-08T05:51:04.294553+0000","last_became_peered":"2024-05-08T05:51:04.294553+0000","last_unstale":"2024-05-08T05:51:07.333377+0000","last_undegraded":"2024-05-08T05:51:07.333377+0000","last_fullsized":"2024-05-08T05:51:07.333377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:40:14.492057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335603+0000","last_change":"2024-05-08T05:51:04.303879+0000","last_active":"2024-05-08T05:51:07.335603+0000","last_peered":"2024-05-08T05:51:07.335603+0000","last_clean":"2024-05-08T05:51:07.335603+0000","last_became_active":"2024-05-08T05:51:04.303423+0000","last_became_peered":"2024-05-08T05:51:04.303423+0000","last_unstale":"2024-05-08T05:51:07.335603+0000","last_undegraded":"2024-05-08T05:51:07.335603+0000","last_fullsized":"2024-05-08T05:51:07.335603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:50:55.905858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334644+0000","last_change":"2024-05-08T05:51:04.309818+0000","last_active":"2024-05-08T05:51:07.334644+0000","last_peered":"2024-05-08T05:51:07.334644+0000","last_clean":"2024-05-08T05:51:07.334644+0000","last_became_active":"2024-05-08T05:51:04.309434+0000","last_became_peered":"2024-05-08T05:51:04.309434+0000","last_unstale":"2024-05-08T05:51:07.334644+0000","last_undegraded":"2024-05-08T05:51:07.334644+0000","last_fullsized":"2024-05-08T05:51:07.334644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:55.832239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334819+0000","last_change":"2024-05-08T05:51:04.316708+0000","last_active":"2024-05-08T05:51:07.334819+0000","last_peered":"2024-05-08T05:51:07.334819+0000","last_clean":"2024-05-08T05:51:07.334819+0000","last_became_active":"2024-05-08T05:51:04.316525+0000","last_became_peered":"2024-05-08T05:51:04.316525+0000","last_unstale":"2024-05-08T05:51:07.334819+0000","last_undegraded":"2024-05-08T05:51:07.334819+0000","last_fullsized":"2024-05-08T05:51:07.334819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:33:38.246611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335158+0000","last_change":"2024-05-08T05:51:04.314780+0000","last_active":"2024-05-08T05:51:07.335158+0000","last_peered":"2024-05-08T05:51:07.335158+0000","last_clean":"2024-05-08T05:51:07.335158+0000","last_became_active":"2024-05-08T05:51:04.314406+0000","last_became_peered":"2024-05-08T05:51:04.314406+0000","last_unstale":"2024-05-08T05:51:07.335158+0000","last_undegraded":"2024-05-08T05:51:07.335158+0000","last_fullsized":"2024-05-08T05:51:07.335158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:39:26.926238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.318993+0000","last_change":"2024-05-08T05:51:04.319425+0000","last_active":"2024-05-08T05:51:07.318993+0000","last_peered":"2024-05-08T05:51:07.318993+0000","last_clean":"2024-05-08T05:51:07.318993+0000","last_became_active":"2024-05-08T05:51:04.319106+0000","last_became_peered":"2024-05-08T05:51:04.319106+0000","last_unstale":"2024-05-08T05:51:07.318993+0000","last_undegraded":"2024-05-08T05:51:07.318993+0000","last_fullsized":"2024-05-08T05:51:07.318993+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:59:54.023162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333643+0000","last_change":"2024-05-08T05:51:04.303547+0000","last_active":"2024-05-08T05:51:07.333643+0000","last_peered":"2024-05-08T05:51:07.333643+0000","last_clean":"2024-05-08T05:51:07.333643+0000","last_became_active":"2024-05-08T05:51:04.303182+0000","last_became_peered":"2024-05-08T05:51:04.303182+0000","last_unstale":"2024-05-08T05:51:07.333643+0000","last_undegraded":"2024-05-08T05:51:07.333643+0000","last_fullsized":"2024-05-08T05:51:07.333643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:11:59.624329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334261+0000","last_change":"2024-05-08T05:51:04.315674+0000","last_active":"2024-05-08T05:51:07.334261+0000","last_peered":"2024-05-08T05:51:07.334261+0000","last_clean":"2024-05-08T05:51:07.334261+0000","last_became_active":"2024-05-08T05:51:04.314911+0000","last_became_peered":"2024-05-08T05:51:04.314911+0000","last_unstale":"2024-05-08T05:51:07.334261+0000","last_undegraded":"2024-05-08T05:51:07.334261+0000","last_fullsized":"2024-05-08T05:51:07.334261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:49.501508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:24.624525+0000","last_change":"2024-05-08T05:51:00.273554+0000","last_active":"2024-05-08T05:51:24.624525+0000","last_peered":"2024-05-08T05:51:24.624525+0000","last_clean":"2024-05-08T05:51:24.624525+0000","last_became_active":"2024-05-08T05:51:00.272858+0000","last_became_peered":"2024-05-08T05:51:00.272858+0000","last_unstale":"2024-05-08T05:51:24.624525+0000","last_undegraded":"2024-05-08T05:51:24.624525+0000","last_fullsized":"2024-05-08T05:51:24.624525+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:47:37.418887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.320427+0000","last_change":"2024-05-08T05:51:04.306392+0000","last_active":"2024-05-08T05:51:07.320427+0000","last_peered":"2024-05-08T05:51:07.320427+0000","last_clean":"2024-05-08T05:51:07.320427+0000","last_became_active":"2024-05-08T05:51:04.304248+0000","last_became_peered":"2024-05-08T05:51:04.304248+0000","last_unstale":"2024-05-08T05:51:07.320427+0000","last_undegraded":"2024-05-08T05:51:07.320427+0000","last_fullsized":"2024-05-08T05:51:07.320427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:49:26.034883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.338794+0000","last_change":"2024-05-08T05:51:01.187292+0000","last_active":"2024-05-08T05:51:07.338794+0000","last_peered":"2024-05-08T05:51:07.338794+0000","last_clean":"2024-05-08T05:51:07.338794+0000","last_became_active":"2024-05-08T05:51:01.185948+0000","last_became_peered":"2024-05-08T05:51:01.185948+0000","last_unstale":"2024-05-08T05:51:07.338794+0000","last_undegraded":"2024-05-08T05:51:07.338794+0000","last_fullsized":"2024-05-08T05:51:07.338794+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:02.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335955+0000","last_change":"2024-05-08T05:51:04.318439+0000","last_active":"2024-05-08T05:51:07.335955+0000","last_peered":"2024-05-08T05:51:07.335955+0000","last_clean":"2024-05-08T05:51:07.335955+0000","last_became_active":"2024-05-08T05:51:04.317722+0000","last_became_peered":"2024-05-08T05:51:04.317722+0000","last_unstale":"2024-05-08T05:51:07.335955+0000","last_undegraded":"2024-05-08T05:51:07.335955+0000","last_fullsized":"2024-05-08T05:51:07.335955+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:36:43.648643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:24.612064+0000","last_change":"2024-05-08T05:51:00.269684+0000","last_active":"2024-05-08T05:51:24.612064+0000","last_peered":"2024-05-08T05:51:24.612064+0000","last_clean":"2024-05-08T05:51:24.612064+0000","last_became_active":"2024-05-08T05:51:00.269329+0000","last_became_peered":"2024-05-08T05:51:00.269329+0000","last_unstale":"2024-05-08T05:51:24.612064+0000","last_undegraded":"2024-05-08T05:51:24.612064+0000","last_fullsized":"2024-05-08T05:51:24.612064+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:27:10.422994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334290+0000","last_change":"2024-05-08T05:51:04.308048+0000","last_active":"2024-05-08T05:51:07.334290+0000","last_peered":"2024-05-08T05:51:07.334290+0000","last_clean":"2024-05-08T05:51:07.334290+0000","last_became_active":"2024-05-08T05:51:04.307550+0000","last_became_peered":"2024-05-08T05:51:04.307550+0000","last_unstale":"2024-05-08T05:51:07.334290+0000","last_undegraded":"2024-05-08T05:51:07.334290+0000","last_fullsized":"2024-05-08T05:51:07.334290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:13.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.341265+0000","last_change":"2024-05-08T05:51:00.268142+0000","last_active":"2024-05-08T05:51:07.341265+0000","last_peered":"2024-05-08T05:51:07.341265+0000","last_clean":"2024-05-08T05:51:07.341265+0000","last_became_active":"2024-05-08T05:51:00.267756+0000","last_became_peered":"2024-05-08T05:51:00.267756+0000","last_unstale":"2024-05-08T05:51:07.341265+0000","last_undegraded":"2024-05-08T05:51:07.341265+0000","last_fullsized":"2024-05-08T05:51:07.341265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:01.439680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333766+0000","last_change":"2024-05-08T05:51:04.292341+0000","last_active":"2024-05-08T05:51:07.333766+0000","last_peered":"2024-05-08T05:51:07.333766+0000","last_clean":"2024-05-08T05:51:07.333766+0000","last_became_active":"2024-05-08T05:51:04.291086+0000","last_became_peered":"2024-05-08T05:51:04.291086+0000","last_unstale":"2024-05-08T05:51:07.333766+0000","last_undegraded":"2024-05-08T05:51:07.333766+0000","last_fullsized":"2024-05-08T05:51:07.333766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:47.108100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":61,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:57.682474+0000","last_change":"2024-05-08T05:51:00.268100+0000","last_active":"2024-05-08T05:51:57.682474+0000","last_peered":"2024-05-08T05:51:57.682474+0000","last_clean":"2024-05-08T05:51:57.682474+0000","last_became_active":"2024-05-08T05:51:00.267678+0000","last_became_peered":"2024-05-08T05:51:00.267678+0000","last_unstale":"2024-05-08T05:51:57.682474+0000","last_undegraded":"2024-05-08T05:51:57.682474+0000","last_fullsized":"2024-05-08T05:51:57.682474+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:11:31.731548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334601+0000","last_change":"2024-05-08T05:51:04.303872+0000","last_active":"2024-05-08T05:51:07.334601+0000","last_peered":"2024-05-08T05:51:07.334601+0000","last_clean":"2024-05-08T05:51:07.334601+0000","last_became_active":"2024-05-08T05:51:04.303456+0000","last_became_peered":"2024-05-08T05:51:04.303456+0000","last_unstale":"2024-05-08T05:51:07.334601+0000","last_undegraded":"2024-05-08T05:51:07.334601+0000","last_fullsized":"2024-05-08T05:51:07.334601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:42:05.098488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:24.622558+0000","last_change":"2024-05-08T05:51:00.275657+0000","last_active":"2024-05-08T05:51:24.622558+0000","last_peered":"2024-05-08T05:51:24.622558+0000","last_clean":"2024-05-08T05:51:24.622558+0000","last_became_active":"2024-05-08T05:51:00.275254+0000","last_became_peered":"2024-05-08T05:51:00.275254+0000","last_unstale":"2024-05-08T05:51:24.622558+0000","last_undegraded":"2024-05-08T05:51:24.622558+0000","last_fullsized":"2024-05-08T05:51:24.622558+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:28.555698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333280+0000","last_change":"2024-05-08T05:51:04.293791+0000","last_active":"2024-05-08T05:51:07.333280+0000","last_peered":"2024-05-08T05:51:07.333280+0000","last_clean":"2024-05-08T05:51:07.333280+0000","last_became_active":"2024-05-08T05:51:04.293405+0000","last_became_peered":"2024-05-08T05:51:04.293405+0000","last_unstale":"2024-05-08T05:51:07.333280+0000","last_undegraded":"2024-05-08T05:51:07.333280+0000","last_fullsized":"2024-05-08T05:51:07.333280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:28.064452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:24.625314+0000","last_change":"2024-05-08T05:51:00.280295+0000","last_active":"2024-05-08T05:51:24.625314+0000","last_peered":"2024-05-08T05:51:24.625314+0000","last_clean":"2024-05-08T05:51:24.625314+0000","last_became_active":"2024-05-08T05:51:00.279721+0000","last_became_peered":"2024-05-08T05:51:00.279721+0000","last_unstale":"2024-05-08T05:51:24.625314+0000","last_undegraded":"2024-05-08T05:51:24.625314+0000","last_fullsized":"2024-05-08T05:51:24.625314+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:06:38.064872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.319121+0000","last_change":"2024-05-08T05:51:04.307546+0000","last_active":"2024-05-08T05:51:07.319121+0000","last_peered":"2024-05-08T05:51:07.319121+0000","last_clean":"2024-05-08T05:51:07.319121+0000","last_became_active":"2024-05-08T05:51:04.307164+0000","last_became_peered":"2024-05-08T05:51:04.307164+0000","last_unstale":"2024-05-08T05:51:07.319121+0000","last_undegraded":"2024-05-08T05:51:07.319121+0000","last_fullsized":"2024-05-08T05:51:07.319121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:36:02.269040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.332947+0000","last_change":"2024-05-08T05:51:00.267987+0000","last_active":"2024-05-08T05:51:07.332947+0000","last_peered":"2024-05-08T05:51:07.332947+0000","last_clean":"2024-05-08T05:51:07.332947+0000","last_became_active":"2024-05-08T05:51:00.267633+0000","last_became_peered":"2024-05-08T05:51:00.267633+0000","last_unstale":"2024-05-08T05:51:07.332947+0000","last_undegraded":"2024-05-08T05:51:07.332947+0000","last_fullsized":"2024-05-08T05:51:07.332947+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:13.034300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333490+0000","last_change":"2024-05-08T05:51:04.311422+0000","last_active":"2024-05-08T05:51:07.333490+0000","last_peered":"2024-05-08T05:51:07.333490+0000","last_clean":"2024-05-08T05:51:07.333490+0000","last_became_active":"2024-05-08T05:51:04.311052+0000","last_became_peered":"2024-05-08T05:51:04.311052+0000","last_unstale":"2024-05-08T05:51:07.333490+0000","last_undegraded":"2024-05-08T05:51:07.333490+0000","last_fullsized":"2024-05-08T05:51:07.333490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:47.790583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":56,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:57.681819+0000","last_change":"2024-05-08T05:51:00.274252+0000","last_active":"2024-05-08T05:51:57.681819+0000","last_peered":"2024-05-08T05:51:57.681819+0000","last_clean":"2024-05-08T05:51:57.681819+0000","last_became_active":"2024-05-08T05:51:00.273942+0000","last_became_peered":"2024-05-08T05:51:00.273942+0000","last_unstale":"2024-05-08T05:51:57.681819+0000","last_undegraded":"2024-05-08T05:51:57.681819+0000","last_fullsized":"2024-05-08T05:51:57.681819+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:36.778032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335581+0000","last_change":"2024-05-08T05:51:04.296414+0000","last_active":"2024-05-08T05:51:07.335581+0000","last_peered":"2024-05-08T05:51:07.335581+0000","last_clean":"2024-05-08T05:51:07.335581+0000","last_became_active":"2024-05-08T05:51:04.296015+0000","last_became_peered":"2024-05-08T05:51:04.296015+0000","last_unstale":"2024-05-08T05:51:07.335581+0000","last_undegraded":"2024-05-08T05:51:07.335581+0000","last_fullsized":"2024-05-08T05:51:07.335581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:48:15.609385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:23.599143+0000","last_change":"2024-05-08T05:51:00.268234+0000","last_active":"2024-05-08T05:51:23.599143+0000","last_peered":"2024-05-08T05:51:23.599143+0000","last_clean":"2024-05-08T05:51:23.599143+0000","last_became_active":"2024-05-08T05:51:00.267846+0000","last_became_peered":"2024-05-08T05:51:00.267846+0000","last_unstale":"2024-05-08T05:51:23.599143+0000","last_undegraded":"2024-05-08T05:51:23.599143+0000","last_fullsized":"2024-05-08T05:51:23.599143+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:26.919004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335890+0000","last_change":"2024-05-08T05:51:04.318215+0000","last_active":"2024-05-08T05:51:07.335890+0000","last_peered":"2024-05-08T05:51:07.335890+0000","last_clean":"2024-05-08T05:51:07.335890+0000","last_became_active":"2024-05-08T05:51:04.317309+0000","last_became_peered":"2024-05-08T05:51:04.317309+0000","last_unstale":"2024-05-08T05:51:07.335890+0000","last_undegraded":"2024-05-08T05:51:07.335890+0000","last_fullsized":"2024-05-08T05:51:07.335890+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:11.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:23.603152+0000","last_change":"2024-05-08T05:51:01.186438+0000","last_active":"2024-05-08T05:51:23.603152+0000","last_peered":"2024-05-08T05:51:23.603152+0000","last_clean":"2024-05-08T05:51:23.603152+0000","last_became_active":"2024-05-08T05:51:01.186039+0000","last_became_peered":"2024-05-08T05:51:01.186039+0000","last_unstale":"2024-05-08T05:51:23.603152+0000","last_undegraded":"2024-05-08T05:51:23.603152+0000","last_fullsized":"2024-05-08T05:51:23.603152+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:34.652184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335496+0000","last_change":"2024-05-08T05:51:04.294993+0000","last_active":"2024-05-08T05:51:07.335496+0000","last_peered":"2024-05-08T05:51:07.335496+0000","last_clean":"2024-05-08T05:51:07.335496+0000","last_became_active":"2024-05-08T05:51:04.294620+0000","last_became_peered":"2024-05-08T05:51:04.294620+0000","last_unstale":"2024-05-08T05:51:07.335496+0000","last_undegraded":"2024-05-08T05:51:07.335496+0000","last_fullsized":"2024-05-08T05:51:07.335496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:13:44.469181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":124,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:58.683888+0000","last_change":"2024-05-08T05:51:00.263385+0000","last_active":"2024-05-08T05:51:58.683888+0000","last_peered":"2024-05-08T05:51:58.683888+0000","last_clean":"2024-05-08T05:51:58.683888+0000","last_became_active":"2024-05-08T05:51:00.262997+0000","last_became_peered":"2024-05-08T05:51:00.262997+0000","last_unstale":"2024-05-08T05:51:58.683888+0000","last_undegraded":"2024-05-08T05:51:58.683888+0000","last_fullsized":"2024-05-08T05:51:58.683888+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:22:09.727481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61750,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":73,"num_read_kb":96,"num_write":31,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335948+0000","last_change":"2024-05-08T05:51:04.294790+0000","last_active":"2024-05-08T05:51:07.335948+0000","last_peered":"2024-05-08T05:51:07.335948+0000","last_clean":"2024-05-08T05:51:07.335948+0000","last_became_active":"2024-05-08T05:51:04.294418+0000","last_became_peered":"2024-05-08T05:51:04.294418+0000","last_unstale":"2024-05-08T05:51:07.335948+0000","last_undegraded":"2024-05-08T05:51:07.335948+0000","last_fullsized":"2024-05-08T05:51:07.335948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:11:10.624434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:24.618463+0000","last_change":"2024-05-08T05:51:00.258507+0000","last_active":"2024-05-08T05:51:24.618463+0000","last_peered":"2024-05-08T05:51:24.618463+0000","last_clean":"2024-05-08T05:51:24.618463+0000","last_became_active":"2024-05-08T05:51:00.258179+0000","last_became_peered":"2024-05-08T05:51:00.258179+0000","last_unstale":"2024-05-08T05:51:24.618463+0000","last_undegraded":"2024-05-08T05:51:24.618463+0000","last_fullsized":"2024-05-08T05:51:24.618463+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:22.376499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333753+0000","last_change":"2024-05-08T05:51:04.310947+0000","last_active":"2024-05-08T05:51:07.333753+0000","last_peered":"2024-05-08T05:51:07.333753+0000","last_clean":"2024-05-08T05:51:07.333753+0000","last_became_active":"2024-05-08T05:51:04.310378+0000","last_became_peered":"2024-05-08T05:51:04.310378+0000","last_unstale":"2024-05-08T05:51:07.333753+0000","last_undegraded":"2024-05-08T05:51:07.333753+0000","last_fullsized":"2024-05-08T05:51:07.333753+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:19.367660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:23.599416+0000","last_change":"2024-05-08T05:51:00.271104+0000","last_active":"2024-05-08T05:51:23.599416+0000","last_peered":"2024-05-08T05:51:23.599416+0000","last_clean":"2024-05-08T05:51:23.599416+0000","last_became_active":"2024-05-08T05:51:00.270497+0000","last_became_peered":"2024-05-08T05:51:00.270497+0000","last_unstale":"2024-05-08T05:51:23.599416+0000","last_undegraded":"2024-05-08T05:51:23.599416+0000","last_fullsized":"2024-05-08T05:51:23.599416+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:40.959423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.338941+0000","last_change":"2024-05-08T05:51:04.310243+0000","last_active":"2024-05-08T05:51:07.338941+0000","last_peered":"2024-05-08T05:51:07.338941+0000","last_clean":"2024-05-08T05:51:07.338941+0000","last_became_active":"2024-05-08T05:51:04.309871+0000","last_became_peered":"2024-05-08T05:51:04.309871+0000","last_unstale":"2024-05-08T05:51:07.338941+0000","last_undegraded":"2024-05-08T05:51:07.338941+0000","last_fullsized":"2024-05-08T05:51:07.338941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:18:21.045621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":61,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:57.690609+0000","last_change":"2024-05-08T05:51:00.266096+0000","last_active":"2024-05-08T05:51:57.690609+0000","last_peered":"2024-05-08T05:51:57.690609+0000","last_clean":"2024-05-08T05:51:57.690609+0000","last_became_active":"2024-05-08T05:51:00.265706+0000","last_became_peered":"2024-05-08T05:51:00.265706+0000","last_unstale":"2024-05-08T05:51:57.690609+0000","last_undegraded":"2024-05-08T05:51:57.690609+0000","last_fullsized":"2024-05-08T05:51:57.690609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:47.388732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.337758+0000","last_change":"2024-05-08T05:51:04.293141+0000","last_active":"2024-05-08T05:51:07.337758+0000","last_peered":"2024-05-08T05:51:07.337758+0000","last_clean":"2024-05-08T05:51:07.337758+0000","last_became_active":"2024-05-08T05:51:04.292772+0000","last_became_peered":"2024-05-08T05:51:04.292772+0000","last_unstale":"2024-05-08T05:51:07.337758+0000","last_undegraded":"2024-05-08T05:51:07.337758+0000","last_fullsized":"2024-05-08T05:51:07.337758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:25:32.801209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:24.621645+0000","last_change":"2024-05-08T05:51:01.186243+0000","last_active":"2024-05-08T05:51:24.621645+0000","last_peered":"2024-05-08T05:51:24.621645+0000","last_clean":"2024-05-08T05:51:24.621645+0000","last_became_active":"2024-05-08T05:51:01.185758+0000","last_became_peered":"2024-05-08T05:51:01.185758+0000","last_unstale":"2024-05-08T05:51:24.621645+0000","last_undegraded":"2024-05-08T05:51:24.621645+0000","last_fullsized":"2024-05-08T05:51:24.621645+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:40:47.946477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333051+0000","last_change":"2024-05-08T05:51:04.290096+0000","last_active":"2024-05-08T05:51:07.333051+0000","last_peered":"2024-05-08T05:51:07.333051+0000","last_clean":"2024-05-08T05:51:07.333051+0000","last_became_active":"2024-05-08T05:51:04.289684+0000","last_became_peered":"2024-05-08T05:51:04.289684+0000","last_unstale":"2024-05-08T05:51:07.333051+0000","last_undegraded":"2024-05-08T05:51:07.333051+0000","last_fullsized":"2024-05-08T05:51:07.333051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:02.400348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:25.609047+0000","last_change":"2024-05-08T05:51:00.266694+0000","last_active":"2024-05-08T05:51:25.609047+0000","last_peered":"2024-05-08T05:51:25.609047+0000","last_clean":"2024-05-08T05:51:25.609047+0000","last_became_active":"2024-05-08T05:51:00.265901+0000","last_became_peered":"2024-05-08T05:51:00.265901+0000","last_unstale":"2024-05-08T05:51:25.609047+0000","last_undegraded":"2024-05-08T05:51:25.609047+0000","last_fullsized":"2024-05-08T05:51:25.609047+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:46.928822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.318692+0000","last_change":"2024-05-08T05:51:04.303982+0000","last_active":"2024-05-08T05:51:07.318692+0000","last_peered":"2024-05-08T05:51:07.318692+0000","last_clean":"2024-05-08T05:51:07.318692+0000","last_became_active":"2024-05-08T05:51:04.302618+0000","last_became_peered":"2024-05-08T05:51:04.302618+0000","last_unstale":"2024-05-08T05:51:07.318692+0000","last_undegraded":"2024-05-08T05:51:07.318692+0000","last_fullsized":"2024-05-08T05:51:07.318692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:44.353183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335062+0000","last_change":"2024-05-08T05:51:01.189324+0000","last_active":"2024-05-08T05:51:07.335062+0000","last_peered":"2024-05-08T05:51:07.335062+0000","last_clean":"2024-05-08T05:51:07.335062+0000","last_became_active":"2024-05-08T05:51:01.188651+0000","last_became_peered":"2024-05-08T05:51:01.188651+0000","last_unstale":"2024-05-08T05:51:07.335062+0000","last_undegraded":"2024-05-08T05:51:07.335062+0000","last_fullsized":"2024-05-08T05:51:07.335062+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:45.149213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.318286+0000","last_change":"2024-05-08T05:51:04.306811+0000","last_active":"2024-05-08T05:51:07.318286+0000","last_peered":"2024-05-08T05:51:07.318286+0000","last_clean":"2024-05-08T05:51:07.318286+0000","last_became_active":"2024-05-08T05:51:04.306423+0000","last_became_peered":"2024-05-08T05:51:04.306423+0000","last_unstale":"2024-05-08T05:51:07.318286+0000","last_undegraded":"2024-05-08T05:51:07.318286+0000","last_fullsized":"2024-05-08T05:51:07.318286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:08.830682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.338121+0000","last_change":"2024-05-08T05:51:00.276543+0000","last_active":"2024-05-08T05:51:07.338121+0000","last_peered":"2024-05-08T05:51:07.338121+0000","last_clean":"2024-05-08T05:51:07.338121+0000","last_became_active":"2024-05-08T05:51:00.276233+0000","last_became_peered":"2024-05-08T05:51:00.276233+0000","last_unstale":"2024-05-08T05:51:07.338121+0000","last_undegraded":"2024-05-08T05:51:07.338121+0000","last_fullsized":"2024-05-08T05:51:07.338121+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:11.229942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.318032+0000","last_change":"2024-05-08T05:51:04.322033+0000","last_active":"2024-05-08T05:51:07.318032+0000","last_peered":"2024-05-08T05:51:07.318032+0000","last_clean":"2024-05-08T05:51:07.318032+0000","last_became_active":"2024-05-08T05:51:04.321555+0000","last_became_peered":"2024-05-08T05:51:04.321555+0000","last_unstale":"2024-05-08T05:51:07.318032+0000","last_undegraded":"2024-05-08T05:51:07.318032+0000","last_fullsized":"2024-05-08T05:51:07.318032+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:21:50.820709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333280+0000","last_change":"2024-05-08T05:51:04.295613+0000","last_active":"2024-05-08T05:51:07.333280+0000","last_peered":"2024-05-08T05:51:07.333280+0000","last_clean":"2024-05-08T05:51:07.333280+0000","last_became_active":"2024-05-08T05:51:04.295278+0000","last_became_peered":"2024-05-08T05:51:04.295278+0000","last_unstale":"2024-05-08T05:51:07.333280+0000","last_undegraded":"2024-05-08T05:51:07.333280+0000","last_fullsized":"2024-05-08T05:51:07.333280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:57.236920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":60,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:59.693076+0000","last_change":"2024-05-08T05:51:01.187556+0000","last_active":"2024-05-08T05:51:59.693076+0000","last_peered":"2024-05-08T05:51:59.693076+0000","last_clean":"2024-05-08T05:51:59.693076+0000","last_became_active":"2024-05-08T05:51:01.186584+0000","last_became_peered":"2024-05-08T05:51:01.186584+0000","last_unstale":"2024-05-08T05:51:59.693076+0000","last_undegraded":"2024-05-08T05:51:59.693076+0000","last_fullsized":"2024-05-08T05:51:59.693076+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:29.036633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.319744+0000","last_change":"2024-05-08T05:51:04.315916+0000","last_active":"2024-05-08T05:51:07.319744+0000","last_peered":"2024-05-08T05:51:07.319744+0000","last_clean":"2024-05-08T05:51:07.319744+0000","last_became_active":"2024-05-08T05:51:04.315180+0000","last_became_peered":"2024-05-08T05:51:04.315180+0000","last_unstale":"2024-05-08T05:51:07.319744+0000","last_undegraded":"2024-05-08T05:51:07.319744+0000","last_fullsized":"2024-05-08T05:51:07.319744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:34:50.431271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:27.632668+0000","last_change":"2024-05-08T05:51:00.272501+0000","last_active":"2024-05-08T05:51:27.632668+0000","last_peered":"2024-05-08T05:51:27.632668+0000","last_clean":"2024-05-08T05:51:27.632668+0000","last_became_active":"2024-05-08T05:51:00.272166+0000","last_became_peered":"2024-05-08T05:51:00.272166+0000","last_unstale":"2024-05-08T05:51:27.632668+0000","last_undegraded":"2024-05-08T05:51:27.632668+0000","last_fullsized":"2024-05-08T05:51:27.632668+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:30.980047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":94,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.337836+0000","last_change":"2024-05-08T05:49:31.820799+0000","last_active":"2024-05-08T05:51:07.337836+0000","last_peered":"2024-05-08T05:51:07.337836+0000","last_clean":"2024-05-08T05:51:07.337836+0000","last_became_active":"2024-05-08T05:49:31.815568+0000","last_became_peered":"2024-05-08T05:49:31.815568+0000","last_unstale":"2024-05-08T05:51:07.337836+0000","last_undegraded":"2024-05-08T05:51:07.337836+0000","last_fullsized":"2024-05-08T05:51:07.337836+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:48.170242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:48.170242+0000","last_clean_scrub_stamp":"2024-05-08T05:45:48.170242+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:39.527036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333551+0000","last_change":"2024-05-08T05:51:04.301375+0000","last_active":"2024-05-08T05:51:07.333551+0000","last_peered":"2024-05-08T05:51:07.333551+0000","last_clean":"2024-05-08T05:51:07.333551+0000","last_became_active":"2024-05-08T05:51:04.301027+0000","last_became_peered":"2024-05-08T05:51:04.301027+0000","last_unstale":"2024-05-08T05:51:07.333551+0000","last_undegraded":"2024-05-08T05:51:07.333551+0000","last_fullsized":"2024-05-08T05:51:07.333551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:21.393242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:24.612134+0000","last_change":"2024-05-08T05:51:00.266700+0000","last_active":"2024-05-08T05:51:24.612134+0000","last_peered":"2024-05-08T05:51:24.612134+0000","last_clean":"2024-05-08T05:51:24.612134+0000","last_became_active":"2024-05-08T05:51:00.266358+0000","last_became_peered":"2024-05-08T05:51:00.266358+0000","last_unstale":"2024-05-08T05:51:24.612134+0000","last_undegraded":"2024-05-08T05:51:24.612134+0000","last_fullsized":"2024-05-08T05:51:24.612134+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:28.737436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:23.602899+0000","last_change":"2024-05-08T05:51:01.187171+0000","last_active":"2024-05-08T05:51:23.602899+0000","last_peered":"2024-05-08T05:51:23.602899+0000","last_clean":"2024-05-08T05:51:23.602899+0000","last_became_active":"2024-05-08T05:51:01.186142+0000","last_became_peered":"2024-05-08T05:51:01.186142+0000","last_unstale":"2024-05-08T05:51:23.602899+0000","last_undegraded":"2024-05-08T05:51:23.602899+0000","last_fullsized":"2024-05-08T05:51:23.602899+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:37:17.021905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334317+0000","last_change":"2024-05-08T05:51:04.311538+0000","last_active":"2024-05-08T05:51:07.334317+0000","last_peered":"2024-05-08T05:51:07.334317+0000","last_clean":"2024-05-08T05:51:07.334317+0000","last_became_active":"2024-05-08T05:51:04.310535+0000","last_became_peered":"2024-05-08T05:51:04.310535+0000","last_unstale":"2024-05-08T05:51:07.334317+0000","last_undegraded":"2024-05-08T05:51:07.334317+0000","last_fullsized":"2024-05-08T05:51:07.334317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:01:04.497986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:52.339473+0000","last_change":"2024-05-08T05:51:00.277923+0000","last_active":"2024-05-08T05:51:52.339473+0000","last_peered":"2024-05-08T05:51:52.339473+0000","last_clean":"2024-05-08T05:51:52.339473+0000","last_became_active":"2024-05-08T05:51:00.274896+0000","last_became_peered":"2024-05-08T05:51:00.274896+0000","last_unstale":"2024-05-08T05:51:52.339473+0000","last_undegraded":"2024-05-08T05:51:52.339473+0000","last_fullsized":"2024-05-08T05:51:52.339473+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:04:32.543451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.320757+0000","last_change":"2024-05-08T05:51:04.306185+0000","last_active":"2024-05-08T05:51:07.320757+0000","last_peered":"2024-05-08T05:51:07.320757+0000","last_clean":"2024-05-08T05:51:07.320757+0000","last_became_active":"2024-05-08T05:51:04.305229+0000","last_became_peered":"2024-05-08T05:51:04.305229+0000","last_unstale":"2024-05-08T05:51:07.320757+0000","last_undegraded":"2024-05-08T05:51:07.320757+0000","last_fullsized":"2024-05-08T05:51:07.320757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:48.859978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:23.601212+0000","last_change":"2024-05-08T05:51:01.140178+0000","last_active":"2024-05-08T05:51:23.601212+0000","last_peered":"2024-05-08T05:51:23.601212+0000","last_clean":"2024-05-08T05:51:23.601212+0000","last_became_active":"2024-05-08T05:51:01.139772+0000","last_became_peered":"2024-05-08T05:51:01.139772+0000","last_unstale":"2024-05-08T05:51:23.601212+0000","last_undegraded":"2024-05-08T05:51:23.601212+0000","last_fullsized":"2024-05-08T05:51:23.601212+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:54:40.132683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335803+0000","last_change":"2024-05-08T05:51:04.319015+0000","last_active":"2024-05-08T05:51:07.335803+0000","last_peered":"2024-05-08T05:51:07.335803+0000","last_clean":"2024-05-08T05:51:07.335803+0000","last_became_active":"2024-05-08T05:51:04.318465+0000","last_became_peered":"2024-05-08T05:51:04.318465+0000","last_unstale":"2024-05-08T05:51:07.335803+0000","last_undegraded":"2024-05-08T05:51:07.335803+0000","last_fullsized":"2024-05-08T05:51:07.335803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:07:19.336656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:23.602608+0000","last_change":"2024-05-08T05:51:01.185619+0000","last_active":"2024-05-08T05:51:23.602608+0000","last_peered":"2024-05-08T05:51:23.602608+0000","last_clean":"2024-05-08T05:51:23.602608+0000","last_became_active":"2024-05-08T05:51:01.185325+0000","last_became_peered":"2024-05-08T05:51:01.185325+0000","last_unstale":"2024-05-08T05:51:23.602608+0000","last_undegraded":"2024-05-08T05:51:23.602608+0000","last_fullsized":"2024-05-08T05:51:23.602608+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:41.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333894+0000","last_change":"2024-05-08T05:51:04.295018+0000","last_active":"2024-05-08T05:51:07.333894+0000","last_peered":"2024-05-08T05:51:07.333894+0000","last_clean":"2024-05-08T05:51:07.333894+0000","last_became_active":"2024-05-08T05:51:04.294432+0000","last_became_peered":"2024-05-08T05:51:04.294432+0000","last_unstale":"2024-05-08T05:51:07.333894+0000","last_undegraded":"2024-05-08T05:51:07.333894+0000","last_fullsized":"2024-05-08T05:51:07.333894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:01.925456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:23.603127+0000","last_change":"2024-05-08T05:51:00.280527+0000","last_active":"2024-05-08T05:51:23.603127+0000","last_peered":"2024-05-08T05:51:23.603127+0000","last_clean":"2024-05-08T05:51:23.603127+0000","last_became_active":"2024-05-08T05:51:00.278718+0000","last_became_peered":"2024-05-08T05:51:00.278718+0000","last_unstale":"2024-05-08T05:51:23.603127+0000","last_undegraded":"2024-05-08T05:51:23.603127+0000","last_fullsized":"2024-05-08T05:51:23.603127+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:48:27.149713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.318355+0000","last_change":"2024-05-08T05:51:04.314543+0000","last_active":"2024-05-08T05:51:07.318355+0000","last_peered":"2024-05-08T05:51:07.318355+0000","last_clean":"2024-05-08T05:51:07.318355+0000","last_became_active":"2024-05-08T05:51:04.314168+0000","last_became_peered":"2024-05-08T05:51:04.314168+0000","last_unstale":"2024-05-08T05:51:07.318355+0000","last_undegraded":"2024-05-08T05:51:07.318355+0000","last_fullsized":"2024-05-08T05:51:07.318355+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:10:23.215322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334501+0000","last_change":"2024-05-08T05:51:00.268081+0000","last_active":"2024-05-08T05:51:07.334501+0000","last_peered":"2024-05-08T05:51:07.334501+0000","last_clean":"2024-05-08T05:51:07.334501+0000","last_became_active":"2024-05-08T05:51:00.267470+0000","last_became_peered":"2024-05-08T05:51:00.267470+0000","last_unstale":"2024-05-08T05:51:07.334501+0000","last_undegraded":"2024-05-08T05:51:07.334501+0000","last_fullsized":"2024-05-08T05:51:07.334501+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:45.796019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.320351+0000","last_change":"2024-05-08T05:51:04.322010+0000","last_active":"2024-05-08T05:51:07.320351+0000","last_peered":"2024-05-08T05:51:07.320351+0000","last_clean":"2024-05-08T05:51:07.320351+0000","last_became_active":"2024-05-08T05:51:04.320867+0000","last_became_peered":"2024-05-08T05:51:04.320867+0000","last_unstale":"2024-05-08T05:51:07.320351+0000","last_undegraded":"2024-05-08T05:51:07.320351+0000","last_fullsized":"2024-05-08T05:51:07.320351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:49.853518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:23.598607+0000","last_change":"2024-05-08T05:51:00.269378+0000","last_active":"2024-05-08T05:51:23.598607+0000","last_peered":"2024-05-08T05:51:23.598607+0000","last_clean":"2024-05-08T05:51:23.598607+0000","last_became_active":"2024-05-08T05:51:00.268992+0000","last_became_peered":"2024-05-08T05:51:00.268992+0000","last_unstale":"2024-05-08T05:51:23.598607+0000","last_undegraded":"2024-05-08T05:51:23.598607+0000","last_fullsized":"2024-05-08T05:51:23.598607+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:21.722501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.338136+0000","last_change":"2024-05-08T05:51:04.302813+0000","last_active":"2024-05-08T05:51:07.338136+0000","last_peered":"2024-05-08T05:51:07.338136+0000","last_clean":"2024-05-08T05:51:07.338136+0000","last_became_active":"2024-05-08T05:51:04.302424+0000","last_became_peered":"2024-05-08T05:51:04.302424+0000","last_unstale":"2024-05-08T05:51:07.338136+0000","last_undegraded":"2024-05-08T05:51:07.338136+0000","last_fullsized":"2024-05-08T05:51:07.338136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:52:17.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:24.622617+0000","last_change":"2024-05-08T05:51:01.187513+0000","last_active":"2024-05-08T05:51:24.622617+0000","last_peered":"2024-05-08T05:51:24.622617+0000","last_clean":"2024-05-08T05:51:24.622617+0000","last_became_active":"2024-05-08T05:51:01.186780+0000","last_became_peered":"2024-05-08T05:51:01.186780+0000","last_unstale":"2024-05-08T05:51:24.622617+0000","last_undegraded":"2024-05-08T05:51:24.622617+0000","last_fullsized":"2024-05-08T05:51:24.622617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:42:26.206351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.339089+0000","last_change":"2024-05-08T05:51:04.295338+0000","last_active":"2024-05-08T05:51:07.339089+0000","last_peered":"2024-05-08T05:51:07.339089+0000","last_clean":"2024-05-08T05:51:07.339089+0000","last_became_active":"2024-05-08T05:51:04.294990+0000","last_became_peered":"2024-05-08T05:51:04.294990+0000","last_unstale":"2024-05-08T05:51:07.339089+0000","last_undegraded":"2024-05-08T05:51:07.339089+0000","last_fullsized":"2024-05-08T05:51:07.339089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:29.411662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.339521+0000","last_change":"2024-05-08T05:51:00.273052+0000","last_active":"2024-05-08T05:51:07.339521+0000","last_peered":"2024-05-08T05:51:07.339521+0000","last_clean":"2024-05-08T05:51:07.339521+0000","last_became_active":"2024-05-08T05:51:00.272718+0000","last_became_peered":"2024-05-08T05:51:00.272718+0000","last_unstale":"2024-05-08T05:51:07.339521+0000","last_undegraded":"2024-05-08T05:51:07.339521+0000","last_fullsized":"2024-05-08T05:51:07.339521+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:44.494378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.338100+0000","last_change":"2024-05-08T05:51:04.306209+0000","last_active":"2024-05-08T05:51:07.338100+0000","last_peered":"2024-05-08T05:51:07.338100+0000","last_clean":"2024-05-08T05:51:07.338100+0000","last_became_active":"2024-05-08T05:51:04.305752+0000","last_became_peered":"2024-05-08T05:51:04.305752+0000","last_unstale":"2024-05-08T05:51:07.338100+0000","last_undegraded":"2024-05-08T05:51:07.338100+0000","last_fullsized":"2024-05-08T05:51:07.338100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:24.586111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.339421+0000","last_change":"2024-05-08T05:51:00.278426+0000","last_active":"2024-05-08T05:51:07.339421+0000","last_peered":"2024-05-08T05:51:07.339421+0000","last_clean":"2024-05-08T05:51:07.339421+0000","last_became_active":"2024-05-08T05:51:00.276764+0000","last_became_peered":"2024-05-08T05:51:00.276764+0000","last_unstale":"2024-05-08T05:51:07.339421+0000","last_undegraded":"2024-05-08T05:51:07.339421+0000","last_fullsized":"2024-05-08T05:51:07.339421+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:34.158628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.338599+0000","last_change":"2024-05-08T05:51:04.294511+0000","last_active":"2024-05-08T05:51:07.338599+0000","last_peered":"2024-05-08T05:51:07.338599+0000","last_clean":"2024-05-08T05:51:07.338599+0000","last_became_active":"2024-05-08T05:51:04.294145+0000","last_became_peered":"2024-05-08T05:51:04.294145+0000","last_unstale":"2024-05-08T05:51:07.338599+0000","last_undegraded":"2024-05-08T05:51:07.338599+0000","last_fullsized":"2024-05-08T05:51:07.338599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:10.034060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.318548+0000","last_change":"2024-05-08T05:51:04.298905+0000","last_active":"2024-05-08T05:51:07.318548+0000","last_peered":"2024-05-08T05:51:07.318548+0000","last_clean":"2024-05-08T05:51:07.318548+0000","last_became_active":"2024-05-08T05:51:04.298586+0000","last_became_peered":"2024-05-08T05:51:04.298586+0000","last_unstale":"2024-05-08T05:51:07.318548+0000","last_undegraded":"2024-05-08T05:51:07.318548+0000","last_fullsized":"2024-05-08T05:51:07.318548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:43:24.899015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334269+0000","last_change":"2024-05-08T05:51:04.300503+0000","last_active":"2024-05-08T05:51:07.334269+0000","last_peered":"2024-05-08T05:51:07.334269+0000","last_clean":"2024-05-08T05:51:07.334269+0000","last_became_active":"2024-05-08T05:51:04.299771+0000","last_became_peered":"2024-05-08T05:51:04.299771+0000","last_unstale":"2024-05-08T05:51:07.334269+0000","last_undegraded":"2024-05-08T05:51:07.334269+0000","last_fullsized":"2024-05-08T05:51:07.334269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:19.790501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333975+0000","last_change":"2024-05-08T05:51:04.320492+0000","last_active":"2024-05-08T05:51:07.333975+0000","last_peered":"2024-05-08T05:51:07.333975+0000","last_clean":"2024-05-08T05:51:07.333975+0000","last_became_active":"2024-05-08T05:51:04.319996+0000","last_became_peered":"2024-05-08T05:51:04.319996+0000","last_unstale":"2024-05-08T05:51:07.333975+0000","last_undegraded":"2024-05-08T05:51:07.333975+0000","last_fullsized":"2024-05-08T05:51:07.333975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:10:33.874558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335111+0000","last_change":"2024-05-08T05:51:04.318604+0000","last_active":"2024-05-08T05:51:07.335111+0000","last_peered":"2024-05-08T05:51:07.335111+0000","last_clean":"2024-05-08T05:51:07.335111+0000","last_became_active":"2024-05-08T05:51:04.318133+0000","last_became_peered":"2024-05-08T05:51:04.318133+0000","last_unstale":"2024-05-08T05:51:07.335111+0000","last_undegraded":"2024-05-08T05:51:07.335111+0000","last_fullsized":"2024-05-08T05:51:07.335111+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:30:47.146559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.336110+0000","last_change":"2024-05-08T05:51:04.315769+0000","last_active":"2024-05-08T05:51:07.336110+0000","last_peered":"2024-05-08T05:51:07.336110+0000","last_clean":"2024-05-08T05:51:07.336110+0000","last_became_active":"2024-05-08T05:51:04.315322+0000","last_became_peered":"2024-05-08T05:51:04.315322+0000","last_unstale":"2024-05-08T05:51:07.336110+0000","last_undegraded":"2024-05-08T05:51:07.336110+0000","last_fullsized":"2024-05-08T05:51:07.336110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:52.096556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.318741+0000","last_change":"2024-05-08T05:51:04.298128+0000","last_active":"2024-05-08T05:51:07.318741+0000","last_peered":"2024-05-08T05:51:07.318741+0000","last_clean":"2024-05-08T05:51:07.318741+0000","last_became_active":"2024-05-08T05:51:04.297743+0000","last_became_peered":"2024-05-08T05:51:04.297743+0000","last_unstale":"2024-05-08T05:51:07.318741+0000","last_undegraded":"2024-05-08T05:51:07.318741+0000","last_fullsized":"2024-05-08T05:51:07.318741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:18.334026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334522+0000","last_change":"2024-05-08T05:51:04.301056+0000","last_active":"2024-05-08T05:51:07.334522+0000","last_peered":"2024-05-08T05:51:07.334522+0000","last_clean":"2024-05-08T05:51:07.334522+0000","last_became_active":"2024-05-08T05:51:04.300733+0000","last_became_peered":"2024-05-08T05:51:04.300733+0000","last_unstale":"2024-05-08T05:51:07.334522+0000","last_undegraded":"2024-05-08T05:51:07.334522+0000","last_fullsized":"2024-05-08T05:51:07.334522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:21.616867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.320699+0000","last_change":"2024-05-08T05:51:04.299079+0000","last_active":"2024-05-08T05:51:07.320699+0000","last_peered":"2024-05-08T05:51:07.320699+0000","last_clean":"2024-05-08T05:51:07.320699+0000","last_became_active":"2024-05-08T05:51:04.298628+0000","last_became_peered":"2024-05-08T05:51:04.298628+0000","last_unstale":"2024-05-08T05:51:07.320699+0000","last_undegraded":"2024-05-08T05:51:07.320699+0000","last_fullsized":"2024-05-08T05:51:07.320699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:45.031695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.332904+0000","last_change":"2024-05-08T05:51:04.296140+0000","last_active":"2024-05-08T05:51:07.332904+0000","last_peered":"2024-05-08T05:51:07.332904+0000","last_clean":"2024-05-08T05:51:07.332904+0000","last_became_active":"2024-05-08T05:51:04.294940+0000","last_became_peered":"2024-05-08T05:51:04.294940+0000","last_unstale":"2024-05-08T05:51:07.332904+0000","last_undegraded":"2024-05-08T05:51:07.332904+0000","last_fullsized":"2024-05-08T05:51:07.332904+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:37.192639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.319272+0000","last_change":"2024-05-08T05:51:04.305803+0000","last_active":"2024-05-08T05:51:07.319272+0000","last_peered":"2024-05-08T05:51:07.319272+0000","last_clean":"2024-05-08T05:51:07.319272+0000","last_became_active":"2024-05-08T05:51:04.305482+0000","last_became_peered":"2024-05-08T05:51:04.305482+0000","last_unstale":"2024-05-08T05:51:07.319272+0000","last_undegraded":"2024-05-08T05:51:07.319272+0000","last_fullsized":"2024-05-08T05:51:07.319272+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:02.050153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334930+0000","last_change":"2024-05-08T05:51:04.293787+0000","last_active":"2024-05-08T05:51:07.334930+0000","last_peered":"2024-05-08T05:51:07.334930+0000","last_clean":"2024-05-08T05:51:07.334930+0000","last_became_active":"2024-05-08T05:51:04.293455+0000","last_became_peered":"2024-05-08T05:51:04.293455+0000","last_unstale":"2024-05-08T05:51:07.334930+0000","last_undegraded":"2024-05-08T05:51:07.334930+0000","last_fullsized":"2024-05-08T05:51:07.334930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:09:53.211411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334467+0000","last_change":"2024-05-08T05:51:04.321442+0000","last_active":"2024-05-08T05:51:07.334467+0000","last_peered":"2024-05-08T05:51:07.334467+0000","last_clean":"2024-05-08T05:51:07.334467+0000","last_became_active":"2024-05-08T05:51:04.320982+0000","last_became_peered":"2024-05-08T05:51:04.320982+0000","last_unstale":"2024-05-08T05:51:07.334467+0000","last_undegraded":"2024-05-08T05:51:07.334467+0000","last_fullsized":"2024-05-08T05:51:07.334467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:45.089609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.318407+0000","last_change":"2024-05-08T05:51:04.294866+0000","last_active":"2024-05-08T05:51:07.318407+0000","last_peered":"2024-05-08T05:51:07.318407+0000","last_clean":"2024-05-08T05:51:07.318407+0000","last_became_active":"2024-05-08T05:51:04.294504+0000","last_became_peered":"2024-05-08T05:51:04.294504+0000","last_unstale":"2024-05-08T05:51:07.318407+0000","last_undegraded":"2024-05-08T05:51:07.318407+0000","last_fullsized":"2024-05-08T05:51:07.318407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:33.139864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65751,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":222,"num_read_kb":181,"num_write":165,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":192996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645339,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":864256,"data_stored":418312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841567,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1071934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005093,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":418548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201323,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":880640,"data_stored":418468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430256,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28456,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716696,"statfs":{"total":95995035648,"available":95965896704,"internally_reserved":0,"allocated":1474560,"data_stored":1010398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593782,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":417937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":479480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":419208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":476573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":418248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986241,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":418765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790014,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36788,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708364,"statfs":{"total":95995035648,"available":95957364736,"internally_reserved":0,"allocated":1486848,"data_stored":1009691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:52:02.850 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:52:03.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:02 smithi112 ceph-mon[30003]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T05:52:03.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:02 smithi112 ceph-mon[30003]: from='client.24658 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:52:03.302 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:03.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:02 smithi162 ceph-mon[34800]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T05:52:03.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:02 smithi162 ceph-mon[34800]: from='client.24658 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:52:03.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:02 smithi150 ceph-mon[34315]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T05:52:03.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:02 smithi150 ceph-mon[34315]: from='client.24658 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:52:04.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]: dispatch 2024-05-08T05:52:04.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-05-08T05:52:04.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-08T05:52:04.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-05-08T05:52:04.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-05-08T05:52:04.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-05-08T05:52:04.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-05-08T05:52:04.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-05-08T05:52:04.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-05-08T05:52:04.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]: dispatch 2024-05-08T05:52:04.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-05-08T05:52:04.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-08T05:52:04.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-05-08T05:52:04.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-05-08T05:52:04.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-05-08T05:52:04.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-05-08T05:52:04.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-05-08T05:52:04.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-05-08T05:52:04.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]: dispatch 2024-05-08T05:52:04.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-05-08T05:52:04.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-08T05:52:04.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-05-08T05:52:04.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-05-08T05:52:04.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-05-08T05:52:04.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-05-08T05:52:04.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-05-08T05:52:04.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-05-08T05:52:05.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:05 smithi112 ceph-mon[30003]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-08T05:52:05.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:05 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]': finished 2024-05-08T05:52:05.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:05 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-05-08T05:52:05.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:05 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-08T05:52:05.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:05 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-05-08T05:52:05.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:05 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-05-08T05:52:05.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:05 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-05-08T05:52:05.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:05 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-05-08T05:52:05.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:05 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-05-08T05:52:05.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:05 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-05-08T05:52:05.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:05 smithi112 ceph-mon[30003]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T05:52:05.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:05 smithi162 ceph-mon[34800]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-08T05:52:05.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:05 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]': finished 2024-05-08T05:52:05.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:05 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-05-08T05:52:05.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:05 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-08T05:52:05.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:05 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-05-08T05:52:05.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:05 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-05-08T05:52:05.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:05 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-05-08T05:52:05.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:05 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-05-08T05:52:05.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:05 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-05-08T05:52:05.319 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:05 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-05-08T05:52:05.319 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:05 smithi162 ceph-mon[34800]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T05:52:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:05 smithi150 ceph-mon[34315]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-08T05:52:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]': finished 2024-05-08T05:52:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-05-08T05:52:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-08T05:52:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-05-08T05:52:05.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-05-08T05:52:05.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-05-08T05:52:05.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-05-08T05:52:05.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-05-08T05:52:05.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-05-08T05:52:05.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:05 smithi150 ceph-mon[34315]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T05:52:05.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.641+0000 7fe76e200700 1 -- 172.21.15.112:0/4207061716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe768109ad0 msgr2=0x7fe768109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:05.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.641+0000 7fe76e200700 1 --2- 172.21.15.112:0/4207061716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe768109ad0 0x7fe768109f50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe754009990 tx=0x7fe754009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:05.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.641+0000 7fe76e200700 1 -- 172.21.15.112:0/4207061716 shutdown_connections 2024-05-08T05:52:05.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.641+0000 7fe76e200700 1 --2- 172.21.15.112:0/4207061716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe76810a490 0x7fe7681169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:05.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.641+0000 7fe76e200700 1 --2- 172.21.15.112:0/4207061716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe76810fb60 0x7fe76810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:05.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.641+0000 7fe76e200700 1 --2- 172.21.15.112:0/4207061716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe768109ad0 0x7fe768109f50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:05.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.641+0000 7fe76e200700 1 -- 172.21.15.112:0/4207061716 >> 172.21.15.112:0/4207061716 conn(0x7fe768076c70 msgr2=0x7fe7681082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:05.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.641+0000 7fe76e200700 1 -- 172.21.15.112:0/4207061716 shutdown_connections 2024-05-08T05:52:05.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.641+0000 7fe76e200700 1 -- 172.21.15.112:0/4207061716 wait complete. 2024-05-08T05:52:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.642+0000 7fe76e200700 1 Processor -- start 2024-05-08T05:52:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.642+0000 7fe76e200700 1 -- start start 2024-05-08T05:52:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.643+0000 7fe76e200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe768109ad0 0x7fe7681a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.643+0000 7fe76e200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe76810a490 0x7fe7681a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.643+0000 7fe76e200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe76810fb60 0x7fe7681a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.643+0000 7fe76e200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe768119d20 con 0x7fe76810fb60 2024-05-08T05:52:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.643+0000 7fe76e200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe768119ba0 con 0x7fe768109ad0 2024-05-08T05:52:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.643+0000 7fe76e200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe768119ea0 con 0x7fe76810a490 2024-05-08T05:52:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.643+0000 7fe767e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe76810fb60 0x7fe7681a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.643+0000 7fe767e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe76810fb60 0x7fe7681a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58088/0 (socket says 172.21.15.112:58088) 2024-05-08T05:52:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.643+0000 7fe767e00700 1 -- 172.21.15.112:0/789687298 learned_addr learned my addr 172.21.15.112:0/789687298 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.643+0000 7fe767400700 1 --2- 172.21.15.112:0/789687298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe768109ad0 0x7fe7681a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.643+0000 7fe766a00700 1 --2- 172.21.15.112:0/789687298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe76810a490 0x7fe7681a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.644+0000 7fe767e00700 1 -- 172.21.15.112:0/789687298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe76810a490 msgr2=0x7fe7681a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.644+0000 7fe767e00700 1 --2- 172.21.15.112:0/789687298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe76810a490 0x7fe7681a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.644+0000 7fe767e00700 1 -- 172.21.15.112:0/789687298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe768109ad0 msgr2=0x7fe7681a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.644+0000 7fe767e00700 1 --2- 172.21.15.112:0/789687298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe768109ad0 0x7fe7681a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.644+0000 7fe767e00700 1 -- 172.21.15.112:0/789687298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7681a52c0 con 0x7fe76810fb60 2024-05-08T05:52:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.644+0000 7fe767e00700 1 --2- 172.21.15.112:0/789687298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe76810fb60 0x7fe7681a4b80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe760007ee0 tx=0x7fe76000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:05.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.644+0000 7fe75fe00700 1 -- 172.21.15.112:0/789687298 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760004bb0 con 0x7fe76810fb60 2024-05-08T05:52:05.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.644+0000 7fe75fe00700 1 -- 172.21.15.112:0/789687298 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe760004d50 con 0x7fe76810fb60 2024-05-08T05:52:05.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.644+0000 7fe75fe00700 1 -- 172.21.15.112:0/789687298 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760005920 con 0x7fe76810fb60 2024-05-08T05:52:05.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.644+0000 7fe76e200700 1 -- 172.21.15.112:0/789687298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7681a55b0 con 0x7fe76810fb60 2024-05-08T05:52:05.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.644+0000 7fe76e200700 1 -- 172.21.15.112:0/789687298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7681ace30 con 0x7fe76810fb60 2024-05-08T05:52:05.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.646+0000 7fe76e200700 1 -- 172.21.15.112:0/789687298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe76806a190 con 0x7fe76810fb60 2024-05-08T05:52:05.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.648+0000 7fe75fe00700 1 -- 172.21.15.112:0/789687298 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe76001a020 con 0x7fe76810fb60 2024-05-08T05:52:05.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.649+0000 7fe75fe00700 1 --2- 172.21.15.112:0/789687298 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe74c0b2990 0x7fe74c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:05.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.650+0000 7fe75fe00700 1 -- 172.21.15.112:0/789687298 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7fe7600e19c0 con 0x7fe76810fb60 2024-05-08T05:52:05.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.650+0000 7fe767400700 1 --2- 172.21.15.112:0/789687298 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe74c0b2990 0x7fe74c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:05.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.650+0000 7fe767400700 1 --2- 172.21.15.112:0/789687298 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe74c0b2990 0x7fe74c0b4e50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe758005950 tx=0x7fe75800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:05.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.654+0000 7fe75fe00700 1 -- 172.21.15.112:0/789687298 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe760010070 con 0x7fe76810fb60 2024-05-08T05:52:05.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.879+0000 7fe76e200700 1 -- 172.21.15.112:0/789687298 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe7681a1610 con 0x7fe74c0b2990 2024-05-08T05:52:05.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.892+0000 7fe75fe00700 1 -- 172.21.15.112:0/789687298 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258731 (secure 0 0 0) 0x7fe7681a1610 con 0x7fe74c0b2990 2024-05-08T05:52:05.893 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:52:05.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.898+0000 7fe76e200700 1 -- 172.21.15.112:0/789687298 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe74c0b2990 msgr2=0x7fe74c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:05.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.898+0000 7fe76e200700 1 --2- 172.21.15.112:0/789687298 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe74c0b2990 0x7fe74c0b4e50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe758005950 tx=0x7fe75800b410 comp rx=0 tx=0).stop 2024-05-08T05:52:05.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.898+0000 7fe76e200700 1 -- 172.21.15.112:0/789687298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe76810fb60 msgr2=0x7fe7681a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:05.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.898+0000 7fe76e200700 1 --2- 172.21.15.112:0/789687298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe76810fb60 0x7fe7681a4b80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe760007ee0 tx=0x7fe76000d3b0 comp rx=0 tx=0).stop 2024-05-08T05:52:05.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.899+0000 7fe76e200700 1 -- 172.21.15.112:0/789687298 shutdown_connections 2024-05-08T05:52:05.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.899+0000 7fe76e200700 1 --2- 172.21.15.112:0/789687298 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe74c0b2990 0x7fe74c0b4e50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:05.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.899+0000 7fe76e200700 1 --2- 172.21.15.112:0/789687298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe76810fb60 0x7fe7681a4b80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:05.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.899+0000 7fe76e200700 1 --2- 172.21.15.112:0/789687298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe768109ad0 0x7fe7681a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:05.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.899+0000 7fe76e200700 1 --2- 172.21.15.112:0/789687298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe76810a490 0x7fe7681a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:05.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.899+0000 7fe76e200700 1 -- 172.21.15.112:0/789687298 >> 172.21.15.112:0/789687298 conn(0x7fe768076c70 msgr2=0x7fe768077330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:05.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.899+0000 7fe76e200700 1 -- 172.21.15.112:0/789687298 shutdown_connections 2024-05-08T05:52:05.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:05.899+0000 7fe76e200700 1 -- 172.21.15.112:0/789687298 wait complete. 2024-05-08T05:52:05.899 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-05-08T05:52:06.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:06 smithi112 ceph-mon[30003]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T05:52:06.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:06 smithi162 ceph-mon[34800]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T05:52:06.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:06 smithi150 ceph-mon[34315]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T05:52:06.817 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-05-08T05:52:05.490653+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656353,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":322,"num_read_kb":254,"num_write":290,"num_write_kb":2264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361444,"kb_used_data":12284,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580380,"statfs":{"total":1151940427776,"available":1151570309120,"internally_reserved":0,"allocated":12578816,"data_stored":6999253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4257,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":29,"num_write":3,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002503"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.021426+0000","last_change":"2024-05-08T05:51:04.320319+0000","last_active":"2024-05-08T05:52:05.021426+0000","last_peered":"2024-05-08T05:52:05.021426+0000","last_clean":"2024-05-08T05:52:05.021426+0000","last_became_active":"2024-05-08T05:51:04.319961+0000","last_became_peered":"2024-05-08T05:51:04.319961+0000","last_unstale":"2024-05-08T05:52:05.021426+0000","last_undegraded":"2024-05-08T05:52:05.021426+0000","last_fullsized":"2024-05-08T05:52:05.021426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:32.463261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335395+0000","last_change":"2024-05-08T05:51:04.315531+0000","last_active":"2024-05-08T05:51:07.335395+0000","last_peered":"2024-05-08T05:51:07.335395+0000","last_clean":"2024-05-08T05:51:07.335395+0000","last_became_active":"2024-05-08T05:51:04.314116+0000","last_became_peered":"2024-05-08T05:51:04.314116+0000","last_unstale":"2024-05-08T05:51:07.335395+0000","last_undegraded":"2024-05-08T05:51:07.335395+0000","last_fullsized":"2024-05-08T05:51:07.335395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:12:03.505888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335448+0000","last_change":"2024-05-08T05:51:04.315826+0000","last_active":"2024-05-08T05:51:07.335448+0000","last_peered":"2024-05-08T05:51:07.335448+0000","last_clean":"2024-05-08T05:51:07.335448+0000","last_became_active":"2024-05-08T05:51:04.315460+0000","last_became_peered":"2024-05-08T05:51:04.315460+0000","last_unstale":"2024-05-08T05:51:07.335448+0000","last_undegraded":"2024-05-08T05:51:07.335448+0000","last_fullsized":"2024-05-08T05:51:07.335448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:22:20.412368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333562+0000","last_change":"2024-05-08T05:51:04.297401+0000","last_active":"2024-05-08T05:51:07.333562+0000","last_peered":"2024-05-08T05:51:07.333562+0000","last_clean":"2024-05-08T05:51:07.333562+0000","last_became_active":"2024-05-08T05:51:04.297061+0000","last_became_peered":"2024-05-08T05:51:04.297061+0000","last_unstale":"2024-05-08T05:51:07.333562+0000","last_undegraded":"2024-05-08T05:51:07.333562+0000","last_fullsized":"2024-05-08T05:51:07.333562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:27.967517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334225+0000","last_change":"2024-05-08T05:51:04.315069+0000","last_active":"2024-05-08T05:51:07.334225+0000","last_peered":"2024-05-08T05:51:07.334225+0000","last_clean":"2024-05-08T05:51:07.334225+0000","last_became_active":"2024-05-08T05:51:04.314513+0000","last_became_peered":"2024-05-08T05:51:04.314513+0000","last_unstale":"2024-05-08T05:51:07.334225+0000","last_undegraded":"2024-05-08T05:51:07.334225+0000","last_fullsized":"2024-05-08T05:51:07.334225+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:02.520130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.016690+0000","last_change":"2024-05-08T05:51:04.302851+0000","last_active":"2024-05-08T05:52:04.016690+0000","last_peered":"2024-05-08T05:52:04.016690+0000","last_clean":"2024-05-08T05:52:04.016690+0000","last_became_active":"2024-05-08T05:51:04.302256+0000","last_became_peered":"2024-05-08T05:51:04.302256+0000","last_unstale":"2024-05-08T05:52:04.016690+0000","last_undegraded":"2024-05-08T05:52:04.016690+0000","last_fullsized":"2024-05-08T05:52:04.016690+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:45.587319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.023054+0000","last_change":"2024-05-08T05:51:04.308674+0000","last_active":"2024-05-08T05:52:05.023054+0000","last_peered":"2024-05-08T05:52:05.023054+0000","last_clean":"2024-05-08T05:52:05.023054+0000","last_became_active":"2024-05-08T05:51:04.308295+0000","last_became_peered":"2024-05-08T05:51:04.308295+0000","last_unstale":"2024-05-08T05:52:05.023054+0000","last_undegraded":"2024-05-08T05:52:05.023054+0000","last_fullsized":"2024-05-08T05:52:05.023054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:59:17.561655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.319096+0000","last_change":"2024-05-08T05:51:04.314511+0000","last_active":"2024-05-08T05:51:07.319096+0000","last_peered":"2024-05-08T05:51:07.319096+0000","last_clean":"2024-05-08T05:51:07.319096+0000","last_became_active":"2024-05-08T05:51:04.314082+0000","last_became_peered":"2024-05-08T05:51:04.314082+0000","last_unstale":"2024-05-08T05:51:07.319096+0000","last_undegraded":"2024-05-08T05:51:07.319096+0000","last_fullsized":"2024-05-08T05:51:07.319096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:24.182391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335715+0000","last_change":"2024-05-08T05:51:04.293745+0000","last_active":"2024-05-08T05:51:07.335715+0000","last_peered":"2024-05-08T05:51:07.335715+0000","last_clean":"2024-05-08T05:51:07.335715+0000","last_became_active":"2024-05-08T05:51:04.293334+0000","last_became_peered":"2024-05-08T05:51:04.293334+0000","last_unstale":"2024-05-08T05:51:07.335715+0000","last_undegraded":"2024-05-08T05:51:07.335715+0000","last_fullsized":"2024-05-08T05:51:07.335715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:16.576967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.016832+0000","last_change":"2024-05-08T05:51:04.297800+0000","last_active":"2024-05-08T05:52:04.016832+0000","last_peered":"2024-05-08T05:52:04.016832+0000","last_clean":"2024-05-08T05:52:04.016832+0000","last_became_active":"2024-05-08T05:51:04.297349+0000","last_became_peered":"2024-05-08T05:51:04.297349+0000","last_unstale":"2024-05-08T05:52:04.016832+0000","last_undegraded":"2024-05-08T05:52:04.016832+0000","last_fullsized":"2024-05-08T05:52:04.016832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:55.784512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.024228+0000","last_change":"2024-05-08T05:51:04.297997+0000","last_active":"2024-05-08T05:52:05.024228+0000","last_peered":"2024-05-08T05:52:05.024228+0000","last_clean":"2024-05-08T05:52:05.024228+0000","last_became_active":"2024-05-08T05:51:04.297516+0000","last_became_peered":"2024-05-08T05:51:04.297516+0000","last_unstale":"2024-05-08T05:52:05.024228+0000","last_undegraded":"2024-05-08T05:52:05.024228+0000","last_fullsized":"2024-05-08T05:52:05.024228+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:25.991555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333377+0000","last_change":"2024-05-08T05:51:04.295610+0000","last_active":"2024-05-08T05:51:07.333377+0000","last_peered":"2024-05-08T05:51:07.333377+0000","last_clean":"2024-05-08T05:51:07.333377+0000","last_became_active":"2024-05-08T05:51:04.294553+0000","last_became_peered":"2024-05-08T05:51:04.294553+0000","last_unstale":"2024-05-08T05:51:07.333377+0000","last_undegraded":"2024-05-08T05:51:07.333377+0000","last_fullsized":"2024-05-08T05:51:07.333377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:40:14.492057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335603+0000","last_change":"2024-05-08T05:51:04.303879+0000","last_active":"2024-05-08T05:51:07.335603+0000","last_peered":"2024-05-08T05:51:07.335603+0000","last_clean":"2024-05-08T05:51:07.335603+0000","last_became_active":"2024-05-08T05:51:04.303423+0000","last_became_peered":"2024-05-08T05:51:04.303423+0000","last_unstale":"2024-05-08T05:51:07.335603+0000","last_undegraded":"2024-05-08T05:51:07.335603+0000","last_fullsized":"2024-05-08T05:51:07.335603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:50:55.905858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334644+0000","last_change":"2024-05-08T05:51:04.309818+0000","last_active":"2024-05-08T05:51:07.334644+0000","last_peered":"2024-05-08T05:51:07.334644+0000","last_clean":"2024-05-08T05:51:07.334644+0000","last_became_active":"2024-05-08T05:51:04.309434+0000","last_became_peered":"2024-05-08T05:51:04.309434+0000","last_unstale":"2024-05-08T05:51:07.334644+0000","last_undegraded":"2024-05-08T05:51:07.334644+0000","last_fullsized":"2024-05-08T05:51:07.334644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:55.832239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334819+0000","last_change":"2024-05-08T05:51:04.316708+0000","last_active":"2024-05-08T05:51:07.334819+0000","last_peered":"2024-05-08T05:51:07.334819+0000","last_clean":"2024-05-08T05:51:07.334819+0000","last_became_active":"2024-05-08T05:51:04.316525+0000","last_became_peered":"2024-05-08T05:51:04.316525+0000","last_unstale":"2024-05-08T05:51:07.334819+0000","last_undegraded":"2024-05-08T05:51:07.334819+0000","last_fullsized":"2024-05-08T05:51:07.334819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:33:38.246611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335158+0000","last_change":"2024-05-08T05:51:04.314780+0000","last_active":"2024-05-08T05:51:07.335158+0000","last_peered":"2024-05-08T05:51:07.335158+0000","last_clean":"2024-05-08T05:51:07.335158+0000","last_became_active":"2024-05-08T05:51:04.314406+0000","last_became_peered":"2024-05-08T05:51:04.314406+0000","last_unstale":"2024-05-08T05:51:07.335158+0000","last_undegraded":"2024-05-08T05:51:07.335158+0000","last_fullsized":"2024-05-08T05:51:07.335158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:39:26.926238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.021323+0000","last_change":"2024-05-08T05:51:04.319425+0000","last_active":"2024-05-08T05:52:05.021323+0000","last_peered":"2024-05-08T05:52:05.021323+0000","last_clean":"2024-05-08T05:52:05.021323+0000","last_became_active":"2024-05-08T05:51:04.319106+0000","last_became_peered":"2024-05-08T05:51:04.319106+0000","last_unstale":"2024-05-08T05:52:05.021323+0000","last_undegraded":"2024-05-08T05:52:05.021323+0000","last_fullsized":"2024-05-08T05:52:05.021323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:59:54.023162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333643+0000","last_change":"2024-05-08T05:51:04.303547+0000","last_active":"2024-05-08T05:51:07.333643+0000","last_peered":"2024-05-08T05:51:07.333643+0000","last_clean":"2024-05-08T05:51:07.333643+0000","last_became_active":"2024-05-08T05:51:04.303182+0000","last_became_peered":"2024-05-08T05:51:04.303182+0000","last_unstale":"2024-05-08T05:51:07.333643+0000","last_undegraded":"2024-05-08T05:51:07.333643+0000","last_fullsized":"2024-05-08T05:51:07.333643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:11:59.624329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334261+0000","last_change":"2024-05-08T05:51:04.315674+0000","last_active":"2024-05-08T05:51:07.334261+0000","last_peered":"2024-05-08T05:51:07.334261+0000","last_clean":"2024-05-08T05:51:07.334261+0000","last_became_active":"2024-05-08T05:51:04.314911+0000","last_became_peered":"2024-05-08T05:51:04.314911+0000","last_unstale":"2024-05-08T05:51:07.334261+0000","last_undegraded":"2024-05-08T05:51:07.334261+0000","last_fullsized":"2024-05-08T05:51:07.334261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:49.501508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:24.624525+0000","last_change":"2024-05-08T05:51:00.273554+0000","last_active":"2024-05-08T05:51:24.624525+0000","last_peered":"2024-05-08T05:51:24.624525+0000","last_clean":"2024-05-08T05:51:24.624525+0000","last_became_active":"2024-05-08T05:51:00.272858+0000","last_became_peered":"2024-05-08T05:51:00.272858+0000","last_unstale":"2024-05-08T05:51:24.624525+0000","last_undegraded":"2024-05-08T05:51:24.624525+0000","last_fullsized":"2024-05-08T05:51:24.624525+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:47:37.418887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.320427+0000","last_change":"2024-05-08T05:51:04.306392+0000","last_active":"2024-05-08T05:51:07.320427+0000","last_peered":"2024-05-08T05:51:07.320427+0000","last_clean":"2024-05-08T05:51:07.320427+0000","last_became_active":"2024-05-08T05:51:04.304248+0000","last_became_peered":"2024-05-08T05:51:04.304248+0000","last_unstale":"2024-05-08T05:51:07.320427+0000","last_undegraded":"2024-05-08T05:51:07.320427+0000","last_fullsized":"2024-05-08T05:51:07.320427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:49:26.034883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.338794+0000","last_change":"2024-05-08T05:51:01.187292+0000","last_active":"2024-05-08T05:51:07.338794+0000","last_peered":"2024-05-08T05:51:07.338794+0000","last_clean":"2024-05-08T05:51:07.338794+0000","last_became_active":"2024-05-08T05:51:01.185948+0000","last_became_peered":"2024-05-08T05:51:01.185948+0000","last_unstale":"2024-05-08T05:51:07.338794+0000","last_undegraded":"2024-05-08T05:51:07.338794+0000","last_fullsized":"2024-05-08T05:51:07.338794+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:02.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335955+0000","last_change":"2024-05-08T05:51:04.318439+0000","last_active":"2024-05-08T05:51:07.335955+0000","last_peered":"2024-05-08T05:51:07.335955+0000","last_clean":"2024-05-08T05:51:07.335955+0000","last_became_active":"2024-05-08T05:51:04.317722+0000","last_became_peered":"2024-05-08T05:51:04.317722+0000","last_unstale":"2024-05-08T05:51:07.335955+0000","last_undegraded":"2024-05-08T05:51:07.335955+0000","last_fullsized":"2024-05-08T05:51:07.335955+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:36:43.648643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:24.612064+0000","last_change":"2024-05-08T05:51:00.269684+0000","last_active":"2024-05-08T05:51:24.612064+0000","last_peered":"2024-05-08T05:51:24.612064+0000","last_clean":"2024-05-08T05:51:24.612064+0000","last_became_active":"2024-05-08T05:51:00.269329+0000","last_became_peered":"2024-05-08T05:51:00.269329+0000","last_unstale":"2024-05-08T05:51:24.612064+0000","last_undegraded":"2024-05-08T05:51:24.612064+0000","last_fullsized":"2024-05-08T05:51:24.612064+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:27:10.422994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334290+0000","last_change":"2024-05-08T05:51:04.308048+0000","last_active":"2024-05-08T05:51:07.334290+0000","last_peered":"2024-05-08T05:51:07.334290+0000","last_clean":"2024-05-08T05:51:07.334290+0000","last_became_active":"2024-05-08T05:51:04.307550+0000","last_became_peered":"2024-05-08T05:51:04.307550+0000","last_unstale":"2024-05-08T05:51:07.334290+0000","last_undegraded":"2024-05-08T05:51:07.334290+0000","last_fullsized":"2024-05-08T05:51:07.334290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:13.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.341265+0000","last_change":"2024-05-08T05:51:00.268142+0000","last_active":"2024-05-08T05:51:07.341265+0000","last_peered":"2024-05-08T05:51:07.341265+0000","last_clean":"2024-05-08T05:51:07.341265+0000","last_became_active":"2024-05-08T05:51:00.267756+0000","last_became_peered":"2024-05-08T05:51:00.267756+0000","last_unstale":"2024-05-08T05:51:07.341265+0000","last_undegraded":"2024-05-08T05:51:07.341265+0000","last_fullsized":"2024-05-08T05:51:07.341265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:01.439680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.023979+0000","last_change":"2024-05-08T05:51:04.292341+0000","last_active":"2024-05-08T05:52:04.023979+0000","last_peered":"2024-05-08T05:52:04.023979+0000","last_clean":"2024-05-08T05:52:04.023979+0000","last_became_active":"2024-05-08T05:51:04.291086+0000","last_became_peered":"2024-05-08T05:51:04.291086+0000","last_unstale":"2024-05-08T05:52:04.023979+0000","last_undegraded":"2024-05-08T05:52:04.023979+0000","last_fullsized":"2024-05-08T05:52:04.023979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:47.108100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.024940+0000","last_change":"2024-05-08T05:51:00.268100+0000","last_active":"2024-05-08T05:52:04.024940+0000","last_peered":"2024-05-08T05:52:04.024940+0000","last_clean":"2024-05-08T05:52:04.024940+0000","last_became_active":"2024-05-08T05:51:00.267678+0000","last_became_peered":"2024-05-08T05:51:00.267678+0000","last_unstale":"2024-05-08T05:52:04.024940+0000","last_undegraded":"2024-05-08T05:52:04.024940+0000","last_fullsized":"2024-05-08T05:52:04.024940+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:11:31.731548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334601+0000","last_change":"2024-05-08T05:51:04.303872+0000","last_active":"2024-05-08T05:51:07.334601+0000","last_peered":"2024-05-08T05:51:07.334601+0000","last_clean":"2024-05-08T05:51:07.334601+0000","last_became_active":"2024-05-08T05:51:04.303456+0000","last_became_peered":"2024-05-08T05:51:04.303456+0000","last_unstale":"2024-05-08T05:51:07.334601+0000","last_undegraded":"2024-05-08T05:51:07.334601+0000","last_fullsized":"2024-05-08T05:51:07.334601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:42:05.098488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.016480+0000","last_change":"2024-05-08T05:51:00.275657+0000","last_active":"2024-05-08T05:52:04.016480+0000","last_peered":"2024-05-08T05:52:04.016480+0000","last_clean":"2024-05-08T05:52:04.016480+0000","last_became_active":"2024-05-08T05:51:00.275254+0000","last_became_peered":"2024-05-08T05:51:00.275254+0000","last_unstale":"2024-05-08T05:52:04.016480+0000","last_undegraded":"2024-05-08T05:52:04.016480+0000","last_fullsized":"2024-05-08T05:52:04.016480+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:28.555698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.025015+0000","last_change":"2024-05-08T05:51:04.293791+0000","last_active":"2024-05-08T05:52:04.025015+0000","last_peered":"2024-05-08T05:52:04.025015+0000","last_clean":"2024-05-08T05:52:04.025015+0000","last_became_active":"2024-05-08T05:51:04.293405+0000","last_became_peered":"2024-05-08T05:51:04.293405+0000","last_unstale":"2024-05-08T05:52:04.025015+0000","last_undegraded":"2024-05-08T05:52:04.025015+0000","last_fullsized":"2024-05-08T05:52:04.025015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:28.064452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:24.625314+0000","last_change":"2024-05-08T05:51:00.280295+0000","last_active":"2024-05-08T05:51:24.625314+0000","last_peered":"2024-05-08T05:51:24.625314+0000","last_clean":"2024-05-08T05:51:24.625314+0000","last_became_active":"2024-05-08T05:51:00.279721+0000","last_became_peered":"2024-05-08T05:51:00.279721+0000","last_unstale":"2024-05-08T05:51:24.625314+0000","last_undegraded":"2024-05-08T05:51:24.625314+0000","last_fullsized":"2024-05-08T05:51:24.625314+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:06:38.064872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.319121+0000","last_change":"2024-05-08T05:51:04.307546+0000","last_active":"2024-05-08T05:51:07.319121+0000","last_peered":"2024-05-08T05:51:07.319121+0000","last_clean":"2024-05-08T05:51:07.319121+0000","last_became_active":"2024-05-08T05:51:04.307164+0000","last_became_peered":"2024-05-08T05:51:04.307164+0000","last_unstale":"2024-05-08T05:51:07.319121+0000","last_undegraded":"2024-05-08T05:51:07.319121+0000","last_fullsized":"2024-05-08T05:51:07.319121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:36:02.269040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.024385+0000","last_change":"2024-05-08T05:51:00.267987+0000","last_active":"2024-05-08T05:52:04.024385+0000","last_peered":"2024-05-08T05:52:04.024385+0000","last_clean":"2024-05-08T05:52:04.024385+0000","last_became_active":"2024-05-08T05:51:00.267633+0000","last_became_peered":"2024-05-08T05:51:00.267633+0000","last_unstale":"2024-05-08T05:52:04.024385+0000","last_undegraded":"2024-05-08T05:52:04.024385+0000","last_fullsized":"2024-05-08T05:52:04.024385+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:13.034300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333490+0000","last_change":"2024-05-08T05:51:04.311422+0000","last_active":"2024-05-08T05:51:07.333490+0000","last_peered":"2024-05-08T05:51:07.333490+0000","last_clean":"2024-05-08T05:51:07.333490+0000","last_became_active":"2024-05-08T05:51:04.311052+0000","last_became_peered":"2024-05-08T05:51:04.311052+0000","last_unstale":"2024-05-08T05:51:07.333490+0000","last_undegraded":"2024-05-08T05:51:07.333490+0000","last_fullsized":"2024-05-08T05:51:07.333490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:47.790583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":63,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.016756+0000","last_change":"2024-05-08T05:51:00.274252+0000","last_active":"2024-05-08T05:52:04.016756+0000","last_peered":"2024-05-08T05:52:04.016756+0000","last_clean":"2024-05-08T05:52:04.016756+0000","last_became_active":"2024-05-08T05:51:00.273942+0000","last_became_peered":"2024-05-08T05:51:00.273942+0000","last_unstale":"2024-05-08T05:52:04.016756+0000","last_undegraded":"2024-05-08T05:52:04.016756+0000","last_fullsized":"2024-05-08T05:52:04.016756+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:36.778032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335581+0000","last_change":"2024-05-08T05:51:04.296414+0000","last_active":"2024-05-08T05:51:07.335581+0000","last_peered":"2024-05-08T05:51:07.335581+0000","last_clean":"2024-05-08T05:51:07.335581+0000","last_became_active":"2024-05-08T05:51:04.296015+0000","last_became_peered":"2024-05-08T05:51:04.296015+0000","last_unstale":"2024-05-08T05:51:07.335581+0000","last_undegraded":"2024-05-08T05:51:07.335581+0000","last_fullsized":"2024-05-08T05:51:07.335581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:48:15.609385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.023660+0000","last_change":"2024-05-08T05:51:00.268234+0000","last_active":"2024-05-08T05:52:05.023660+0000","last_peered":"2024-05-08T05:52:05.023660+0000","last_clean":"2024-05-08T05:52:05.023660+0000","last_became_active":"2024-05-08T05:51:00.267846+0000","last_became_peered":"2024-05-08T05:51:00.267846+0000","last_unstale":"2024-05-08T05:52:05.023660+0000","last_undegraded":"2024-05-08T05:52:05.023660+0000","last_fullsized":"2024-05-08T05:52:05.023660+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:26.919004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335890+0000","last_change":"2024-05-08T05:51:04.318215+0000","last_active":"2024-05-08T05:51:07.335890+0000","last_peered":"2024-05-08T05:51:07.335890+0000","last_clean":"2024-05-08T05:51:07.335890+0000","last_became_active":"2024-05-08T05:51:04.317309+0000","last_became_peered":"2024-05-08T05:51:04.317309+0000","last_unstale":"2024-05-08T05:51:07.335890+0000","last_undegraded":"2024-05-08T05:51:07.335890+0000","last_fullsized":"2024-05-08T05:51:07.335890+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:11.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.024080+0000","last_change":"2024-05-08T05:51:01.186438+0000","last_active":"2024-05-08T05:52:04.024080+0000","last_peered":"2024-05-08T05:52:04.024080+0000","last_clean":"2024-05-08T05:52:04.024080+0000","last_became_active":"2024-05-08T05:51:01.186039+0000","last_became_peered":"2024-05-08T05:51:01.186039+0000","last_unstale":"2024-05-08T05:52:04.024080+0000","last_undegraded":"2024-05-08T05:52:04.024080+0000","last_fullsized":"2024-05-08T05:52:04.024080+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:34.652184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335496+0000","last_change":"2024-05-08T05:51:04.294993+0000","last_active":"2024-05-08T05:51:07.335496+0000","last_peered":"2024-05-08T05:51:07.335496+0000","last_clean":"2024-05-08T05:51:07.335496+0000","last_became_active":"2024-05-08T05:51:04.294620+0000","last_became_peered":"2024-05-08T05:51:04.294620+0000","last_unstale":"2024-05-08T05:51:07.335496+0000","last_undegraded":"2024-05-08T05:51:07.335496+0000","last_fullsized":"2024-05-08T05:51:07.335496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:13:44.469181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":136,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.016776+0000","last_change":"2024-05-08T05:51:00.263385+0000","last_active":"2024-05-08T05:52:04.016776+0000","last_peered":"2024-05-08T05:52:04.016776+0000","last_clean":"2024-05-08T05:52:04.016776+0000","last_became_active":"2024-05-08T05:51:00.262997+0000","last_became_peered":"2024-05-08T05:51:00.262997+0000","last_unstale":"2024-05-08T05:52:04.016776+0000","last_undegraded":"2024-05-08T05:52:04.016776+0000","last_fullsized":"2024-05-08T05:52:04.016776+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:22:09.727481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61750,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":83,"num_read_kb":101,"num_write":31,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335948+0000","last_change":"2024-05-08T05:51:04.294790+0000","last_active":"2024-05-08T05:51:07.335948+0000","last_peered":"2024-05-08T05:51:07.335948+0000","last_clean":"2024-05-08T05:51:07.335948+0000","last_became_active":"2024-05-08T05:51:04.294418+0000","last_became_peered":"2024-05-08T05:51:04.294418+0000","last_unstale":"2024-05-08T05:51:07.335948+0000","last_undegraded":"2024-05-08T05:51:07.335948+0000","last_fullsized":"2024-05-08T05:51:07.335948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:11:10.624434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:24.618463+0000","last_change":"2024-05-08T05:51:00.258507+0000","last_active":"2024-05-08T05:51:24.618463+0000","last_peered":"2024-05-08T05:51:24.618463+0000","last_clean":"2024-05-08T05:51:24.618463+0000","last_became_active":"2024-05-08T05:51:00.258179+0000","last_became_peered":"2024-05-08T05:51:00.258179+0000","last_unstale":"2024-05-08T05:51:24.618463+0000","last_undegraded":"2024-05-08T05:51:24.618463+0000","last_fullsized":"2024-05-08T05:51:24.618463+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:22.376499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333753+0000","last_change":"2024-05-08T05:51:04.310947+0000","last_active":"2024-05-08T05:51:07.333753+0000","last_peered":"2024-05-08T05:51:07.333753+0000","last_clean":"2024-05-08T05:51:07.333753+0000","last_became_active":"2024-05-08T05:51:04.310378+0000","last_became_peered":"2024-05-08T05:51:04.310378+0000","last_unstale":"2024-05-08T05:51:07.333753+0000","last_undegraded":"2024-05-08T05:51:07.333753+0000","last_fullsized":"2024-05-08T05:51:07.333753+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:19.367660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.024468+0000","last_change":"2024-05-08T05:51:00.271104+0000","last_active":"2024-05-08T05:52:04.024468+0000","last_peered":"2024-05-08T05:52:04.024468+0000","last_clean":"2024-05-08T05:52:04.024468+0000","last_became_active":"2024-05-08T05:51:00.270497+0000","last_became_peered":"2024-05-08T05:51:00.270497+0000","last_unstale":"2024-05-08T05:52:04.024468+0000","last_undegraded":"2024-05-08T05:52:04.024468+0000","last_fullsized":"2024-05-08T05:52:04.024468+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:40.959423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.024362+0000","last_change":"2024-05-08T05:51:04.310243+0000","last_active":"2024-05-08T05:52:05.024362+0000","last_peered":"2024-05-08T05:52:05.024362+0000","last_clean":"2024-05-08T05:52:05.024362+0000","last_became_active":"2024-05-08T05:51:04.309871+0000","last_became_peered":"2024-05-08T05:51:04.309871+0000","last_unstale":"2024-05-08T05:52:05.024362+0000","last_undegraded":"2024-05-08T05:52:05.024362+0000","last_fullsized":"2024-05-08T05:52:05.024362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:18:21.045621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":66,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:52:02.699372+0000","last_change":"2024-05-08T05:51:00.266096+0000","last_active":"2024-05-08T05:52:02.699372+0000","last_peered":"2024-05-08T05:52:02.699372+0000","last_clean":"2024-05-08T05:52:02.699372+0000","last_became_active":"2024-05-08T05:51:00.265706+0000","last_became_peered":"2024-05-08T05:51:00.265706+0000","last_unstale":"2024-05-08T05:52:02.699372+0000","last_undegraded":"2024-05-08T05:52:02.699372+0000","last_fullsized":"2024-05-08T05:52:02.699372+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:47.388732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.023143+0000","last_change":"2024-05-08T05:51:04.293141+0000","last_active":"2024-05-08T05:52:05.023143+0000","last_peered":"2024-05-08T05:52:05.023143+0000","last_clean":"2024-05-08T05:52:05.023143+0000","last_became_active":"2024-05-08T05:51:04.292772+0000","last_became_peered":"2024-05-08T05:51:04.292772+0000","last_unstale":"2024-05-08T05:52:05.023143+0000","last_undegraded":"2024-05-08T05:52:05.023143+0000","last_fullsized":"2024-05-08T05:52:05.023143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:25:32.801209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:24.621645+0000","last_change":"2024-05-08T05:51:01.186243+0000","last_active":"2024-05-08T05:51:24.621645+0000","last_peered":"2024-05-08T05:51:24.621645+0000","last_clean":"2024-05-08T05:51:24.621645+0000","last_became_active":"2024-05-08T05:51:01.185758+0000","last_became_peered":"2024-05-08T05:51:01.185758+0000","last_unstale":"2024-05-08T05:51:24.621645+0000","last_undegraded":"2024-05-08T05:51:24.621645+0000","last_fullsized":"2024-05-08T05:51:24.621645+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:40:47.946477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.023806+0000","last_change":"2024-05-08T05:51:04.290096+0000","last_active":"2024-05-08T05:52:04.023806+0000","last_peered":"2024-05-08T05:52:04.023806+0000","last_clean":"2024-05-08T05:52:04.023806+0000","last_became_active":"2024-05-08T05:51:04.289684+0000","last_became_peered":"2024-05-08T05:51:04.289684+0000","last_unstale":"2024-05-08T05:52:04.023806+0000","last_undegraded":"2024-05-08T05:52:04.023806+0000","last_fullsized":"2024-05-08T05:52:04.023806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:02.400348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.034507+0000","last_change":"2024-05-08T05:52:05.034004+0000","last_active":"2024-05-08T05:52:05.034507+0000","last_peered":"2024-05-08T05:52:05.034507+0000","last_clean":"2024-05-08T05:52:05.034507+0000","last_became_active":"2024-05-08T05:52:05.023497+0000","last_became_peered":"2024-05-08T05:52:05.023497+0000","last_unstale":"2024-05-08T05:52:05.034507+0000","last_undegraded":"2024-05-08T05:52:05.034507+0000","last_fullsized":"2024-05-08T05:52:05.034507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:06.462940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.027648+0000","last_change":"2024-05-08T05:52:05.027648+0000","last_active":"2024-05-08T05:52:05.027648+0000","last_peered":"2024-05-08T05:52:05.027648+0000","last_clean":"2024-05-08T05:52:05.027648+0000","last_became_active":"2024-05-08T05:52:05.027262+0000","last_became_peered":"2024-05-08T05:52:05.027262+0000","last_unstale":"2024-05-08T05:52:05.027648+0000","last_undegraded":"2024-05-08T05:52:05.027648+0000","last_fullsized":"2024-05-08T05:52:05.027648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:36:18.588967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335062+0000","last_change":"2024-05-08T05:51:01.189324+0000","last_active":"2024-05-08T05:51:07.335062+0000","last_peered":"2024-05-08T05:51:07.335062+0000","last_clean":"2024-05-08T05:51:07.335062+0000","last_became_active":"2024-05-08T05:51:01.188651+0000","last_became_peered":"2024-05-08T05:51:01.188651+0000","last_unstale":"2024-05-08T05:51:07.335062+0000","last_undegraded":"2024-05-08T05:51:07.335062+0000","last_fullsized":"2024-05-08T05:51:07.335062+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:45.149213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.016522+0000","last_change":"2024-05-08T05:51:04.306811+0000","last_active":"2024-05-08T05:52:04.016522+0000","last_peered":"2024-05-08T05:52:04.016522+0000","last_clean":"2024-05-08T05:52:04.016522+0000","last_became_active":"2024-05-08T05:51:04.306423+0000","last_became_peered":"2024-05-08T05:51:04.306423+0000","last_unstale":"2024-05-08T05:52:04.016522+0000","last_undegraded":"2024-05-08T05:52:04.016522+0000","last_fullsized":"2024-05-08T05:52:04.016522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:08.830682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.023391+0000","last_change":"2024-05-08T05:51:00.276543+0000","last_active":"2024-05-08T05:52:05.023391+0000","last_peered":"2024-05-08T05:52:05.023391+0000","last_clean":"2024-05-08T05:52:05.023391+0000","last_became_active":"2024-05-08T05:51:00.276233+0000","last_became_peered":"2024-05-08T05:51:00.276233+0000","last_unstale":"2024-05-08T05:52:05.023391+0000","last_undegraded":"2024-05-08T05:52:05.023391+0000","last_fullsized":"2024-05-08T05:52:05.023391+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:11.229942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.016516+0000","last_change":"2024-05-08T05:51:04.322033+0000","last_active":"2024-05-08T05:52:04.016516+0000","last_peered":"2024-05-08T05:52:04.016516+0000","last_clean":"2024-05-08T05:52:04.016516+0000","last_became_active":"2024-05-08T05:51:04.321555+0000","last_became_peered":"2024-05-08T05:51:04.321555+0000","last_unstale":"2024-05-08T05:52:04.016516+0000","last_undegraded":"2024-05-08T05:52:04.016516+0000","last_fullsized":"2024-05-08T05:52:04.016516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:21:50.820709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333280+0000","last_change":"2024-05-08T05:51:04.295613+0000","last_active":"2024-05-08T05:51:07.333280+0000","last_peered":"2024-05-08T05:51:07.333280+0000","last_clean":"2024-05-08T05:51:07.333280+0000","last_became_active":"2024-05-08T05:51:04.295278+0000","last_became_peered":"2024-05-08T05:51:04.295278+0000","last_unstale":"2024-05-08T05:51:07.333280+0000","last_undegraded":"2024-05-08T05:51:07.333280+0000","last_fullsized":"2024-05-08T05:51:07.333280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:57.236920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":60,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:59.693076+0000","last_change":"2024-05-08T05:51:01.187556+0000","last_active":"2024-05-08T05:51:59.693076+0000","last_peered":"2024-05-08T05:51:59.693076+0000","last_clean":"2024-05-08T05:51:59.693076+0000","last_became_active":"2024-05-08T05:51:01.186584+0000","last_became_peered":"2024-05-08T05:51:01.186584+0000","last_unstale":"2024-05-08T05:51:59.693076+0000","last_undegraded":"2024-05-08T05:51:59.693076+0000","last_fullsized":"2024-05-08T05:51:59.693076+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:29.036633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.319744+0000","last_change":"2024-05-08T05:51:04.315916+0000","last_active":"2024-05-08T05:51:07.319744+0000","last_peered":"2024-05-08T05:51:07.319744+0000","last_clean":"2024-05-08T05:51:07.319744+0000","last_became_active":"2024-05-08T05:51:04.315180+0000","last_became_peered":"2024-05-08T05:51:04.315180+0000","last_unstale":"2024-05-08T05:51:07.319744+0000","last_undegraded":"2024-05-08T05:51:07.319744+0000","last_fullsized":"2024-05-08T05:51:07.319744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:34:50.431271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:27.632668+0000","last_change":"2024-05-08T05:51:00.272501+0000","last_active":"2024-05-08T05:51:27.632668+0000","last_peered":"2024-05-08T05:51:27.632668+0000","last_clean":"2024-05-08T05:51:27.632668+0000","last_became_active":"2024-05-08T05:51:00.272166+0000","last_became_peered":"2024-05-08T05:51:00.272166+0000","last_unstale":"2024-05-08T05:51:27.632668+0000","last_undegraded":"2024-05-08T05:51:27.632668+0000","last_fullsized":"2024-05-08T05:51:27.632668+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:30.980047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":98,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.023329+0000","last_change":"2024-05-08T05:49:31.820799+0000","last_active":"2024-05-08T05:52:05.023329+0000","last_peered":"2024-05-08T05:52:05.023329+0000","last_clean":"2024-05-08T05:52:05.023329+0000","last_became_active":"2024-05-08T05:49:31.815568+0000","last_became_peered":"2024-05-08T05:49:31.815568+0000","last_unstale":"2024-05-08T05:52:05.023329+0000","last_undegraded":"2024-05-08T05:52:05.023329+0000","last_fullsized":"2024-05-08T05:52:05.023329+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:48.170242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:48.170242+0000","last_clean_scrub_stamp":"2024-05-08T05:45:48.170242+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:39.527036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333551+0000","last_change":"2024-05-08T05:51:04.301375+0000","last_active":"2024-05-08T05:51:07.333551+0000","last_peered":"2024-05-08T05:51:07.333551+0000","last_clean":"2024-05-08T05:51:07.333551+0000","last_became_active":"2024-05-08T05:51:04.301027+0000","last_became_peered":"2024-05-08T05:51:04.301027+0000","last_unstale":"2024-05-08T05:51:07.333551+0000","last_undegraded":"2024-05-08T05:51:07.333551+0000","last_fullsized":"2024-05-08T05:51:07.333551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:21.393242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.037950+0000","last_change":"2024-05-08T05:52:05.037427+0000","last_active":"2024-05-08T05:52:05.037950+0000","last_peered":"2024-05-08T05:52:05.037950+0000","last_clean":"2024-05-08T05:52:05.037950+0000","last_became_active":"2024-05-08T05:52:05.023644+0000","last_became_peered":"2024-05-08T05:52:05.023644+0000","last_unstale":"2024-05-08T05:52:05.037950+0000","last_undegraded":"2024-05-08T05:52:05.037950+0000","last_fullsized":"2024-05-08T05:52:05.037950+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:49.193375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:23.602899+0000","last_change":"2024-05-08T05:51:01.187171+0000","last_active":"2024-05-08T05:51:23.602899+0000","last_peered":"2024-05-08T05:51:23.602899+0000","last_clean":"2024-05-08T05:51:23.602899+0000","last_became_active":"2024-05-08T05:51:01.186142+0000","last_became_peered":"2024-05-08T05:51:01.186142+0000","last_unstale":"2024-05-08T05:51:23.602899+0000","last_undegraded":"2024-05-08T05:51:23.602899+0000","last_fullsized":"2024-05-08T05:51:23.602899+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:37:17.021905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334317+0000","last_change":"2024-05-08T05:51:04.311538+0000","last_active":"2024-05-08T05:51:07.334317+0000","last_peered":"2024-05-08T05:51:07.334317+0000","last_clean":"2024-05-08T05:51:07.334317+0000","last_became_active":"2024-05-08T05:51:04.310535+0000","last_became_peered":"2024-05-08T05:51:04.310535+0000","last_unstale":"2024-05-08T05:51:07.334317+0000","last_undegraded":"2024-05-08T05:51:07.334317+0000","last_fullsized":"2024-05-08T05:51:07.334317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:01:04.497986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:52.339473+0000","last_change":"2024-05-08T05:51:00.277923+0000","last_active":"2024-05-08T05:51:52.339473+0000","last_peered":"2024-05-08T05:51:52.339473+0000","last_clean":"2024-05-08T05:51:52.339473+0000","last_became_active":"2024-05-08T05:51:00.274896+0000","last_became_peered":"2024-05-08T05:51:00.274896+0000","last_unstale":"2024-05-08T05:51:52.339473+0000","last_undegraded":"2024-05-08T05:51:52.339473+0000","last_fullsized":"2024-05-08T05:51:52.339473+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:04:32.543451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.320757+0000","last_change":"2024-05-08T05:51:04.306185+0000","last_active":"2024-05-08T05:51:07.320757+0000","last_peered":"2024-05-08T05:51:07.320757+0000","last_clean":"2024-05-08T05:51:07.320757+0000","last_became_active":"2024-05-08T05:51:04.305229+0000","last_became_peered":"2024-05-08T05:51:04.305229+0000","last_unstale":"2024-05-08T05:51:07.320757+0000","last_undegraded":"2024-05-08T05:51:07.320757+0000","last_fullsized":"2024-05-08T05:51:07.320757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:48.859978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:23.601212+0000","last_change":"2024-05-08T05:51:01.140178+0000","last_active":"2024-05-08T05:51:23.601212+0000","last_peered":"2024-05-08T05:51:23.601212+0000","last_clean":"2024-05-08T05:51:23.601212+0000","last_became_active":"2024-05-08T05:51:01.139772+0000","last_became_peered":"2024-05-08T05:51:01.139772+0000","last_unstale":"2024-05-08T05:51:23.601212+0000","last_undegraded":"2024-05-08T05:51:23.601212+0000","last_fullsized":"2024-05-08T05:51:23.601212+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:54:40.132683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335803+0000","last_change":"2024-05-08T05:51:04.319015+0000","last_active":"2024-05-08T05:51:07.335803+0000","last_peered":"2024-05-08T05:51:07.335803+0000","last_clean":"2024-05-08T05:51:07.335803+0000","last_became_active":"2024-05-08T05:51:04.318465+0000","last_became_peered":"2024-05-08T05:51:04.318465+0000","last_unstale":"2024-05-08T05:51:07.335803+0000","last_undegraded":"2024-05-08T05:51:07.335803+0000","last_fullsized":"2024-05-08T05:51:07.335803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:07:19.336656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:23.602608+0000","last_change":"2024-05-08T05:51:01.185619+0000","last_active":"2024-05-08T05:51:23.602608+0000","last_peered":"2024-05-08T05:51:23.602608+0000","last_clean":"2024-05-08T05:51:23.602608+0000","last_became_active":"2024-05-08T05:51:01.185325+0000","last_became_peered":"2024-05-08T05:51:01.185325+0000","last_unstale":"2024-05-08T05:51:23.602608+0000","last_undegraded":"2024-05-08T05:51:23.602608+0000","last_fullsized":"2024-05-08T05:51:23.602608+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:41.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333894+0000","last_change":"2024-05-08T05:51:04.295018+0000","last_active":"2024-05-08T05:51:07.333894+0000","last_peered":"2024-05-08T05:51:07.333894+0000","last_clean":"2024-05-08T05:51:07.333894+0000","last_became_active":"2024-05-08T05:51:04.294432+0000","last_became_peered":"2024-05-08T05:51:04.294432+0000","last_unstale":"2024-05-08T05:51:07.333894+0000","last_undegraded":"2024-05-08T05:51:07.333894+0000","last_fullsized":"2024-05-08T05:51:07.333894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:01.925456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:23.603127+0000","last_change":"2024-05-08T05:51:00.280527+0000","last_active":"2024-05-08T05:51:23.603127+0000","last_peered":"2024-05-08T05:51:23.603127+0000","last_clean":"2024-05-08T05:51:23.603127+0000","last_became_active":"2024-05-08T05:51:00.278718+0000","last_became_peered":"2024-05-08T05:51:00.278718+0000","last_unstale":"2024-05-08T05:51:23.603127+0000","last_undegraded":"2024-05-08T05:51:23.603127+0000","last_fullsized":"2024-05-08T05:51:23.603127+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:48:27.149713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.021353+0000","last_change":"2024-05-08T05:51:04.314543+0000","last_active":"2024-05-08T05:52:05.021353+0000","last_peered":"2024-05-08T05:52:05.021353+0000","last_clean":"2024-05-08T05:52:05.021353+0000","last_became_active":"2024-05-08T05:51:04.314168+0000","last_became_peered":"2024-05-08T05:51:04.314168+0000","last_unstale":"2024-05-08T05:52:05.021353+0000","last_undegraded":"2024-05-08T05:52:05.021353+0000","last_fullsized":"2024-05-08T05:52:05.021353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:10:23.215322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334501+0000","last_change":"2024-05-08T05:51:00.268081+0000","last_active":"2024-05-08T05:51:07.334501+0000","last_peered":"2024-05-08T05:51:07.334501+0000","last_clean":"2024-05-08T05:51:07.334501+0000","last_became_active":"2024-05-08T05:51:00.267470+0000","last_became_peered":"2024-05-08T05:51:00.267470+0000","last_unstale":"2024-05-08T05:51:07.334501+0000","last_undegraded":"2024-05-08T05:51:07.334501+0000","last_fullsized":"2024-05-08T05:51:07.334501+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:45.796019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.320351+0000","last_change":"2024-05-08T05:51:04.322010+0000","last_active":"2024-05-08T05:51:07.320351+0000","last_peered":"2024-05-08T05:51:07.320351+0000","last_clean":"2024-05-08T05:51:07.320351+0000","last_became_active":"2024-05-08T05:51:04.320867+0000","last_became_peered":"2024-05-08T05:51:04.320867+0000","last_unstale":"2024-05-08T05:51:07.320351+0000","last_undegraded":"2024-05-08T05:51:07.320351+0000","last_fullsized":"2024-05-08T05:51:07.320351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:49.853518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:23.598607+0000","last_change":"2024-05-08T05:51:00.269378+0000","last_active":"2024-05-08T05:51:23.598607+0000","last_peered":"2024-05-08T05:51:23.598607+0000","last_clean":"2024-05-08T05:51:23.598607+0000","last_became_active":"2024-05-08T05:51:00.268992+0000","last_became_peered":"2024-05-08T05:51:00.268992+0000","last_unstale":"2024-05-08T05:51:23.598607+0000","last_undegraded":"2024-05-08T05:51:23.598607+0000","last_fullsized":"2024-05-08T05:51:23.598607+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:21.722501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.023450+0000","last_change":"2024-05-08T05:51:04.302813+0000","last_active":"2024-05-08T05:52:05.023450+0000","last_peered":"2024-05-08T05:52:05.023450+0000","last_clean":"2024-05-08T05:52:05.023450+0000","last_became_active":"2024-05-08T05:51:04.302424+0000","last_became_peered":"2024-05-08T05:51:04.302424+0000","last_unstale":"2024-05-08T05:52:05.023450+0000","last_undegraded":"2024-05-08T05:52:05.023450+0000","last_fullsized":"2024-05-08T05:52:05.023450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:52:17.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:24.622617+0000","last_change":"2024-05-08T05:51:01.187513+0000","last_active":"2024-05-08T05:51:24.622617+0000","last_peered":"2024-05-08T05:51:24.622617+0000","last_clean":"2024-05-08T05:51:24.622617+0000","last_became_active":"2024-05-08T05:51:01.186780+0000","last_became_peered":"2024-05-08T05:51:01.186780+0000","last_unstale":"2024-05-08T05:51:24.622617+0000","last_undegraded":"2024-05-08T05:51:24.622617+0000","last_fullsized":"2024-05-08T05:51:24.622617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:42:26.206351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.024131+0000","last_change":"2024-05-08T05:51:04.295338+0000","last_active":"2024-05-08T05:52:05.024131+0000","last_peered":"2024-05-08T05:52:05.024131+0000","last_clean":"2024-05-08T05:52:05.024131+0000","last_became_active":"2024-05-08T05:51:04.294990+0000","last_became_peered":"2024-05-08T05:51:04.294990+0000","last_unstale":"2024-05-08T05:52:05.024131+0000","last_undegraded":"2024-05-08T05:52:05.024131+0000","last_fullsized":"2024-05-08T05:52:05.024131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:29.411662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.023832+0000","last_change":"2024-05-08T05:51:00.273052+0000","last_active":"2024-05-08T05:52:04.023832+0000","last_peered":"2024-05-08T05:52:04.023832+0000","last_clean":"2024-05-08T05:52:04.023832+0000","last_became_active":"2024-05-08T05:51:00.272718+0000","last_became_peered":"2024-05-08T05:51:00.272718+0000","last_unstale":"2024-05-08T05:52:04.023832+0000","last_undegraded":"2024-05-08T05:52:04.023832+0000","last_fullsized":"2024-05-08T05:52:04.023832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:44.494378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.024560+0000","last_change":"2024-05-08T05:51:04.306209+0000","last_active":"2024-05-08T05:52:05.024560+0000","last_peered":"2024-05-08T05:52:05.024560+0000","last_clean":"2024-05-08T05:52:05.024560+0000","last_became_active":"2024-05-08T05:51:04.305752+0000","last_became_peered":"2024-05-08T05:51:04.305752+0000","last_unstale":"2024-05-08T05:52:05.024560+0000","last_undegraded":"2024-05-08T05:52:05.024560+0000","last_fullsized":"2024-05-08T05:52:05.024560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:24.586111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.339421+0000","last_change":"2024-05-08T05:51:00.278426+0000","last_active":"2024-05-08T05:51:07.339421+0000","last_peered":"2024-05-08T05:51:07.339421+0000","last_clean":"2024-05-08T05:51:07.339421+0000","last_became_active":"2024-05-08T05:51:00.276764+0000","last_became_peered":"2024-05-08T05:51:00.276764+0000","last_unstale":"2024-05-08T05:51:07.339421+0000","last_undegraded":"2024-05-08T05:51:07.339421+0000","last_fullsized":"2024-05-08T05:51:07.339421+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:34.158628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.024192+0000","last_change":"2024-05-08T05:51:04.294511+0000","last_active":"2024-05-08T05:52:05.024192+0000","last_peered":"2024-05-08T05:52:05.024192+0000","last_clean":"2024-05-08T05:52:05.024192+0000","last_became_active":"2024-05-08T05:51:04.294145+0000","last_became_peered":"2024-05-08T05:51:04.294145+0000","last_unstale":"2024-05-08T05:52:05.024192+0000","last_undegraded":"2024-05-08T05:52:05.024192+0000","last_fullsized":"2024-05-08T05:52:05.024192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:10.034060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.021379+0000","last_change":"2024-05-08T05:51:04.298905+0000","last_active":"2024-05-08T05:52:05.021379+0000","last_peered":"2024-05-08T05:52:05.021379+0000","last_clean":"2024-05-08T05:52:05.021379+0000","last_became_active":"2024-05-08T05:51:04.298586+0000","last_became_peered":"2024-05-08T05:51:04.298586+0000","last_unstale":"2024-05-08T05:52:05.021379+0000","last_undegraded":"2024-05-08T05:52:05.021379+0000","last_fullsized":"2024-05-08T05:52:05.021379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:43:24.899015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334269+0000","last_change":"2024-05-08T05:51:04.300503+0000","last_active":"2024-05-08T05:51:07.334269+0000","last_peered":"2024-05-08T05:51:07.334269+0000","last_clean":"2024-05-08T05:51:07.334269+0000","last_became_active":"2024-05-08T05:51:04.299771+0000","last_became_peered":"2024-05-08T05:51:04.299771+0000","last_unstale":"2024-05-08T05:51:07.334269+0000","last_undegraded":"2024-05-08T05:51:07.334269+0000","last_fullsized":"2024-05-08T05:51:07.334269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:19.790501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.333975+0000","last_change":"2024-05-08T05:51:04.320492+0000","last_active":"2024-05-08T05:51:07.333975+0000","last_peered":"2024-05-08T05:51:07.333975+0000","last_clean":"2024-05-08T05:51:07.333975+0000","last_became_active":"2024-05-08T05:51:04.319996+0000","last_became_peered":"2024-05-08T05:51:04.319996+0000","last_unstale":"2024-05-08T05:51:07.333975+0000","last_undegraded":"2024-05-08T05:51:07.333975+0000","last_fullsized":"2024-05-08T05:51:07.333975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:10:33.874558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.335111+0000","last_change":"2024-05-08T05:51:04.318604+0000","last_active":"2024-05-08T05:51:07.335111+0000","last_peered":"2024-05-08T05:51:07.335111+0000","last_clean":"2024-05-08T05:51:07.335111+0000","last_became_active":"2024-05-08T05:51:04.318133+0000","last_became_peered":"2024-05-08T05:51:04.318133+0000","last_unstale":"2024-05-08T05:51:07.335111+0000","last_undegraded":"2024-05-08T05:51:07.335111+0000","last_fullsized":"2024-05-08T05:51:07.335111+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:30:47.146559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.336110+0000","last_change":"2024-05-08T05:51:04.315769+0000","last_active":"2024-05-08T05:51:07.336110+0000","last_peered":"2024-05-08T05:51:07.336110+0000","last_clean":"2024-05-08T05:51:07.336110+0000","last_became_active":"2024-05-08T05:51:04.315322+0000","last_became_peered":"2024-05-08T05:51:04.315322+0000","last_unstale":"2024-05-08T05:51:07.336110+0000","last_undegraded":"2024-05-08T05:51:07.336110+0000","last_fullsized":"2024-05-08T05:51:07.336110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:52.096556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.020892+0000","last_change":"2024-05-08T05:51:04.298128+0000","last_active":"2024-05-08T05:52:05.020892+0000","last_peered":"2024-05-08T05:52:05.020892+0000","last_clean":"2024-05-08T05:52:05.020892+0000","last_became_active":"2024-05-08T05:51:04.297743+0000","last_became_peered":"2024-05-08T05:51:04.297743+0000","last_unstale":"2024-05-08T05:52:05.020892+0000","last_undegraded":"2024-05-08T05:52:05.020892+0000","last_fullsized":"2024-05-08T05:52:05.020892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:18.334026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334522+0000","last_change":"2024-05-08T05:51:04.301056+0000","last_active":"2024-05-08T05:51:07.334522+0000","last_peered":"2024-05-08T05:51:07.334522+0000","last_clean":"2024-05-08T05:51:07.334522+0000","last_became_active":"2024-05-08T05:51:04.300733+0000","last_became_peered":"2024-05-08T05:51:04.300733+0000","last_unstale":"2024-05-08T05:51:07.334522+0000","last_undegraded":"2024-05-08T05:51:07.334522+0000","last_fullsized":"2024-05-08T05:51:07.334522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:21.616867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.320699+0000","last_change":"2024-05-08T05:51:04.299079+0000","last_active":"2024-05-08T05:51:07.320699+0000","last_peered":"2024-05-08T05:51:07.320699+0000","last_clean":"2024-05-08T05:51:07.320699+0000","last_became_active":"2024-05-08T05:51:04.298628+0000","last_became_peered":"2024-05-08T05:51:04.298628+0000","last_unstale":"2024-05-08T05:51:07.320699+0000","last_undegraded":"2024-05-08T05:51:07.320699+0000","last_fullsized":"2024-05-08T05:51:07.320699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:45.031695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T05:52:04.024515+0000","last_change":"2024-05-08T05:51:04.296140+0000","last_active":"2024-05-08T05:52:04.024515+0000","last_peered":"2024-05-08T05:52:04.024515+0000","last_clean":"2024-05-08T05:52:04.024515+0000","last_became_active":"2024-05-08T05:51:04.294940+0000","last_became_peered":"2024-05-08T05:51:04.294940+0000","last_unstale":"2024-05-08T05:52:04.024515+0000","last_undegraded":"2024-05-08T05:52:04.024515+0000","last_fullsized":"2024-05-08T05:52:04.024515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:37.192639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.319272+0000","last_change":"2024-05-08T05:51:04.305803+0000","last_active":"2024-05-08T05:51:07.319272+0000","last_peered":"2024-05-08T05:51:07.319272+0000","last_clean":"2024-05-08T05:51:07.319272+0000","last_became_active":"2024-05-08T05:51:04.305482+0000","last_became_peered":"2024-05-08T05:51:04.305482+0000","last_unstale":"2024-05-08T05:51:07.319272+0000","last_undegraded":"2024-05-08T05:51:07.319272+0000","last_fullsized":"2024-05-08T05:51:07.319272+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:02.050153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334930+0000","last_change":"2024-05-08T05:51:04.293787+0000","last_active":"2024-05-08T05:51:07.334930+0000","last_peered":"2024-05-08T05:51:07.334930+0000","last_clean":"2024-05-08T05:51:07.334930+0000","last_became_active":"2024-05-08T05:51:04.293455+0000","last_became_peered":"2024-05-08T05:51:04.293455+0000","last_unstale":"2024-05-08T05:51:07.334930+0000","last_undegraded":"2024-05-08T05:51:07.334930+0000","last_fullsized":"2024-05-08T05:51:07.334930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:09:53.211411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.334467+0000","last_change":"2024-05-08T05:51:04.321442+0000","last_active":"2024-05-08T05:51:07.334467+0000","last_peered":"2024-05-08T05:51:07.334467+0000","last_clean":"2024-05-08T05:51:07.334467+0000","last_became_active":"2024-05-08T05:51:04.320982+0000","last_became_peered":"2024-05-08T05:51:04.320982+0000","last_unstale":"2024-05-08T05:51:07.334467+0000","last_undegraded":"2024-05-08T05:51:07.334467+0000","last_fullsized":"2024-05-08T05:51:07.334467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:45.089609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T05:52:05.020586+0000","last_change":"2024-05-08T05:51:04.294866+0000","last_active":"2024-05-08T05:52:05.020586+0000","last_peered":"2024-05-08T05:52:05.020586+0000","last_clean":"2024-05-08T05:52:05.020586+0000","last_became_active":"2024-05-08T05:51:04.294504+0000","last_became_peered":"2024-05-08T05:51:04.294504+0000","last_unstale":"2024-05-08T05:52:05.020586+0000","last_undegraded":"2024-05-08T05:52:05.020586+0000","last_fullsized":"2024-05-08T05:52:05.020586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:33.139864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65751,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":246,"num_read_kb":190,"num_write":165,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":197275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645340,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":880640,"data_stored":427029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841568,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1071934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005094,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":418548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201323,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":880640,"data_stored":418468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430257,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1490944,"data_stored":1019115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593782,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":417937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":479480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":419208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":954368,"data_stored":480830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":418248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986242,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":418765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790015,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36788,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708364,"statfs":{"total":95995035648,"available":95957364736,"internally_reserved":0,"allocated":1486848,"data_stored":1009691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:52:06.821 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T05:52:06.822 INFO:tasks.cephadm:Setting up client nodes... 2024-05-08T05:52:06.822 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-08T05:52:07.314 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:07.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:07 smithi112 ceph-mon[30003]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 639 B/s wr, 7 op/s; 2 B/s, 0 objects/s recovering 2024-05-08T05:52:07.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:07 smithi112 ceph-mon[30003]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:52:07.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:07 smithi162 ceph-mon[34800]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 639 B/s wr, 7 op/s; 2 B/s, 0 objects/s recovering 2024-05-08T05:52:07.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:07 smithi162 ceph-mon[34800]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:52:07.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:07 smithi150 ceph-mon[34315]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 639 B/s wr, 7 op/s; 2 B/s, 0 objects/s recovering 2024-05-08T05:52:07.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:07 smithi150 ceph-mon[34315]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:52:09.271 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:09 smithi112 ceph-mon[30003]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 16 B/s, 1 objects/s recovering 2024-05-08T05:52:09.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:09 smithi162 ceph-mon[34800]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 16 B/s, 1 objects/s recovering 2024-05-08T05:52:09.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:09 smithi150 ceph-mon[34315]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 16 B/s, 1 objects/s recovering 2024-05-08T05:52:09.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.551+0000 7fe6dd600700 1 -- 172.21.15.112:0/4235323615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6d8075f00 msgr2=0x7fe6d8076380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:09.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.551+0000 7fe6dd600700 1 --2- 172.21.15.112:0/4235323615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6d8075f00 0x7fe6d8076380 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009990 tx=0x7fe6c4009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.551+0000 7fe6dd600700 1 -- 172.21.15.112:0/4235323615 shutdown_connections 2024-05-08T05:52:09.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.551+0000 7fe6dd600700 1 --2- 172.21.15.112:0/4235323615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6d810b230 0x7fe6d810d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.551+0000 7fe6dd600700 1 --2- 172.21.15.112:0/4235323615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6d80774d0 0x7fe6d8075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.551+0000 7fe6dd600700 1 --2- 172.21.15.112:0/4235323615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6d8075f00 0x7fe6d8076380 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.551+0000 7fe6dd600700 1 -- 172.21.15.112:0/4235323615 >> 172.21.15.112:0/4235323615 conn(0x7fe6d80fe8e0 msgr2=0x7fe6d8100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:09.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.551+0000 7fe6dd600700 1 -- 172.21.15.112:0/4235323615 shutdown_connections 2024-05-08T05:52:09.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.551+0000 7fe6dd600700 1 -- 172.21.15.112:0/4235323615 wait complete. 2024-05-08T05:52:09.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.552+0000 7fe6dd600700 1 Processor -- start 2024-05-08T05:52:09.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.553+0000 7fe6dd600700 1 -- start start 2024-05-08T05:52:09.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.553+0000 7fe6dd600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6d8075f00 0x7fe6d819b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:09.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.553+0000 7fe6dd600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6d80774d0 0x7fe6d819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:09.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.553+0000 7fe6dd600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6d810b230 0x7fe6d81a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:09.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.553+0000 7fe6dd600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8111070 con 0x7fe6d8075f00 2024-05-08T05:52:09.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.553+0000 7fe6dd600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8110ef0 con 0x7fe6d810b230 2024-05-08T05:52:09.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.553+0000 7fe6dd600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81111f0 con 0x7fe6d80774d0 2024-05-08T05:52:09.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.553+0000 7fe6d7e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6d8075f00 0x7fe6d819b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:09.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.553+0000 7fe6d7400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6d80774d0 0x7fe6d819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:09.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.553+0000 7fe6d7e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6d8075f00 0x7fe6d819b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58104/0 (socket says 172.21.15.112:58104) 2024-05-08T05:52:09.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.553+0000 7fe6d7e00700 1 -- 172.21.15.112:0/3580854443 learned_addr learned my addr 172.21.15.112:0/3580854443 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:09.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.554+0000 7fe6dcc00700 1 --2- 172.21.15.112:0/3580854443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6d810b230 0x7fe6d81a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:09.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.554+0000 7fe6d7400700 1 -- 172.21.15.112:0/3580854443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6d810b230 msgr2=0x7fe6d81a2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:09.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.554+0000 7fe6d7400700 1 --2- 172.21.15.112:0/3580854443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6d810b230 0x7fe6d81a2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.554+0000 7fe6d7400700 1 -- 172.21.15.112:0/3580854443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6d8075f00 msgr2=0x7fe6d819b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:09.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.554+0000 7fe6d7400700 1 --2- 172.21.15.112:0/3580854443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6d8075f00 0x7fe6d819b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.554+0000 7fe6d7400700 1 -- 172.21.15.112:0/3580854443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d81a32a0 con 0x7fe6d80774d0 2024-05-08T05:52:09.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.554+0000 7fe6d7e00700 1 --2- 172.21.15.112:0/3580854443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6d8075f00 0x7fe6d819b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:52:09.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.554+0000 7fe6d7400700 1 --2- 172.21.15.112:0/3580854443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6d80774d0 0x7fe6d819bb80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009990 tx=0x7fe6c40057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:09.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.555+0000 7fe6d4c00700 1 -- 172.21.15.112:0/3580854443 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4026070 con 0x7fe6d80774d0 2024-05-08T05:52:09.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.555+0000 7fe6dd600700 1 -- 172.21.15.112:0/3580854443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d81a3530 con 0x7fe6d80774d0 2024-05-08T05:52:09.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.555+0000 7fe6d4c00700 1 -- 172.21.15.112:0/3580854443 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6c4021410 con 0x7fe6d80774d0 2024-05-08T05:52:09.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.555+0000 7fe6d4c00700 1 -- 172.21.15.112:0/3580854443 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4017650 con 0x7fe6d80774d0 2024-05-08T05:52:09.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.555+0000 7fe6dd600700 1 -- 172.21.15.112:0/3580854443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d81a3990 con 0x7fe6d80774d0 2024-05-08T05:52:09.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.556+0000 7fe6dd600700 1 -- 172.21.15.112:0/3580854443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d8064b00 con 0x7fe6d80774d0 2024-05-08T05:52:09.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.559+0000 7fe6d4c00700 1 -- 172.21.15.112:0/3580854443 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe6c4029070 con 0x7fe6d80774d0 2024-05-08T05:52:09.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.560+0000 7fe6d4c00700 1 --2- 172.21.15.112:0/3580854443 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6bc0b29b0 0x7fe6bc0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:09.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.561+0000 7fe6d4c00700 1 -- 172.21.15.112:0/3580854443 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7fe6c40e3170 con 0x7fe6d80774d0 2024-05-08T05:52:09.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.561+0000 7fe6d7e00700 1 --2- 172.21.15.112:0/3580854443 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6bc0b29b0 0x7fe6bc0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:09.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.562+0000 7fe6d7e00700 1 --2- 172.21.15.112:0/3580854443 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6bc0b29b0 0x7fe6bc0b4e70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8005950 tx=0x7fe6c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:09.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.562+0000 7fe6d4c00700 1 -- 172.21.15.112:0/3580854443 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe6c4013070 con 0x7fe6d80774d0 2024-05-08T05:52:09.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.884+0000 7fe6dd600700 1 -- 172.21.15.112:0/3580854443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fe6d802e5b0 con 0x7fe6d80774d0 2024-05-08T05:52:09.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.892+0000 7fe6d4c00700 1 -- 172.21.15.112:0/3580854443 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fe6d802e5b0 con 0x7fe6d80774d0 2024-05-08T05:52:09.893 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-05-08T05:52:09.893 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQAJEztmwfHcNBAAFjyP7fkoGGCJKd2USLT64w== 2024-05-08T05:52:09.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.895+0000 7fe6dd600700 1 -- 172.21.15.112:0/3580854443 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6bc0b29b0 msgr2=0x7fe6bc0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:09.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.895+0000 7fe6dd600700 1 --2- 172.21.15.112:0/3580854443 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6bc0b29b0 0x7fe6bc0b4e70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8005950 tx=0x7fe6c8009260 comp rx=0 tx=0).stop 2024-05-08T05:52:09.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.895+0000 7fe6dd600700 1 -- 172.21.15.112:0/3580854443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6d80774d0 msgr2=0x7fe6d819bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:09.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.895+0000 7fe6dd600700 1 --2- 172.21.15.112:0/3580854443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6d80774d0 0x7fe6d819bb80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009990 tx=0x7fe6c40057a0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.896+0000 7fe6dd600700 1 -- 172.21.15.112:0/3580854443 shutdown_connections 2024-05-08T05:52:09.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.896+0000 7fe6dd600700 1 --2- 172.21.15.112:0/3580854443 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6bc0b29b0 0x7fe6bc0b4e70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.896+0000 7fe6dd600700 1 --2- 172.21.15.112:0/3580854443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6d810b230 0x7fe6d81a2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.896+0000 7fe6dd600700 1 --2- 172.21.15.112:0/3580854443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6d8075f00 0x7fe6d819b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.896+0000 7fe6dd600700 1 --2- 172.21.15.112:0/3580854443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6d80774d0 0x7fe6d819bb80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.896+0000 7fe6dd600700 1 -- 172.21.15.112:0/3580854443 >> 172.21.15.112:0/3580854443 conn(0x7fe6d80fe8e0 msgr2=0x7fe6d8107130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:09.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.896+0000 7fe6dd600700 1 -- 172.21.15.112:0/3580854443 shutdown_connections 2024-05-08T05:52:09.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:09.896+0000 7fe6dd600700 1 -- 172.21.15.112:0/3580854443 wait complete. 2024-05-08T05:52:10.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:10 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3580854443' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T05:52:10.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:10 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T05:52:10.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:10 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T05:52:10.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:10 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3580854443' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T05:52:10.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:10 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T05:52:10.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:10 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T05:52:10.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:10 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3580854443' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T05:52:10.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:10 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T05:52:10.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:10 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T05:52:10.884 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:52:10.884 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-08T05:52:10.884 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T05:52:11.014 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T05:52:11.014 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-08T05:52:11.014 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-08T05:52:11.128 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:11 smithi112 ceph-mon[30003]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 16 B/s, 1 objects/s recovering 2024-05-08T05:52:11.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:11 smithi162 ceph-mon[34800]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 16 B/s, 1 objects/s recovering 2024-05-08T05:52:11.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:11 smithi150 ceph-mon[34315]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 16 B/s, 1 objects/s recovering 2024-05-08T05:52:11.497 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:13.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:13 smithi112 ceph-mon[30003]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 16 B/s, 1 objects/s recovering 2024-05-08T05:52:13.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:13 smithi162 ceph-mon[34800]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 16 B/s, 1 objects/s recovering 2024-05-08T05:52:13.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:13 smithi150 ceph-mon[34315]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 16 B/s, 1 objects/s recovering 2024-05-08T05:52:13.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.527+0000 7f15d5400700 1 -- 172.21.15.112:0/3506869218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d010fc60 msgr2=0x7f15d0112070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.527+0000 7f15d5400700 1 --2- 172.21.15.112:0/3506869218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d010fc60 0x7f15d0112070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009f90 tx=0x7f15c800b330 comp rx=0 tx=0).stop 2024-05-08T05:52:13.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.527+0000 7f15d5400700 1 -- 172.21.15.112:0/3506869218 shutdown_connections 2024-05-08T05:52:13.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.527+0000 7f15d5400700 1 --2- 172.21.15.112:0/3506869218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d010fc60 0x7f15d0112070 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.527+0000 7f15d5400700 1 --2- 172.21.15.112:0/3506869218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15d0075870 0x7f15d0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.527+0000 7f15d5400700 1 --2- 172.21.15.112:0/3506869218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d0076190 0x7f15d010f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.527+0000 7f15d5400700 1 -- 172.21.15.112:0/3506869218 >> 172.21.15.112:0/3506869218 conn(0x7f15d00feeb0 msgr2=0x7f15d01012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:13.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.527+0000 7f15d5400700 1 -- 172.21.15.112:0/3506869218 shutdown_connections 2024-05-08T05:52:13.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.527+0000 7f15d5400700 1 -- 172.21.15.112:0/3506869218 wait complete. 2024-05-08T05:52:13.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.528+0000 7f15d5400700 1 Processor -- start 2024-05-08T05:52:13.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.528+0000 7f15d5400700 1 -- start start 2024-05-08T05:52:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.529+0000 7f15d5400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15d0075870 0x7f15d01a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.529+0000 7f15d5400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d0076190 0x7f15d01a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.529+0000 7f15d5400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d010fc60 0x7f15d019ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.529+0000 7f15d5400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15d0115790 con 0x7f15d010fc60 2024-05-08T05:52:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.529+0000 7f15d5400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15d0115610 con 0x7f15d0076190 2024-05-08T05:52:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.529+0000 7f15d5400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f15d0115910 con 0x7f15d0075870 2024-05-08T05:52:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.529+0000 7f15d4a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d010fc60 0x7f15d019ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.529+0000 7f15d4a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d010fc60 0x7f15d019ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58134/0 (socket says 172.21.15.112:58134) 2024-05-08T05:52:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.529+0000 7f15d4a00700 1 -- 172.21.15.112:0/2733852057 learned_addr learned my addr 172.21.15.112:0/2733852057 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.529+0000 7f15cf400700 1 --2- 172.21.15.112:0/2733852057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d0076190 0x7f15d01a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.529+0000 7f15cfe00700 1 --2- 172.21.15.112:0/2733852057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15d0075870 0x7f15d01a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.530+0000 7f15cf400700 1 -- 172.21.15.112:0/2733852057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15d0075870 msgr2=0x7f15d01a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.530+0000 7f15cf400700 1 --2- 172.21.15.112:0/2733852057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15d0075870 0x7f15d01a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.530+0000 7f15cf400700 1 -- 172.21.15.112:0/2733852057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d010fc60 msgr2=0x7f15d019ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.530+0000 7f15cf400700 1 --2- 172.21.15.112:0/2733852057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d010fc60 0x7f15d019ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.530+0000 7f15cf400700 1 -- 172.21.15.112:0/2733852057 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d019c1f0 con 0x7f15d0076190 2024-05-08T05:52:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.530+0000 7f15d4a00700 1 --2- 172.21.15.112:0/2733852057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d010fc60 0x7f15d019ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:52:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.530+0000 7f15cf400700 1 --2- 172.21.15.112:0/2733852057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d0076190 0x7f15d01a1e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00e9f0 tx=0x7f15bc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.530+0000 7f15ccc00700 1 -- 172.21.15.112:0/2733852057 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc00c9c0 con 0x7f15d0076190 2024-05-08T05:52:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.530+0000 7f15d5400700 1 -- 172.21.15.112:0/2733852057 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d019c4e0 con 0x7f15d0076190 2024-05-08T05:52:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.530+0000 7f15ccc00700 1 -- 172.21.15.112:0/2733852057 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15bc00cb60 con 0x7f15d0076190 2024-05-08T05:52:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.530+0000 7f15ccc00700 1 -- 172.21.15.112:0/2733852057 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc018680 con 0x7f15d0076190 2024-05-08T05:52:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.530+0000 7f15d5400700 1 -- 172.21.15.112:0/2733852057 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d01a87a0 con 0x7f15d0076190 2024-05-08T05:52:13.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.531+0000 7f15d5400700 1 -- 172.21.15.112:0/2733852057 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d006a190 con 0x7f15d0076190 2024-05-08T05:52:13.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.534+0000 7f15ccc00700 1 -- 172.21.15.112:0/2733852057 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f15bc010430 con 0x7f15d0076190 2024-05-08T05:52:13.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.535+0000 7f15ccc00700 1 --2- 172.21.15.112:0/2733852057 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f15b40b2910 0x7f15b40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.536+0000 7f15ccc00700 1 -- 172.21.15.112:0/2733852057 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7f15bc0e2870 con 0x7f15d0076190 2024-05-08T05:52:13.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.536+0000 7f15cfe00700 1 --2- 172.21.15.112:0/2733852057 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f15b40b2910 0x7f15b40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.536+0000 7f15cfe00700 1 --2- 172.21.15.112:0/2733852057 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f15b40b2910 0x7f15b40b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f15d019cbd0 tx=0x7f15c0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:13.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.537+0000 7f15ccc00700 1 -- 172.21.15.112:0/2733852057 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15bc0aa3c0 con 0x7f15d0076190 2024-05-08T05:52:13.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.835+0000 7f15d5400700 1 -- 172.21.15.112:0/2733852057 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f15d002e510 con 0x7f15d0076190 2024-05-08T05:52:13.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.842+0000 7f15ccc00700 1 -- 172.21.15.112:0/2733852057 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288653 (secure 0 0 0) 0x7f15bc0a9cc0 con 0x7f15d0076190 2024-05-08T05:52:13.847 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:52:13.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.847+0000 7f15d5400700 1 -- 172.21.15.112:0/2733852057 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f15b40b2910 msgr2=0x7f15b40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.847+0000 7f15d5400700 1 --2- 172.21.15.112:0/2733852057 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f15b40b2910 0x7f15b40b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f15d019cbd0 tx=0x7f15c0009310 comp rx=0 tx=0).stop 2024-05-08T05:52:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.847+0000 7f15d5400700 1 -- 172.21.15.112:0/2733852057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d0076190 msgr2=0x7f15d01a1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.848+0000 7f15d5400700 1 --2- 172.21.15.112:0/2733852057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d0076190 0x7f15d01a1e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00e9f0 tx=0x7f15bc00ed00 comp rx=0 tx=0).stop 2024-05-08T05:52:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.848+0000 7f15d5400700 1 -- 172.21.15.112:0/2733852057 shutdown_connections 2024-05-08T05:52:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.848+0000 7f15d5400700 1 --2- 172.21.15.112:0/2733852057 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f15b40b2910 0x7f15b40b4dd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.848+0000 7f15d5400700 1 --2- 172.21.15.112:0/2733852057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d010fc60 0x7f15d019ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.848+0000 7f15d5400700 1 --2- 172.21.15.112:0/2733852057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15d0075870 0x7f15d01a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.848+0000 7f15d5400700 1 --2- 172.21.15.112:0/2733852057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d0076190 0x7f15d01a1e10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.848+0000 7f15d5400700 1 -- 172.21.15.112:0/2733852057 >> 172.21.15.112:0/2733852057 conn(0x7f15d00feeb0 msgr2=0x7f15d00ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.848+0000 7f15d5400700 1 -- 172.21.15.112:0/2733852057 shutdown_connections 2024-05-08T05:52:13.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:13.848+0000 7f15d5400700 1 -- 172.21.15.112:0/2733852057 wait complete. 2024-05-08T05:52:14.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:14 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2733852057' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T05:52:14.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:14 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/2733852057' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T05:52:14.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:14 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/2733852057' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T05:52:14.761 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1211679607},{"type":"v1","addr":"172.21.15.112:6801","nonce":1211679607}]},"active_addr":"172.21.15.112:6801/1211679607","active_change":"2024-05-08T05:43:03.313846+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.112:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":74091236}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":861445710}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3302469559}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":4195767286}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2916208843}]}]} 2024-05-08T05:52:14.769 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-08T05:52:14.769 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-08T05:52:14.769 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:52:15.219 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:15.373 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:15 smithi112 ceph-mon[30003]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 14 B/s, 1 objects/s recovering 2024-05-08T05:52:15.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:15 smithi150 ceph-mon[34315]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 14 B/s, 1 objects/s recovering 2024-05-08T05:52:15.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:15 smithi162 ceph-mon[34800]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 14 B/s, 1 objects/s recovering 2024-05-08T05:52:17.233 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:17 smithi112 ceph-mon[30003]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 12 B/s, 1 objects/s recovering 2024-05-08T05:52:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:17 smithi150 ceph-mon[34315]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 12 B/s, 1 objects/s recovering 2024-05-08T05:52:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.481+0000 7f9505000700 1 -- 172.21.15.112:0/1298230050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95000756d0 msgr2=0x7f9500075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.481+0000 7f9505000700 1 --2- 172.21.15.112:0/1298230050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95000756d0 0x7f9500075ab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009990 tx=0x7f94f0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.481+0000 7f9505000700 1 -- 172.21.15.112:0/1298230050 shutdown_connections 2024-05-08T05:52:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.481+0000 7f9505000700 1 --2- 172.21.15.112:0/1298230050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f950010fea0 0x7f9500112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.481+0000 7f9505000700 1 --2- 172.21.15.112:0/1298230050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95000756d0 0x7f9500075ab0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.481+0000 7f9505000700 1 --2- 172.21.15.112:0/1298230050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9500075ff0 0x7f950010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:17.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.481+0000 7f9505000700 1 -- 172.21.15.112:0/1298230050 >> 172.21.15.112:0/1298230050 conn(0x7f950007a600 msgr2=0x7f950007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:17.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.482+0000 7f9505000700 1 -- 172.21.15.112:0/1298230050 shutdown_connections 2024-05-08T05:52:17.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.482+0000 7f9505000700 1 -- 172.21.15.112:0/1298230050 wait complete. 2024-05-08T05:52:17.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.483+0000 7f9505000700 1 Processor -- start 2024-05-08T05:52:17.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.483+0000 7f9505000700 1 -- start start 2024-05-08T05:52:17.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.483+0000 7f9505000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95000756d0 0x7f950019bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:17.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.483+0000 7f9505000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9500075ff0 0x7f950019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.483+0000 7f9505000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f950010fea0 0x7f95001a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.483+0000 7f9505000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9500115890 con 0x7f95000756d0 2024-05-08T05:52:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.483+0000 7f9505000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9500115710 con 0x7f9500075ff0 2024-05-08T05:52:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.483+0000 7f9505000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9500115a10 con 0x7f950010fea0 2024-05-08T05:52:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.484+0000 7f94fd400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9500075ff0 0x7f950019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.484+0000 7f94fd400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9500075ff0 0x7f950019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:39374/0 (socket says 172.21.15.112:39374) 2024-05-08T05:52:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.484+0000 7f94fd400700 1 -- 172.21.15.112:0/3034558031 learned_addr learned my addr 172.21.15.112:0/3034558031 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.484+0000 7f94fe800700 1 --2- 172.21.15.112:0/3034558031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f950010fea0 0x7f95001a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.484+0000 7f94fde00700 1 --2- 172.21.15.112:0/3034558031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95000756d0 0x7f950019bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.484+0000 7f94fd400700 1 -- 172.21.15.112:0/3034558031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f950010fea0 msgr2=0x7f95001a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.484+0000 7f94fd400700 1 --2- 172.21.15.112:0/3034558031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f950010fea0 0x7f95001a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.484+0000 7f94fd400700 1 -- 172.21.15.112:0/3034558031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95000756d0 msgr2=0x7f950019bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.484+0000 7f94fd400700 1 --2- 172.21.15.112:0/3034558031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95000756d0 0x7f950019bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.484+0000 7f94fd400700 1 -- 172.21.15.112:0/3034558031 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95001a0f30 con 0x7f9500075ff0 2024-05-08T05:52:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.485+0000 7f94fd400700 1 --2- 172.21.15.112:0/3034558031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9500075ff0 0x7f950019c460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00e9c0 tx=0x7f94ec00ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.485+0000 7f94f6a00700 1 -- 172.21.15.112:0/3034558031 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec00cbd0 con 0x7f9500075ff0 2024-05-08T05:52:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.485+0000 7f94f6a00700 1 -- 172.21.15.112:0/3034558031 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94ec00cd70 con 0x7f9500075ff0 2024-05-08T05:52:17.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.485+0000 7f94f6a00700 1 -- 172.21.15.112:0/3034558031 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec010430 con 0x7f9500075ff0 2024-05-08T05:52:17.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.485+0000 7f9505000700 1 -- 172.21.15.112:0/3034558031 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95001a1220 con 0x7f9500075ff0 2024-05-08T05:52:17.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.485+0000 7f9505000700 1 -- 172.21.15.112:0/3034558031 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95001a8aa0 con 0x7f9500075ff0 2024-05-08T05:52:17.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.486+0000 7f9505000700 1 -- 172.21.15.112:0/3034558031 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f950006a190 con 0x7f9500075ff0 2024-05-08T05:52:17.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.490+0000 7f94f6a00700 1 -- 172.21.15.112:0/3034558031 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f94ec004750 con 0x7f9500075ff0 2024-05-08T05:52:17.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.491+0000 7f94f6a00700 1 --2- 172.21.15.112:0/3034558031 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f94e40b2990 0x7f94e40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:17.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.491+0000 7f94f6a00700 1 -- 172.21.15.112:0/3034558031 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7f94ec0e2960 con 0x7f9500075ff0 2024-05-08T05:52:17.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.492+0000 7f94fde00700 1 --2- 172.21.15.112:0/3034558031 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f94e40b2990 0x7f94e40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:17.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.492+0000 7f94fde00700 1 --2- 172.21.15.112:0/3034558031 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f94e40b2990 0x7f94e40b4e50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f94f000b2e0 tx=0x7f94f001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:17.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.492+0000 7f94f6a00700 1 -- 172.21.15.112:0/3034558031 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f94ec054df0 con 0x7f9500075ff0 2024-05-08T05:52:17.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:17 smithi162 ceph-mon[34800]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 12 B/s, 1 objects/s recovering 2024-05-08T05:52:17.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.727+0000 7f9505000700 1 -- 172.21.15.112:0/3034558031 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f950019d280 con 0x7f9500075ff0 2024-05-08T05:52:17.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.729+0000 7f94f6a00700 1 -- 172.21.15.112:0/3034558031 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22783 (secure 0 0 0) 0x7f94ec0aa4b0 con 0x7f9500075ff0 2024-05-08T05:52:17.730 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:52:17.730 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":82,"fsid":"7a07ff60-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:48.498871+0000","modified":"2024-05-08T05:52:05.013847+0000","last_up_change":"2024-05-08T05:49:54.039380+0000","last_in_change":"2024-05-08T05:49:34.443850+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:47.485105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:50:58.918909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.175022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b378542-1bbe-4614-8805-8a6b875ada55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6803","nonce":3419091820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6805","nonce":3419091820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6809","nonce":3419091820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6807","nonce":3419091820}]},"public_addr":"172.21.15.112:6803/3419091820","cluster_addr":"172.21.15.112:6805/3419091820","heartbeat_back_addr":"172.21.15.112:6809/3419091820","heartbeat_front_addr":"172.21.15.112:6807/3419091820","state":["exists","up"]},{"osd":1,"uuid":"826ef036-2db7-470e-9956-d9a58b9181e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6801","nonce":2728419797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6803","nonce":2728419797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6807","nonce":2728419797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6805","nonce":2728419797}]},"public_addr":"172.21.15.150:6801/2728419797","cluster_addr":"172.21.15.150:6803/2728419797","heartbeat_back_addr":"172.21.15.150:6807/2728419797","heartbeat_front_addr":"172.21.15.150:6805/2728419797","state":["exists","up"]},{"osd":2,"uuid":"32e710e7-2a87-4863-bc7a-906d31aa4ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6801","nonce":1556203215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6803","nonce":1556203215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6807","nonce":1556203215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6805","nonce":1556203215}]},"public_addr":"172.21.15.162:6801/1556203215","cluster_addr":"172.21.15.162:6803/1556203215","heartbeat_back_addr":"172.21.15.162:6807/1556203215","heartbeat_front_addr":"172.21.15.162:6805/1556203215","state":["exists","up"]},{"osd":3,"uuid":"ba1d4a5f-6a95-4b9e-8a48-8524a899b004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6811","nonce":3325718755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6813","nonce":3325718755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6817","nonce":3325718755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6815","nonce":3325718755}]},"public_addr":"172.21.15.112:6811/3325718755","cluster_addr":"172.21.15.112:6813/3325718755","heartbeat_back_addr":"172.21.15.112:6817/3325718755","heartbeat_front_addr":"172.21.15.112:6815/3325718755","state":["exists","up"]},{"osd":4,"uuid":"cb4f1a42-8de8-4b50-b3bc-602de9f4803a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6809","nonce":1548914683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6811","nonce":1548914683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6815","nonce":1548914683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6813","nonce":1548914683}]},"public_addr":"172.21.15.150:6809/1548914683","cluster_addr":"172.21.15.150:6811/1548914683","heartbeat_back_addr":"172.21.15.150:6815/1548914683","heartbeat_front_addr":"172.21.15.150:6813/1548914683","state":["exists","up"]},{"osd":5,"uuid":"a26c03b8-050c-4a2c-8f23-5801fffb7c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6809","nonce":409836750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6811","nonce":409836750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6815","nonce":409836750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6813","nonce":409836750}]},"public_addr":"172.21.15.162:6809/409836750","cluster_addr":"172.21.15.162:6811/409836750","heartbeat_back_addr":"172.21.15.162:6815/409836750","heartbeat_front_addr":"172.21.15.162:6813/409836750","state":["exists","up"]},{"osd":6,"uuid":"8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6819","nonce":1098984186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6821","nonce":1098984186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6825","nonce":1098984186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6823","nonce":1098984186}]},"public_addr":"172.21.15.112:6819/1098984186","cluster_addr":"172.21.15.112:6821/1098984186","heartbeat_back_addr":"172.21.15.112:6825/1098984186","heartbeat_front_addr":"172.21.15.112:6823/1098984186","state":["exists","up"]},{"osd":7,"uuid":"af3629b4-9bed-4669-97a6-48deba503a96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6817","nonce":4009836564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6819","nonce":4009836564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6823","nonce":4009836564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6821","nonce":4009836564}]},"public_addr":"172.21.15.150:6817/4009836564","cluster_addr":"172.21.15.150:6819/4009836564","heartbeat_back_addr":"172.21.15.150:6823/4009836564","heartbeat_front_addr":"172.21.15.150:6821/4009836564","state":["exists","up"]},{"osd":8,"uuid":"9333e87d-dd78-4467-bbde-c2b44f4ea07f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6817","nonce":852791606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6819","nonce":852791606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6823","nonce":852791606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6821","nonce":852791606}]},"public_addr":"172.21.15.162:6817/852791606","cluster_addr":"172.21.15.162:6819/852791606","heartbeat_back_addr":"172.21.15.162:6823/852791606","heartbeat_front_addr":"172.21.15.162:6821/852791606","state":["exists","up"]},{"osd":9,"uuid":"09723581-1995-42e0-8db7-776ea98a918c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6827","nonce":2186764674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6829","nonce":2186764674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6833","nonce":2186764674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6831","nonce":2186764674}]},"public_addr":"172.21.15.112:6827/2186764674","cluster_addr":"172.21.15.112:6829/2186764674","heartbeat_back_addr":"172.21.15.112:6833/2186764674","heartbeat_front_addr":"172.21.15.112:6831/2186764674","state":["exists","up"]},{"osd":10,"uuid":"34e08e3e-1231-4899-a5b3-76d449105bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6825","nonce":2838995474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6827","nonce":2838995474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6831","nonce":2838995474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6829","nonce":2838995474}]},"public_addr":"172.21.15.150:6825/2838995474","cluster_addr":"172.21.15.150:6827/2838995474","heartbeat_back_addr":"172.21.15.150:6831/2838995474","heartbeat_front_addr":"172.21.15.150:6829/2838995474","state":["exists","up"]},{"osd":11,"uuid":"6fc34ec0-78ca-403c-acb7-b29d83ff64e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6825","nonce":3702614331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6827","nonce":3702614331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6831","nonce":3702614331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6829","nonce":3702614331}]},"public_addr":"172.21.15.162:6825/3702614331","cluster_addr":"172.21.15.162:6827/3702614331","heartbeat_back_addr":"172.21.15.162:6831/3702614331","heartbeat_front_addr":"172.21.15.162:6829/3702614331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:54.518702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:19.939543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:44.425586+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:16.126471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:41.696071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:05.517405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:38.549318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:03.799649+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:28.007282+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:01.395459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:27.963637+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:52.181212+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3667148497":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6800/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3425327158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1404611205":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3765334043":"2024-05-09T05:42:29.819307+0000","172.21.15.112:6800/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/2814303790":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1091751583":"2024-05-09T05:42:29.819307+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:52:17.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.733+0000 7f9505000700 1 -- 172.21.15.112:0/3034558031 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f94e40b2990 msgr2=0x7f94e40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:17.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.733+0000 7f9505000700 1 --2- 172.21.15.112:0/3034558031 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f94e40b2990 0x7f94e40b4e50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f94f000b2e0 tx=0x7f94f001a040 comp rx=0 tx=0).stop 2024-05-08T05:52:17.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.733+0000 7f9505000700 1 -- 172.21.15.112:0/3034558031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9500075ff0 msgr2=0x7f950019c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:17.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.733+0000 7f9505000700 1 --2- 172.21.15.112:0/3034558031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9500075ff0 0x7f950019c460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00e9c0 tx=0x7f94ec00ed80 comp rx=0 tx=0).stop 2024-05-08T05:52:17.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.733+0000 7f9505000700 1 -- 172.21.15.112:0/3034558031 shutdown_connections 2024-05-08T05:52:17.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.733+0000 7f9505000700 1 --2- 172.21.15.112:0/3034558031 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f94e40b2990 0x7f94e40b4e50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:17.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.733+0000 7f9505000700 1 --2- 172.21.15.112:0/3034558031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f950010fea0 0x7f95001a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:17.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.733+0000 7f9505000700 1 --2- 172.21.15.112:0/3034558031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95000756d0 0x7f950019bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:17.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.733+0000 7f9505000700 1 --2- 172.21.15.112:0/3034558031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9500075ff0 0x7f950019c460 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:17.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.733+0000 7f9505000700 1 -- 172.21.15.112:0/3034558031 >> 172.21.15.112:0/3034558031 conn(0x7f950007a600 msgr2=0x7f9500101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:17.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.733+0000 7f9505000700 1 -- 172.21.15.112:0/3034558031 shutdown_connections 2024-05-08T05:52:17.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:17.733+0000 7f9505000700 1 -- 172.21.15.112:0/3034558031 wait complete. 2024-05-08T05:52:18.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:18 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3034558031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:52:18.479 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:18 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3034558031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:52:18.550 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-08T05:52:18.550 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:52:18.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:18 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3034558031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:52:18.995 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:19.370 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:19 smithi112 ceph-mon[30003]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-05-08T05:52:19.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:19 smithi150 ceph-mon[34315]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-05-08T05:52:19.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:19 smithi162 ceph-mon[34800]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-05-08T05:52:21.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.117+0000 7fb546600700 1 -- 172.21.15.112:0/3290869958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5400756d0 msgr2=0x7fb540075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:21.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.117+0000 7fb546600700 1 --2- 172.21.15.112:0/3290869958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5400756d0 0x7fb540075ab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb530009990 tx=0x7fb530009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.117+0000 7fb546600700 1 -- 172.21.15.112:0/3290869958 shutdown_connections 2024-05-08T05:52:21.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.117+0000 7fb546600700 1 --2- 172.21.15.112:0/3290869958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb54010fea0 0x7fb540112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.117+0000 7fb546600700 1 --2- 172.21.15.112:0/3290869958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5400756d0 0x7fb540075ab0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.117+0000 7fb546600700 1 --2- 172.21.15.112:0/3290869958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb540075ff0 0x7fb54010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.117+0000 7fb546600700 1 -- 172.21.15.112:0/3290869958 >> 172.21.15.112:0/3290869958 conn(0x7fb54007a600 msgr2=0x7fb54007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:21.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.118+0000 7fb546600700 1 -- 172.21.15.112:0/3290869958 shutdown_connections 2024-05-08T05:52:21.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.118+0000 7fb546600700 1 -- 172.21.15.112:0/3290869958 wait complete. 2024-05-08T05:52:21.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.119+0000 7fb546600700 1 Processor -- start 2024-05-08T05:52:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.119+0000 7fb546600700 1 -- start start 2024-05-08T05:52:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.119+0000 7fb546600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5400756d0 0x7fb54019be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.119+0000 7fb546600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb540075ff0 0x7fb54019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.119+0000 7fb546600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb54010fea0 0x7fb5401a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.119+0000 7fb546600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5401158a0 con 0x7fb540075ff0 2024-05-08T05:52:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.119+0000 7fb546600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb540115720 con 0x7fb5400756d0 2024-05-08T05:52:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.119+0000 7fb546600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb540115a20 con 0x7fb54010fea0 2024-05-08T05:52:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.120+0000 7fb53ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb540075ff0 0x7fb54019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.120+0000 7fb53f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5400756d0 0x7fb54019be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.120+0000 7fb53f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5400756d0 0x7fb54019be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:39396/0 (socket says 172.21.15.112:39396) 2024-05-08T05:52:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.120+0000 7fb53f400700 1 -- 172.21.15.112:0/3542767133 learned_addr learned my addr 172.21.15.112:0/3542767133 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:21.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.120+0000 7fb53fe00700 1 --2- 172.21.15.112:0/3542767133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb54010fea0 0x7fb5401a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:21.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.120+0000 7fb53fe00700 1 -- 172.21.15.112:0/3542767133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5400756d0 msgr2=0x7fb54019be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:21.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.120+0000 7fb53fe00700 1 --2- 172.21.15.112:0/3542767133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5400756d0 0x7fb54019be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.120+0000 7fb53fe00700 1 -- 172.21.15.112:0/3542767133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb540075ff0 msgr2=0x7fb54019c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:21.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.120+0000 7fb53fe00700 1 --2- 172.21.15.112:0/3542767133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb540075ff0 0x7fb54019c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.120+0000 7fb53fe00700 1 -- 172.21.15.112:0/3542767133 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5401a0dd0 con 0x7fb54010fea0 2024-05-08T05:52:21.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.120+0000 7fb53fe00700 1 --2- 172.21.15.112:0/3542767133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb54010fea0 0x7fb5401a0690 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb53800ef90 tx=0x7fb53800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:21.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.121+0000 7fb537e00700 1 -- 172.21.15.112:0/3542767133 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5380092e0 con 0x7fb54010fea0 2024-05-08T05:52:21.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.121+0000 7fb537e00700 1 -- 172.21.15.112:0/3542767133 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb538004bb0 con 0x7fb54010fea0 2024-05-08T05:52:21.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.121+0000 7fb546600700 1 -- 172.21.15.112:0/3542767133 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5401a1060 con 0x7fb54010fea0 2024-05-08T05:52:21.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.121+0000 7fb537e00700 1 -- 172.21.15.112:0/3542767133 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538004020 con 0x7fb54010fea0 2024-05-08T05:52:21.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.121+0000 7fb546600700 1 -- 172.21.15.112:0/3542767133 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5401a8940 con 0x7fb54010fea0 2024-05-08T05:52:21.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.122+0000 7fb546600700 1 -- 172.21.15.112:0/3542767133 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb54006a190 con 0x7fb54010fea0 2024-05-08T05:52:21.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.124+0000 7fb537e00700 1 -- 172.21.15.112:0/3542767133 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb53801b030 con 0x7fb54010fea0 2024-05-08T05:52:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.125+0000 7fb537e00700 1 --2- 172.21.15.112:0/3542767133 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb5240b2990 0x7fb5240b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.125+0000 7fb537e00700 1 -- 172.21.15.112:0/3542767133 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7fb5380e1e60 con 0x7fb54010fea0 2024-05-08T05:52:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.125+0000 7fb53f400700 1 --2- 172.21.15.112:0/3542767133 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb5240b2990 0x7fb5240b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.126+0000 7fb53f400700 1 --2- 172.21.15.112:0/3542767133 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb5240b2990 0x7fb5240b4e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb530006010 tx=0x7fb53000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:21.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.130+0000 7fb537e00700 1 -- 172.21.15.112:0/3542767133 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb538022020 con 0x7fb54010fea0 2024-05-08T05:52:21.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:21 smithi112 ceph-mon[30003]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:21.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.363+0000 7fb546600700 1 -- 172.21.15.112:0/3542767133 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb54002e510 con 0x7fb54010fea0 2024-05-08T05:52:21.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.366+0000 7fb537e00700 1 -- 172.21.15.112:0/3542767133 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22783 (secure 0 0 0) 0x7fb538010040 con 0x7fb54010fea0 2024-05-08T05:52:21.367 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:52:21.367 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":82,"fsid":"7a07ff60-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:48.498871+0000","modified":"2024-05-08T05:52:05.013847+0000","last_up_change":"2024-05-08T05:49:54.039380+0000","last_in_change":"2024-05-08T05:49:34.443850+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:47.485105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:50:58.918909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.175022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b378542-1bbe-4614-8805-8a6b875ada55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6803","nonce":3419091820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6805","nonce":3419091820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6809","nonce":3419091820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6807","nonce":3419091820}]},"public_addr":"172.21.15.112:6803/3419091820","cluster_addr":"172.21.15.112:6805/3419091820","heartbeat_back_addr":"172.21.15.112:6809/3419091820","heartbeat_front_addr":"172.21.15.112:6807/3419091820","state":["exists","up"]},{"osd":1,"uuid":"826ef036-2db7-470e-9956-d9a58b9181e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6801","nonce":2728419797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6803","nonce":2728419797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6807","nonce":2728419797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6805","nonce":2728419797}]},"public_addr":"172.21.15.150:6801/2728419797","cluster_addr":"172.21.15.150:6803/2728419797","heartbeat_back_addr":"172.21.15.150:6807/2728419797","heartbeat_front_addr":"172.21.15.150:6805/2728419797","state":["exists","up"]},{"osd":2,"uuid":"32e710e7-2a87-4863-bc7a-906d31aa4ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6801","nonce":1556203215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6803","nonce":1556203215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6807","nonce":1556203215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6805","nonce":1556203215}]},"public_addr":"172.21.15.162:6801/1556203215","cluster_addr":"172.21.15.162:6803/1556203215","heartbeat_back_addr":"172.21.15.162:6807/1556203215","heartbeat_front_addr":"172.21.15.162:6805/1556203215","state":["exists","up"]},{"osd":3,"uuid":"ba1d4a5f-6a95-4b9e-8a48-8524a899b004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6811","nonce":3325718755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6813","nonce":3325718755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6817","nonce":3325718755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6815","nonce":3325718755}]},"public_addr":"172.21.15.112:6811/3325718755","cluster_addr":"172.21.15.112:6813/3325718755","heartbeat_back_addr":"172.21.15.112:6817/3325718755","heartbeat_front_addr":"172.21.15.112:6815/3325718755","state":["exists","up"]},{"osd":4,"uuid":"cb4f1a42-8de8-4b50-b3bc-602de9f4803a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6809","nonce":1548914683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6811","nonce":1548914683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6815","nonce":1548914683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6813","nonce":1548914683}]},"public_addr":"172.21.15.150:6809/1548914683","cluster_addr":"172.21.15.150:6811/1548914683","heartbeat_back_addr":"172.21.15.150:6815/1548914683","heartbeat_front_addr":"172.21.15.150:6813/1548914683","state":["exists","up"]},{"osd":5,"uuid":"a26c03b8-050c-4a2c-8f23-5801fffb7c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6809","nonce":409836750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6811","nonce":409836750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6815","nonce":409836750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6813","nonce":409836750}]},"public_addr":"172.21.15.162:6809/409836750","cluster_addr":"172.21.15.162:6811/409836750","heartbeat_back_addr":"172.21.15.162:6815/409836750","heartbeat_front_addr":"172.21.15.162:6813/409836750","state":["exists","up"]},{"osd":6,"uuid":"8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6819","nonce":1098984186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6821","nonce":1098984186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6825","nonce":1098984186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6823","nonce":1098984186}]},"public_addr":"172.21.15.112:6819/1098984186","cluster_addr":"172.21.15.112:6821/1098984186","heartbeat_back_addr":"172.21.15.112:6825/1098984186","heartbeat_front_addr":"172.21.15.112:6823/1098984186","state":["exists","up"]},{"osd":7,"uuid":"af3629b4-9bed-4669-97a6-48deba503a96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6817","nonce":4009836564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6819","nonce":4009836564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6823","nonce":4009836564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6821","nonce":4009836564}]},"public_addr":"172.21.15.150:6817/4009836564","cluster_addr":"172.21.15.150:6819/4009836564","heartbeat_back_addr":"172.21.15.150:6823/4009836564","heartbeat_front_addr":"172.21.15.150:6821/4009836564","state":["exists","up"]},{"osd":8,"uuid":"9333e87d-dd78-4467-bbde-c2b44f4ea07f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6817","nonce":852791606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6819","nonce":852791606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6823","nonce":852791606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6821","nonce":852791606}]},"public_addr":"172.21.15.162:6817/852791606","cluster_addr":"172.21.15.162:6819/852791606","heartbeat_back_addr":"172.21.15.162:6823/852791606","heartbeat_front_addr":"172.21.15.162:6821/852791606","state":["exists","up"]},{"osd":9,"uuid":"09723581-1995-42e0-8db7-776ea98a918c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6827","nonce":2186764674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6829","nonce":2186764674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6833","nonce":2186764674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6831","nonce":2186764674}]},"public_addr":"172.21.15.112:6827/2186764674","cluster_addr":"172.21.15.112:6829/2186764674","heartbeat_back_addr":"172.21.15.112:6833/2186764674","heartbeat_front_addr":"172.21.15.112:6831/2186764674","state":["exists","up"]},{"osd":10,"uuid":"34e08e3e-1231-4899-a5b3-76d449105bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6825","nonce":2838995474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6827","nonce":2838995474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6831","nonce":2838995474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6829","nonce":2838995474}]},"public_addr":"172.21.15.150:6825/2838995474","cluster_addr":"172.21.15.150:6827/2838995474","heartbeat_back_addr":"172.21.15.150:6831/2838995474","heartbeat_front_addr":"172.21.15.150:6829/2838995474","state":["exists","up"]},{"osd":11,"uuid":"6fc34ec0-78ca-403c-acb7-b29d83ff64e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6825","nonce":3702614331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6827","nonce":3702614331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6831","nonce":3702614331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6829","nonce":3702614331}]},"public_addr":"172.21.15.162:6825/3702614331","cluster_addr":"172.21.15.162:6827/3702614331","heartbeat_back_addr":"172.21.15.162:6831/3702614331","heartbeat_front_addr":"172.21.15.162:6829/3702614331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:54.518702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:19.939543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:44.425586+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:16.126471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:41.696071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:05.517405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:38.549318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:03.799649+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:28.007282+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:01.395459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:27.963637+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:52.181212+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3667148497":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6800/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3425327158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1404611205":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3765334043":"2024-05-09T05:42:29.819307+0000","172.21.15.112:6800/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/2814303790":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1091751583":"2024-05-09T05:42:29.819307+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:52:21.368 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:21 smithi162 ceph-mon[34800]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:21.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.369+0000 7fb546600700 1 -- 172.21.15.112:0/3542767133 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb5240b2990 msgr2=0x7fb5240b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:21.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.369+0000 7fb546600700 1 --2- 172.21.15.112:0/3542767133 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb5240b2990 0x7fb5240b4e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb530006010 tx=0x7fb53000b280 comp rx=0 tx=0).stop 2024-05-08T05:52:21.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.369+0000 7fb546600700 1 -- 172.21.15.112:0/3542767133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb54010fea0 msgr2=0x7fb5401a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:21.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.369+0000 7fb546600700 1 --2- 172.21.15.112:0/3542767133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb54010fea0 0x7fb5401a0690 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb53800ef90 tx=0x7fb53800c3a0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.369+0000 7fb546600700 1 -- 172.21.15.112:0/3542767133 shutdown_connections 2024-05-08T05:52:21.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.369+0000 7fb546600700 1 --2- 172.21.15.112:0/3542767133 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb5240b2990 0x7fb5240b4e50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.369+0000 7fb546600700 1 --2- 172.21.15.112:0/3542767133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb54010fea0 0x7fb5401a0690 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.369+0000 7fb546600700 1 --2- 172.21.15.112:0/3542767133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5400756d0 0x7fb54019be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.369+0000 7fb546600700 1 --2- 172.21.15.112:0/3542767133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb540075ff0 0x7fb54019c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.369+0000 7fb546600700 1 -- 172.21.15.112:0/3542767133 >> 172.21.15.112:0/3542767133 conn(0x7fb54007a600 msgr2=0x7fb540101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:21.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.369+0000 7fb546600700 1 -- 172.21.15.112:0/3542767133 shutdown_connections 2024-05-08T05:52:21.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:21.369+0000 7fb546600700 1 -- 172.21.15.112:0/3542767133 wait complete. 2024-05-08T05:52:21.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:21 smithi150 ceph-mon[34315]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:22.194 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-08T05:52:22.195 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-08T05:52:22.195 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-08T05:52:22.195 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-08T05:52:22.196 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-08T05:52:22.196 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-08T05:52:22.196 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-08T05:52:22.197 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-08T05:52:22.197 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-08T05:52:22.198 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-08T05:52:22.198 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-08T05:52:22.199 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-08T05:52:22.355 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:22 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3542767133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:52:22.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:22 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3542767133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:52:22.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:22 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3542767133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:52:23.142 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:23.175 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:23.247 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:23.264 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:23.400 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:23 smithi112 ceph-mon[30003]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:23.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:23 smithi150 ceph-mon[34315]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:23.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:23 smithi162 ceph-mon[34800]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:24.582 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:24.595 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:24.598 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:25.073 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:25.073 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:25.077 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:25.209 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:25 smithi112 ceph-mon[30003]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:52:25.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:25 smithi150 ceph-mon[34315]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:52:25.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:25 smithi162 ceph-mon[34800]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:52:26.518 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:26.520 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:27.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:27 smithi150 ceph-mon[34315]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:27.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:27 smithi112 ceph-mon[30003]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:27 smithi162 ceph-mon[34800]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:29.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:29 smithi112 ceph-mon[30003]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:29.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:29 smithi150 ceph-mon[34315]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:29.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:29 smithi162 ceph-mon[34800]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:31.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:31 smithi150 ceph-mon[34315]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:31.553 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:31 smithi112 ceph-mon[30003]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:31.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:31 smithi162 ceph-mon[34800]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:33.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:33 smithi150 ceph-mon[34315]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:33.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:33 smithi112 ceph-mon[30003]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:33.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:33 smithi162 ceph-mon[34800]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:35.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:35 smithi112 ceph-mon[30003]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:52:35.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:35 smithi150 ceph-mon[34315]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:52:35.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.664+0000 7f89d8600700 1 -- 172.21.15.112:0/3870093187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89d0103750 msgr2=0x7f89d0103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.664+0000 7f89d8600700 1 --2- 172.21.15.112:0/3870093187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89d0103750 0x7f89d0103b30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f89c000b0d0 tx=0x7f89c000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.665+0000 7f89d8600700 1 -- 172.21.15.112:0/3870093187 shutdown_connections 2024-05-08T05:52:35.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.665+0000 7f89d8600700 1 --2- 172.21.15.112:0/3870093187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89d0108850 0x7f89d01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.665+0000 7f89d8600700 1 --2- 172.21.15.112:0/3870093187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89d0103750 0x7f89d0103b30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.665+0000 7f89d8600700 1 --2- 172.21.15.112:0/3870093187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d0104100 0x7f89d0108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.665+0000 7f89d8600700 1 -- 172.21.15.112:0/3870093187 >> 172.21.15.112:0/3870093187 conn(0x7f89d007a600 msgr2=0x7f89d007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.665+0000 7f89d8600700 1 -- 172.21.15.112:0/3870093187 shutdown_connections 2024-05-08T05:52:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.665+0000 7f89d8600700 1 -- 172.21.15.112:0/3870093187 wait complete. 2024-05-08T05:52:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.666+0000 7f89d8600700 1 Processor -- start 2024-05-08T05:52:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.666+0000 7f89d8600700 1 -- start start 2024-05-08T05:52:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.666+0000 7f89d8600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89d0103750 0x7f89d019bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.666+0000 7f89d8600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d0104100 0x7f89d019c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.666+0000 7f89d8600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89d0108850 0x7f89d01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.666+0000 7f89d8600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89d0113670 con 0x7f89d0103750 2024-05-08T05:52:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.666+0000 7f89d8600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01134f0 con 0x7f89d0104100 2024-05-08T05:52:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.666+0000 7f89d8600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01137f0 con 0x7f89d0108850 2024-05-08T05:52:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.667+0000 7f89d5c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89d0103750 0x7f89d019bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.667+0000 7f89d5200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d0104100 0x7f89d019c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:35.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.667+0000 7f89d5c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89d0103750 0x7f89d019bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46390/0 (socket says 172.21.15.112:46390) 2024-05-08T05:52:35.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.667+0000 7f89d5c00700 1 -- 172.21.15.112:0/330407860 learned_addr learned my addr 172.21.15.112:0/330407860 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:35.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.667+0000 7f89d6600700 1 --2- 172.21.15.112:0/330407860 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89d0108850 0x7f89d01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:35.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.667+0000 7f89d5200700 1 -- 172.21.15.112:0/330407860 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89d0108850 msgr2=0x7f89d01a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.667+0000 7f89d5200700 1 --2- 172.21.15.112:0/330407860 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89d0108850 0x7f89d01a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.667+0000 7f89d5200700 1 -- 172.21.15.112:0/330407860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89d0103750 msgr2=0x7f89d019bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.667+0000 7f89d5200700 1 --2- 172.21.15.112:0/330407860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89d0103750 0x7f89d019bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.667+0000 7f89d5200700 1 -- 172.21.15.112:0/330407860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d01a0d50 con 0x7f89d0104100 2024-05-08T05:52:35.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.668+0000 7f89d5200700 1 --2- 172.21.15.112:0/330407860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d0104100 0x7f89d019c280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00e9f0 tx=0x7f89bc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:35.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.668+0000 7f89ce000700 1 -- 172.21.15.112:0/330407860 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00cbd0 con 0x7f89d0104100 2024-05-08T05:52:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.668+0000 7f89d8600700 1 -- 172.21.15.112:0/330407860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d01a1040 con 0x7f89d0104100 2024-05-08T05:52:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.668+0000 7f89ce000700 1 -- 172.21.15.112:0/330407860 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89bc00cd70 con 0x7f89d0104100 2024-05-08T05:52:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.668+0000 7f89ce000700 1 -- 172.21.15.112:0/330407860 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc018680 con 0x7f89d0104100 2024-05-08T05:52:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.668+0000 7f89d8600700 1 -- 172.21.15.112:0/330407860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d01a88c0 con 0x7f89d0104100 2024-05-08T05:52:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.669+0000 7f89d8600700 1 -- 172.21.15.112:0/330407860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f89d0076a30 con 0x7f89d0104100 2024-05-08T05:52:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.673+0000 7f89ce000700 1 -- 172.21.15.112:0/330407860 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f89bc010430 con 0x7f89d0104100 2024-05-08T05:52:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.674+0000 7f89ce000700 1 --2- 172.21.15.112:0/330407860 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f89b40b2990 0x7f89b40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.674+0000 7f89ce000700 1 -- 172.21.15.112:0/330407860 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7f89bc0e29e0 con 0x7f89d0104100 2024-05-08T05:52:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.674+0000 7f89d5c00700 1 --2- 172.21.15.112:0/330407860 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f89b40b2990 0x7f89b40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.674+0000 7f89ce000700 1 --2- 172.21.15.112:0/330407860 >> [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820] conn(0x7f89b40bc550 0x7f89b40be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.674+0000 7f89ce000700 1 -- 172.21.15.112:0/330407860 --> [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f89b40bf0c0 con 0x7f89b40bc550 2024-05-08T05:52:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.674+0000 7f89ce000700 1 -- 172.21.15.112:0/330407860 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f89bc0e2d90 con 0x7f89d0104100 2024-05-08T05:52:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.675+0000 7f89d6600700 1 --2- 172.21.15.112:0/330407860 >> [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820] conn(0x7f89b40bc550 0x7f89b40be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.675+0000 7f89d5c00700 1 --2- 172.21.15.112:0/330407860 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f89b40b2990 0x7f89b40b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009fd0 tx=0x7f89c000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.675+0000 7f89d6600700 1 --2- 172.21.15.112:0/330407860 >> [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820] conn(0x7f89b40bc550 0x7f89b40be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.676+0000 7f89ce000700 1 -- 172.21.15.112:0/330407860 <== osd.0 v2:172.21.15.112:6802/3419091820 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f89b40bf0c0 con 0x7f89b40bc550 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.694+0000 7f89d8600700 1 -- 172.21.15.112:0/330407860 --> [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f89d0104be0 con 0x7f89b40bc550 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.694+0000 7f89ce000700 1 -- 172.21.15.112:0/330407860 <== osd.0 v2:172.21.15.112:6802/3419091820 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f89d0104be0 con 0x7f89b40bc550 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.695+0000 7f89d8600700 1 -- 172.21.15.112:0/330407860 >> [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820] conn(0x7f89b40bc550 msgr2=0x7f89b40be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.695+0000 7f89d8600700 1 --2- 172.21.15.112:0/330407860 >> [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820] conn(0x7f89b40bc550 0x7f89b40be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.695+0000 7f89d8600700 1 -- 172.21.15.112:0/330407860 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f89b40b2990 msgr2=0x7f89b40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.695+0000 7f89d8600700 1 --2- 172.21.15.112:0/330407860 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f89b40b2990 0x7f89b40b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009fd0 tx=0x7f89c000b040 comp rx=0 tx=0).stop 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.695+0000 7f89d8600700 1 -- 172.21.15.112:0/330407860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d0104100 msgr2=0x7f89d019c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.695+0000 7f89d8600700 1 --2- 172.21.15.112:0/330407860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89d0104100 0x7f89d019c280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00e9f0 tx=0x7f89bc00ed00 comp rx=0 tx=0).stop 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.695+0000 7f89d6600700 1 -- 172.21.15.112:0/330407860 reap_dead start 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.695+0000 7f89d8600700 1 -- 172.21.15.112:0/330407860 shutdown_connections 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.695+0000 7f89d8600700 1 -- 172.21.15.112:0/330407860 >> 172.21.15.112:0/330407860 conn(0x7f89d007a600 msgr2=0x7f89d0101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.695+0000 7f89d8600700 1 -- 172.21.15.112:0/330407860 shutdown_connections 2024-05-08T05:52:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:29.695+0000 7f89d8600700 1 -- 172.21.15.112:0/330407860 wait complete. 2024-05-08T05:52:35.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:35 smithi162 ceph-mon[34800]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:52:35.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.732+0000 7f7678400700 1 -- 172.21.15.112:0/3035676242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7670102530 msgr2=0x7f7670112220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.732+0000 7f7678400700 1 --2- 172.21.15.112:0/3035676242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7670102530 0x7f7670112220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f766800b0a0 tx=0x7f766800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.733+0000 7f7678400700 1 -- 172.21.15.112:0/3035676242 shutdown_connections 2024-05-08T05:52:35.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.733+0000 7f7678400700 1 --2- 172.21.15.112:0/3035676242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7670102530 0x7f7670112220 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.733+0000 7f7678400700 1 --2- 172.21.15.112:0/3035676242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76701043b0 0x7f7670104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.733+0000 7f7678400700 1 --2- 172.21.15.112:0/3035676242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7670101a90 0x7f7670101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.733+0000 7f7678400700 1 -- 172.21.15.112:0/3035676242 >> 172.21.15.112:0/3035676242 conn(0x7f7670078a10 msgr2=0x7f76700ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:35.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.733+0000 7f7678400700 1 -- 172.21.15.112:0/3035676242 shutdown_connections 2024-05-08T05:52:35.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.733+0000 7f7678400700 1 -- 172.21.15.112:0/3035676242 wait complete. 2024-05-08T05:52:35.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.734+0000 7f7678400700 1 Processor -- start 2024-05-08T05:52:35.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.734+0000 7f7678400700 1 -- start start 2024-05-08T05:52:35.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.734+0000 7f7678400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7670101a90 0x7f767019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:35.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.734+0000 7f7678400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7670102530 0x7f767019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:35.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.734+0000 7f7678400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76701043b0 0x7f76701a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:35.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.734+0000 7f7678400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76701158b0 con 0x7f7670102530 2024-05-08T05:52:35.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.734+0000 7f7678400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7670115730 con 0x7f7670101a90 2024-05-08T05:52:35.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.734+0000 7f7678400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7670115a30 con 0x7f76701043b0 2024-05-08T05:52:35.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.735+0000 7f7675000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7670102530 0x7f767019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:35.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.735+0000 7f7676400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76701043b0 0x7f76701a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:35.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.735+0000 7f7675a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7670101a90 0x7f767019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.735+0000 7f7675000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7670102530 0x7f767019c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47364/0 (socket says 172.21.15.112:47364) 2024-05-08T05:52:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.735+0000 7f7675000700 1 -- 172.21.15.112:0/3445764473 learned_addr learned my addr 172.21.15.112:0/3445764473 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.735+0000 7f7675a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7670101a90 0x7f767019bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:51258/0 (socket says 172.21.15.112:51258) 2024-05-08T05:52:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.735+0000 7f7676400700 1 -- 172.21.15.112:0/3445764473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7670101a90 msgr2=0x7f767019bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.735+0000 7f7676400700 1 --2- 172.21.15.112:0/3445764473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7670101a90 0x7f767019bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.735+0000 7f7676400700 1 -- 172.21.15.112:0/3445764473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7670102530 msgr2=0x7f767019c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.735+0000 7f7676400700 1 --2- 172.21.15.112:0/3445764473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7670102530 0x7f767019c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.735+0000 7f7676400700 1 -- 172.21.15.112:0/3445764473 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76701a0d70 con 0x7f76701043b0 2024-05-08T05:52:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.735+0000 7f7676400700 1 --2- 172.21.15.112:0/3445764473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76701043b0 0x7f76701a0630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f766800b070 tx=0x7f7668004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.736+0000 7f766e000700 1 -- 172.21.15.112:0/3445764473 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668021070 con 0x7f76701043b0 2024-05-08T05:52:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.736+0000 7f766e000700 1 -- 172.21.15.112:0/3445764473 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7668007e30 con 0x7f76701043b0 2024-05-08T05:52:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.736+0000 7f7678400700 1 -- 172.21.15.112:0/3445764473 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76701a1000 con 0x7f76701043b0 2024-05-08T05:52:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.736+0000 7f766e000700 1 -- 172.21.15.112:0/3445764473 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76680045f0 con 0x7f76701043b0 2024-05-08T05:52:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.736+0000 7f7678400700 1 -- 172.21.15.112:0/3445764473 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7670104db0 con 0x7f76701043b0 2024-05-08T05:52:35.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.737+0000 7f7678400700 1 -- 172.21.15.112:0/3445764473 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7670063c30 con 0x7f76701043b0 2024-05-08T05:52:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.739+0000 7f766e000700 1 -- 172.21.15.112:0/3445764473 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f7668007980 con 0x7f76701043b0 2024-05-08T05:52:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.741+0000 7f766e000700 1 --2- 172.21.15.112:0/3445764473 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f76580b2990 0x7f76580b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.741+0000 7f766e000700 1 -- 172.21.15.112:0/3445764473 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7f766800f020 con 0x7f76701043b0 2024-05-08T05:52:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.741+0000 7f7675a00700 1 --2- 172.21.15.112:0/3445764473 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f76580b2990 0x7f76580b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.741+0000 7f766e000700 1 --2- 172.21.15.112:0/3445764473 >> [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674] conn(0x7f76580bc550 0x7f76580be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.741+0000 7f766e000700 1 -- 172.21.15.112:0/3445764473 --> [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76580bf0c0 con 0x7f76580bc550 2024-05-08T05:52:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.741+0000 7f766e000700 1 -- 172.21.15.112:0/3445764473 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f76680e3cc0 con 0x7f76701043b0 2024-05-08T05:52:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.742+0000 7f7675000700 1 --2- 172.21.15.112:0/3445764473 >> [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674] conn(0x7f76580bc550 0x7f76580be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.742+0000 7f7675a00700 1 --2- 172.21.15.112:0/3445764473 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f76580b2990 0x7f76580b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7664009be0 tx=0x7f7664009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.743+0000 7f7675000700 1 --2- 172.21.15.112:0/3445764473 >> [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674] conn(0x7f76580bc550 0x7f76580be9f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.744+0000 7f766e000700 1 -- 172.21.15.112:0/3445764473 <== osd.9 v2:172.21.15.112:6826/2186764674 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76580bf0c0 con 0x7f76580bc550 2024-05-08T05:52:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.762+0000 7f7678400700 1 -- 172.21.15.112:0/3445764473 --> [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f76700640b0 con 0x7f76580bc550 2024-05-08T05:52:35.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.762+0000 7f766e000700 1 -- 172.21.15.112:0/3445764473 <== osd.9 v2:172.21.15.112:6826/2186764674 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f76700640b0 con 0x7f76580bc550 2024-05-08T05:52:35.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.763+0000 7f7678400700 1 -- 172.21.15.112:0/3445764473 >> [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674] conn(0x7f76580bc550 msgr2=0x7f76580be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.763+0000 7f7678400700 1 --2- 172.21.15.112:0/3445764473 >> [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674] conn(0x7f76580bc550 0x7f76580be9f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.763+0000 7f7678400700 1 -- 172.21.15.112:0/3445764473 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f76580b2990 msgr2=0x7f76580b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.763+0000 7f7678400700 1 --2- 172.21.15.112:0/3445764473 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f76580b2990 0x7f76580b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7664009be0 tx=0x7f7664009260 comp rx=0 tx=0).stop 2024-05-08T05:52:35.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.763+0000 7f7678400700 1 -- 172.21.15.112:0/3445764473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76701043b0 msgr2=0x7f76701a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.763+0000 7f7678400700 1 --2- 172.21.15.112:0/3445764473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76701043b0 0x7f76701a0630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f766800b070 tx=0x7f7668004c80 comp rx=0 tx=0).stop 2024-05-08T05:52:35.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.763+0000 7f7676400700 1 -- 172.21.15.112:0/3445764473 reap_dead start 2024-05-08T05:52:35.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.763+0000 7f7678400700 1 -- 172.21.15.112:0/3445764473 shutdown_connections 2024-05-08T05:52:35.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.763+0000 7f7678400700 1 -- 172.21.15.112:0/3445764473 >> 172.21.15.112:0/3445764473 conn(0x7f7670078a10 msgr2=0x7f76700ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:35.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.763+0000 7f7678400700 1 -- 172.21.15.112:0/3445764473 shutdown_connections 2024-05-08T05:52:35.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:35.763+0000 7f7678400700 1 -- 172.21.15.112:0/3445764473 wait complete. 2024-05-08T05:52:37.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.024+0000 7fde09c00700 1 -- 172.21.15.112:0/3009098753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde040756d0 msgr2=0x7fde04075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:37.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.024+0000 7fde09c00700 1 --2- 172.21.15.112:0/3009098753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde040756d0 0x7fde04075ab0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009990 tx=0x7fddf4009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:37.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.024+0000 7fde09c00700 1 -- 172.21.15.112:0/3009098753 shutdown_connections 2024-05-08T05:52:37.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.024+0000 7fde09c00700 1 --2- 172.21.15.112:0/3009098753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde0410fea0 0x7fde04112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:37.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.025+0000 7fde09c00700 1 --2- 172.21.15.112:0/3009098753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde040756d0 0x7fde04075ab0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:37.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.025+0000 7fde09c00700 1 --2- 172.21.15.112:0/3009098753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde04075ff0 0x7fde0410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:37.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.025+0000 7fde09c00700 1 -- 172.21.15.112:0/3009098753 >> 172.21.15.112:0/3009098753 conn(0x7fde0407a600 msgr2=0x7fde0407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:37.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.025+0000 7fde09c00700 1 -- 172.21.15.112:0/3009098753 shutdown_connections 2024-05-08T05:52:37.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.025+0000 7fde09c00700 1 -- 172.21.15.112:0/3009098753 wait complete. 2024-05-08T05:52:37.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.026+0000 7fde09c00700 1 Processor -- start 2024-05-08T05:52:37.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.026+0000 7fde09c00700 1 -- start start 2024-05-08T05:52:37.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.026+0000 7fde09c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde040756d0 0x7fde0419bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:37.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.026+0000 7fde09c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde04075ff0 0x7fde0419c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:37.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.026+0000 7fde09c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde0410fea0 0x7fde041a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:37.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.026+0000 7fde09c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde041158d0 con 0x7fde04075ff0 2024-05-08T05:52:37.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.026+0000 7fde09c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fde04115750 con 0x7fde0410fea0 2024-05-08T05:52:37.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.026+0000 7fde09c00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fde04115a50 con 0x7fde040756d0 2024-05-08T05:52:37.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.027+0000 7fddfa000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde04075ff0 0x7fde0419c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:37.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.027+0000 7fde02a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde040756d0 0x7fde0419bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:37.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.027+0000 7fddfa000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde04075ff0 0x7fde0419c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47374/0 (socket says 172.21.15.112:47374) 2024-05-08T05:52:37.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.027+0000 7fddfa000700 1 -- 172.21.15.112:0/1047973003 learned_addr learned my addr 172.21.15.112:0/1047973003 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:37.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.027+0000 7fde03400700 1 --2- 172.21.15.112:0/1047973003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde0410fea0 0x7fde041a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:37.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.027+0000 7fddfa000700 1 -- 172.21.15.112:0/1047973003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde040756d0 msgr2=0x7fde0419bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:37.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.027+0000 7fddfa000700 1 --2- 172.21.15.112:0/1047973003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde040756d0 0x7fde0419bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:37.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.027+0000 7fddfa000700 1 -- 172.21.15.112:0/1047973003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde0410fea0 msgr2=0x7fde041a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:37.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.027+0000 7fddfa000700 1 --2- 172.21.15.112:0/1047973003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde0410fea0 0x7fde041a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.027+0000 7fddfa000700 1 -- 172.21.15.112:0/1047973003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde041a0d50 con 0x7fde04075ff0 2024-05-08T05:52:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.028+0000 7fddfa000700 1 --2- 172.21.15.112:0/1047973003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde04075ff0 0x7fde0419c280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fddec00eb30 tx=0x7fddec00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.028+0000 7fddfbe00700 1 -- 172.21.15.112:0/1047973003 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec00cbd0 con 0x7fde04075ff0 2024-05-08T05:52:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.028+0000 7fddfbe00700 1 -- 172.21.15.112:0/1047973003 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddec0044d0 con 0x7fde04075ff0 2024-05-08T05:52:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.028+0000 7fddfbe00700 1 -- 172.21.15.112:0/1047973003 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec003e30 con 0x7fde04075ff0 2024-05-08T05:52:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.028+0000 7fde09c00700 1 -- 172.21.15.112:0/1047973003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde041a1040 con 0x7fde04075ff0 2024-05-08T05:52:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.028+0000 7fde09c00700 1 -- 172.21.15.112:0/1047973003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde041a88c0 con 0x7fde04075ff0 2024-05-08T05:52:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.029+0000 7fde09c00700 1 -- 172.21.15.112:0/1047973003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fde04063c30 con 0x7fde04075ff0 2024-05-08T05:52:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.031+0000 7fddfbe00700 1 -- 172.21.15.112:0/1047973003 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fddec004000 con 0x7fde04075ff0 2024-05-08T05:52:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.033+0000 7fddfbe00700 1 --2- 172.21.15.112:0/1047973003 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fdde80b2910 0x7fdde80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.033+0000 7fddfbe00700 1 -- 172.21.15.112:0/1047973003 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7fddec0e2c10 con 0x7fde04075ff0 2024-05-08T05:52:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.033+0000 7fde02a00700 1 --2- 172.21.15.112:0/1047973003 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fdde80b2910 0x7fdde80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.033+0000 7fddfbe00700 1 --2- 172.21.15.112:0/1047973003 >> [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564] conn(0x7fdde80bc4d0 0x7fdde80be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.033+0000 7fddfbe00700 1 -- 172.21.15.112:0/1047973003 --> [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fddec022e50 con 0x7fdde80bc4d0 2024-05-08T05:52:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.033+0000 7fddfbe00700 1 -- 172.21.15.112:0/1047973003 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fddec0e2fc0 con 0x7fde04075ff0 2024-05-08T05:52:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.034+0000 7fde03400700 1 --2- 172.21.15.112:0/1047973003 >> [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564] conn(0x7fdde80bc4d0 0x7fdde80be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.034+0000 7fde02a00700 1 --2- 172.21.15.112:0/1047973003 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fdde80b2910 0x7fdde80b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fddf400b2e0 tx=0x7fddf4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.034+0000 7fde03400700 1 --2- 172.21.15.112:0/1047973003 >> [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564] conn(0x7fdde80bc4d0 0x7fdde80be970 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.035+0000 7fddfbe00700 1 -- 172.21.15.112:0/1047973003 <== osd.7 v2:172.21.15.150:6816/4009836564 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fddec022e50 con 0x7fdde80bc4d0 2024-05-08T05:52:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.053+0000 7fde09c00700 1 -- 172.21.15.112:0/1047973003 --> [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fde040640b0 con 0x7fdde80bc4d0 2024-05-08T05:52:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.054+0000 7fddfbe00700 1 -- 172.21.15.112:0/1047973003 <== osd.7 v2:172.21.15.150:6816/4009836564 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fde040640b0 con 0x7fdde80bc4d0 2024-05-08T05:52:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.055+0000 7fde09c00700 1 -- 172.21.15.112:0/1047973003 >> [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564] conn(0x7fdde80bc4d0 msgr2=0x7fdde80be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.055+0000 7fde09c00700 1 --2- 172.21.15.112:0/1047973003 >> [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564] conn(0x7fdde80bc4d0 0x7fdde80be970 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.055+0000 7fde09c00700 1 -- 172.21.15.112:0/1047973003 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fdde80b2910 msgr2=0x7fdde80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.055+0000 7fde09c00700 1 --2- 172.21.15.112:0/1047973003 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fdde80b2910 0x7fdde80b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fddf400b2e0 tx=0x7fddf4005fd0 comp rx=0 tx=0).stop 2024-05-08T05:52:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.055+0000 7fde09c00700 1 -- 172.21.15.112:0/1047973003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde04075ff0 msgr2=0x7fde0419c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.055+0000 7fde09c00700 1 --2- 172.21.15.112:0/1047973003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde04075ff0 0x7fde0419c280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fddec00eb30 tx=0x7fddec00ee40 comp rx=0 tx=0).stop 2024-05-08T05:52:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.055+0000 7fde03400700 1 -- 172.21.15.112:0/1047973003 reap_dead start 2024-05-08T05:52:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.055+0000 7fde09c00700 1 -- 172.21.15.112:0/1047973003 shutdown_connections 2024-05-08T05:52:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.055+0000 7fde09c00700 1 -- 172.21.15.112:0/1047973003 >> 172.21.15.112:0/1047973003 conn(0x7fde0407a600 msgr2=0x7fde041006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.055+0000 7fde09c00700 1 -- 172.21.15.112:0/1047973003 shutdown_connections 2024-05-08T05:52:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:37.055+0000 7fde09c00700 1 -- 172.21.15.112:0/1047973003 wait complete. 2024-05-08T05:52:37.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:37 smithi150 ceph-mon[34315]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:37.458 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:37 smithi112 ceph-mon[30003]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:37.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:37 smithi162 ceph-mon[34800]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:38.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.714+0000 7fe95c600700 1 -- 172.21.15.112:0/1506352157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe954103eb0 msgr2=0x7fe954110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:38.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.714+0000 7fe95c600700 1 --2- 172.21.15.112:0/1506352157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe954103eb0 0x7fe954110390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00b0a0 tx=0x7fe94c00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:52:38.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.714+0000 7fe95c600700 1 -- 172.21.15.112:0/1506352157 shutdown_connections 2024-05-08T05:52:38.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.714+0000 7fe95c600700 1 --2- 172.21.15.112:0/1506352157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe954103eb0 0x7fe954110390 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:38.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.714+0000 7fe95c600700 1 --2- 172.21.15.112:0/1506352157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe954109600 0x7fe9541099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:38.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.714+0000 7fe95c600700 1 --2- 172.21.15.112:0/1506352157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9541034f0 0x7fe954103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:38.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.714+0000 7fe95c600700 1 -- 172.21.15.112:0/1506352157 >> 172.21.15.112:0/1506352157 conn(0x7fe95407a650 msgr2=0x7fe95407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:38.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.714+0000 7fe95c600700 1 -- 172.21.15.112:0/1506352157 shutdown_connections 2024-05-08T05:52:38.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.714+0000 7fe95c600700 1 -- 172.21.15.112:0/1506352157 wait complete. 2024-05-08T05:52:38.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.716+0000 7fe95c600700 1 Processor -- start 2024-05-08T05:52:38.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.716+0000 7fe95c600700 1 -- start start 2024-05-08T05:52:38.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.716+0000 7fe95c600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9541034f0 0x7fe95419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:38.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.716+0000 7fe95c600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe954103eb0 0x7fe95419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:38.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.716+0000 7fe95c600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954109600 0x7fe954199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:38.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.716+0000 7fe95c600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9541136a0 con 0x7fe954109600 2024-05-08T05:52:38.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.716+0000 7fe95c600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe954113520 con 0x7fe9541034f0 2024-05-08T05:52:38.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.716+0000 7fe95c600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe954113820 con 0x7fe954103eb0 2024-05-08T05:52:38.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.717+0000 7fe959200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe954103eb0 0x7fe95419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:38.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.717+0000 7fe959200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe954103eb0 0x7fe95419fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:52610/0 (socket says 172.21.15.112:52610) 2024-05-08T05:52:38.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.717+0000 7fe959200700 1 -- 172.21.15.112:0/849407843 learned_addr learned my addr 172.21.15.112:0/849407843 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:38.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.717+0000 7fe959c00700 1 --2- 172.21.15.112:0/849407843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9541034f0 0x7fe95419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.717+0000 7fe95a600700 1 --2- 172.21.15.112:0/849407843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954109600 0x7fe954199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.717+0000 7fe959200700 1 -- 172.21.15.112:0/849407843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9541034f0 msgr2=0x7fe95419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.717+0000 7fe959200700 1 --2- 172.21.15.112:0/849407843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9541034f0 0x7fe95419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.717+0000 7fe959200700 1 -- 172.21.15.112:0/849407843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954109600 msgr2=0x7fe954199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.717+0000 7fe959200700 1 --2- 172.21.15.112:0/849407843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954109600 0x7fe954199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.717+0000 7fe959200700 1 -- 172.21.15.112:0/849407843 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95419a160 con 0x7fe954103eb0 2024-05-08T05:52:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.717+0000 7fe959200700 1 --2- 172.21.15.112:0/849407843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe954103eb0 0x7fe95419fd80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe94000e9a0 tx=0x7fe94000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.718+0000 7fe952000700 1 -- 172.21.15.112:0/849407843 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94000cbd0 con 0x7fe954103eb0 2024-05-08T05:52:38.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.718+0000 7fe952000700 1 -- 172.21.15.112:0/849407843 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94000cd70 con 0x7fe954103eb0 2024-05-08T05:52:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.718+0000 7fe952000700 1 -- 172.21.15.112:0/849407843 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940018680 con 0x7fe954103eb0 2024-05-08T05:52:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.718+0000 7fe95c600700 1 -- 172.21.15.112:0/849407843 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95419a450 con 0x7fe954103eb0 2024-05-08T05:52:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.718+0000 7fe95c600700 1 -- 172.21.15.112:0/849407843 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9541a6710 con 0x7fe954103eb0 2024-05-08T05:52:38.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.719+0000 7fe95c600700 1 -- 172.21.15.112:0/849407843 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe954063b70 con 0x7fe954103eb0 2024-05-08T05:52:38.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.722+0000 7fe952000700 1 -- 172.21.15.112:0/849407843 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe940010430 con 0x7fe954103eb0 2024-05-08T05:52:38.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.723+0000 7fe952000700 1 --2- 172.21.15.112:0/849407843 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe93c0b2990 0x7fe93c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.724+0000 7fe952000700 1 -- 172.21.15.112:0/849407843 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7fe9400e32b0 con 0x7fe954103eb0 2024-05-08T05:52:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.724+0000 7fe959c00700 1 --2- 172.21.15.112:0/849407843 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe93c0b2990 0x7fe93c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:38.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.724+0000 7fe952000700 1 --2- 172.21.15.112:0/849407843 >> [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750] conn(0x7fe93c0bc550 0x7fe93c0be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:38.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.724+0000 7fe952000700 1 -- 172.21.15.112:0/849407843 --> [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe940016ec0 con 0x7fe93c0bc550 2024-05-08T05:52:38.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.724+0000 7fe952000700 1 -- 172.21.15.112:0/849407843 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fe9400e3700 con 0x7fe954103eb0 2024-05-08T05:52:38.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.725+0000 7fe95a600700 1 --2- 172.21.15.112:0/849407843 >> [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750] conn(0x7fe93c0bc550 0x7fe93c0be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:38.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.725+0000 7fe959c00700 1 --2- 172.21.15.112:0/849407843 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe93c0b2990 0x7fe93c0b4e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe95419ab40 tx=0x7fe948009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:38.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.725+0000 7fe95a600700 1 --2- 172.21.15.112:0/849407843 >> [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750] conn(0x7fe93c0bc550 0x7fe93c0be9f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:38.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.726+0000 7fe952000700 1 -- 172.21.15.112:0/849407843 <== osd.5 v2:172.21.15.162:6808/409836750 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe940016ec0 con 0x7fe93c0bc550 2024-05-08T05:52:38.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.745+0000 7fe95c600700 1 -- 172.21.15.112:0/849407843 --> [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe954063ff0 con 0x7fe93c0bc550 2024-05-08T05:52:38.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.745+0000 7fe952000700 1 -- 172.21.15.112:0/849407843 <== osd.5 v2:172.21.15.162:6808/409836750 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe954063ff0 con 0x7fe93c0bc550 2024-05-08T05:52:38.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.746+0000 7fe95c600700 1 -- 172.21.15.112:0/849407843 >> [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750] conn(0x7fe93c0bc550 msgr2=0x7fe93c0be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:38.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.746+0000 7fe95c600700 1 --2- 172.21.15.112:0/849407843 >> [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750] conn(0x7fe93c0bc550 0x7fe93c0be9f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:38.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.746+0000 7fe95c600700 1 -- 172.21.15.112:0/849407843 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe93c0b2990 msgr2=0x7fe93c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:38.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.746+0000 7fe95c600700 1 --2- 172.21.15.112:0/849407843 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe93c0b2990 0x7fe93c0b4e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe95419ab40 tx=0x7fe948009260 comp rx=0 tx=0).stop 2024-05-08T05:52:38.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.746+0000 7fe95c600700 1 -- 172.21.15.112:0/849407843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe954103eb0 msgr2=0x7fe95419fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:38.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.746+0000 7fe95c600700 1 --2- 172.21.15.112:0/849407843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe954103eb0 0x7fe95419fd80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe94000e9a0 tx=0x7fe94000ecb0 comp rx=0 tx=0).stop 2024-05-08T05:52:38.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.746+0000 7fe95a600700 1 -- 172.21.15.112:0/849407843 reap_dead start 2024-05-08T05:52:38.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.746+0000 7fe95c600700 1 -- 172.21.15.112:0/849407843 shutdown_connections 2024-05-08T05:52:38.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.746+0000 7fe95c600700 1 -- 172.21.15.112:0/849407843 >> 172.21.15.112:0/849407843 conn(0x7fe95407a650 msgr2=0x7fe954102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:38.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.746+0000 7fe95c600700 1 -- 172.21.15.112:0/849407843 shutdown_connections 2024-05-08T05:52:38.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.747+0000 7fe95c600700 1 -- 172.21.15.112:0/849407843 wait complete. 2024-05-08T05:52:38.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.618+0000 7ff00f800700 1 -- 172.21.15.112:0/506617443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00806ba40 msgr2=0x7ff00806be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:38.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.618+0000 7ff00f800700 1 --2- 172.21.15.112:0/506617443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00806ba40 0x7ff00806be20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feffc009990 tx=0x7feffc009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:38.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.619+0000 7ff00f800700 1 -- 172.21.15.112:0/506617443 shutdown_connections 2024-05-08T05:52:38.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.619+0000 7ff00f800700 1 --2- 172.21.15.112:0/506617443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff00810ff40 0x7ff008112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:38.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.619+0000 7ff00f800700 1 --2- 172.21.15.112:0/506617443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00806ba40 0x7ff00806be20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:38.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.619+0000 7ff00f800700 1 --2- 172.21.15.112:0/506617443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00806c360 0x7ff00810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:38.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.619+0000 7ff00f800700 1 -- 172.21.15.112:0/506617443 >> 172.21.15.112:0/506617443 conn(0x7ff008079310 msgr2=0x7ff008079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.619+0000 7ff00f800700 1 -- 172.21.15.112:0/506617443 shutdown_connections 2024-05-08T05:52:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.619+0000 7ff00f800700 1 -- 172.21.15.112:0/506617443 wait complete. 2024-05-08T05:52:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.620+0000 7ff00f800700 1 Processor -- start 2024-05-08T05:52:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.620+0000 7ff00f800700 1 -- start start 2024-05-08T05:52:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.620+0000 7ff00f800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00806ba40 0x7ff00819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.620+0000 7ff00f800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00806c360 0x7ff00819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.620+0000 7ff00f800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff00810ff40 0x7ff0081a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.620+0000 7ff00f800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0081158b0 con 0x7ff00806ba40 2024-05-08T05:52:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.620+0000 7ff00f800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff008115730 con 0x7ff00806c360 2024-05-08T05:52:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.620+0000 7ff00f800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff008115a30 con 0x7ff00810ff40 2024-05-08T05:52:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.621+0000 7ff007e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00806c360 0x7ff00819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.621+0000 7ff007e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00806c360 0x7ff00819c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:51286/0 (socket says 172.21.15.112:51286) 2024-05-08T05:52:38.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.621+0000 7ff007e00700 1 -- 172.21.15.112:0/1105684948 learned_addr learned my addr 172.21.15.112:0/1105684948 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.621+0000 7ff00ce00700 1 --2- 172.21.15.112:0/1105684948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00806ba40 0x7ff00819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.621+0000 7ff00d800700 1 --2- 172.21.15.112:0/1105684948 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff00810ff40 0x7ff0081a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.621+0000 7ff007e00700 1 -- 172.21.15.112:0/1105684948 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff00810ff40 msgr2=0x7ff0081a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.621+0000 7ff007e00700 1 --2- 172.21.15.112:0/1105684948 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff00810ff40 0x7ff0081a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.621+0000 7ff007e00700 1 -- 172.21.15.112:0/1105684948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00806ba40 msgr2=0x7ff00819be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.621+0000 7ff007e00700 1 --2- 172.21.15.112:0/1105684948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00806ba40 0x7ff00819be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.621+0000 7ff007e00700 1 -- 172.21.15.112:0/1105684948 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0081a0e70 con 0x7ff00806c360 2024-05-08T05:52:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.621+0000 7ff007e00700 1 --2- 172.21.15.112:0/1105684948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00806c360 0x7ff00819c3a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feff400eb30 tx=0x7feff400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.622+0000 7ff005600700 1 -- 172.21.15.112:0/1105684948 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff400cbd0 con 0x7ff00806c360 2024-05-08T05:52:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.622+0000 7ff00f800700 1 -- 172.21.15.112:0/1105684948 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0081a1160 con 0x7ff00806c360 2024-05-08T05:52:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.622+0000 7ff005600700 1 -- 172.21.15.112:0/1105684948 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff40044d0 con 0x7ff00806c360 2024-05-08T05:52:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.622+0000 7ff005600700 1 -- 172.21.15.112:0/1105684948 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4003e30 con 0x7ff00806c360 2024-05-08T05:52:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.622+0000 7ff00f800700 1 -- 172.21.15.112:0/1105684948 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0081a89e0 con 0x7ff00806c360 2024-05-08T05:52:38.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.623+0000 7ff00f800700 1 -- 172.21.15.112:0/1105684948 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff008063bb0 con 0x7ff00806c360 2024-05-08T05:52:38.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.625+0000 7ff005600700 1 -- 172.21.15.112:0/1105684948 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7feff4004000 con 0x7ff00806c360 2024-05-08T05:52:38.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.627+0000 7ff005600700 1 --2- 172.21.15.112:0/1105684948 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feff00b2990 0x7feff00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:38.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.627+0000 7ff005600700 1 -- 172.21.15.112:0/1105684948 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7feff40e2a10 con 0x7ff00806c360 2024-05-08T05:52:38.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.627+0000 7ff00ce00700 1 --2- 172.21.15.112:0/1105684948 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feff00b2990 0x7feff00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:38.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.627+0000 7ff005600700 1 --2- 172.21.15.112:0/1105684948 >> [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186] conn(0x7feff00bc550 0x7feff00be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:38.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.627+0000 7ff005600700 1 -- 172.21.15.112:0/1105684948 --> [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feff00bf0c0 con 0x7feff00bc550 2024-05-08T05:52:38.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.627+0000 7ff005600700 1 -- 172.21.15.112:0/1105684948 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7feff40e2dc0 con 0x7ff00806c360 2024-05-08T05:52:38.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.627+0000 7ff00d800700 1 --2- 172.21.15.112:0/1105684948 >> [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186] conn(0x7feff00bc550 0x7feff00be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:38.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.628+0000 7ff00ce00700 1 --2- 172.21.15.112:0/1105684948 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feff00b2990 0x7feff00b4e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7feffc006010 tx=0x7feffc00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:38.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.628+0000 7ff00d800700 1 --2- 172.21.15.112:0/1105684948 >> [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186] conn(0x7feff00bc550 0x7feff00be9f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:38.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.629+0000 7ff005600700 1 -- 172.21.15.112:0/1105684948 <== osd.6 v2:172.21.15.112:6818/1098984186 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feff00bf0c0 con 0x7feff00bc550 2024-05-08T05:52:38.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.653+0000 7ff00f800700 1 -- 172.21.15.112:0/1105684948 --> [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff008063fb0 con 0x7feff00bc550 2024-05-08T05:52:38.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.653+0000 7ff005600700 1 -- 172.21.15.112:0/1105684948 <== osd.6 v2:172.21.15.112:6818/1098984186 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff008063fb0 con 0x7feff00bc550 2024-05-08T05:52:38.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.654+0000 7ff00f800700 1 -- 172.21.15.112:0/1105684948 >> [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186] conn(0x7feff00bc550 msgr2=0x7feff00be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:38.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.654+0000 7ff00f800700 1 --2- 172.21.15.112:0/1105684948 >> [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186] conn(0x7feff00bc550 0x7feff00be9f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:38.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.654+0000 7ff00f800700 1 -- 172.21.15.112:0/1105684948 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feff00b2990 msgr2=0x7feff00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:38.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.654+0000 7ff00f800700 1 --2- 172.21.15.112:0/1105684948 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feff00b2990 0x7feff00b4e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7feffc006010 tx=0x7feffc00b260 comp rx=0 tx=0).stop 2024-05-08T05:52:38.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.654+0000 7ff00f800700 1 -- 172.21.15.112:0/1105684948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00806c360 msgr2=0x7ff00819c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:38.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.654+0000 7ff00f800700 1 --2- 172.21.15.112:0/1105684948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00806c360 0x7ff00819c3a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feff400eb30 tx=0x7feff400ee40 comp rx=0 tx=0).stop 2024-05-08T05:52:38.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.654+0000 7ff00d800700 1 -- 172.21.15.112:0/1105684948 reap_dead start 2024-05-08T05:52:38.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.654+0000 7ff00f800700 1 -- 172.21.15.112:0/1105684948 shutdown_connections 2024-05-08T05:52:38.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.654+0000 7ff00f800700 1 -- 172.21.15.112:0/1105684948 >> 172.21.15.112:0/1105684948 conn(0x7ff008079310 msgr2=0x7ff008111530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:38.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.654+0000 7ff00f800700 1 -- 172.21.15.112:0/1105684948 shutdown_connections 2024-05-08T05:52:38.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:38.654+0000 7ff00f800700 1 -- 172.21.15.112:0/1105684948 wait complete. 2024-05-08T05:52:39.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:39 smithi150 ceph-mon[34315]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:39.462 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:39 smithi112 ceph-mon[30003]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:39.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:39 smithi162 ceph-mon[34800]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:41.069 INFO:teuthology.orchestra.run.smithi112.stdout:34359738461 2024-05-08T05:52:41.069 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-08T05:52:41.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:41 smithi150 ceph-mon[34315]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:52:41.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:41 smithi112 ceph-mon[30003]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:52:41.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:41 smithi162 ceph-mon[34800]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:52:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:261993005101 2024-05-08T05:52:42.748 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-08T05:52:43.192 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:43 smithi112 ceph-mon[30003]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:43.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:43 smithi150 ceph-mon[34315]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:43.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:43 smithi162 ceph-mon[34800]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.528+0000 7f467f000700 1 -- 172.21.15.112:0/1412395636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4678109ad0 msgr2=0x7f4678109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.528+0000 7f467f000700 1 --2- 172.21.15.112:0/1412395636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4678109ad0 0x7f4678109f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f46600098f0 tx=0x7f4660009c00 comp rx=0 tx=0).stop 2024-05-08T05:52:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.528+0000 7f467f000700 1 -- 172.21.15.112:0/1412395636 shutdown_connections 2024-05-08T05:52:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.528+0000 7f467f000700 1 --2- 172.21.15.112:0/1412395636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f467810a490 0x7f46781169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.528+0000 7f467f000700 1 --2- 172.21.15.112:0/1412395636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f467810fb60 0x7f467810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.528+0000 7f467f000700 1 --2- 172.21.15.112:0/1412395636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4678109ad0 0x7f4678109f50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.528+0000 7f467f000700 1 -- 172.21.15.112:0/1412395636 >> 172.21.15.112:0/1412395636 conn(0x7f4678076c70 msgr2=0x7f4678079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.528+0000 7f467f000700 1 -- 172.21.15.112:0/1412395636 shutdown_connections 2024-05-08T05:52:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.528+0000 7f467f000700 1 -- 172.21.15.112:0/1412395636 wait complete. 2024-05-08T05:52:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.530+0000 7f467f000700 1 Processor -- start 2024-05-08T05:52:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.530+0000 7f467f000700 1 -- start start 2024-05-08T05:52:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.530+0000 7f467f000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4678109ad0 0x7f46781a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.530+0000 7f467f000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f467810a490 0x7f46781a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.530+0000 7f467f000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f467810fb60 0x7f46781a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.530+0000 7f467f000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4678119d60 con 0x7f467810a490 2024-05-08T05:52:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.530+0000 7f467f000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4678119be0 con 0x7f467810fb60 2024-05-08T05:52:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.530+0000 7f467f000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4678119ee0 con 0x7f4678109ad0 2024-05-08T05:52:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.531+0000 7f466f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f467810a490 0x7f46781a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.531+0000 7f4677e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4678109ad0 0x7f46781a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.531+0000 7f467d000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f467810fb60 0x7f46781a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.531+0000 7f4677e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4678109ad0 0x7f46781a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:52638/0 (socket says 172.21.15.112:52638) 2024-05-08T05:52:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.531+0000 7f467d000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f467810fb60 0x7f46781a4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:51318/0 (socket says 172.21.15.112:51318) 2024-05-08T05:52:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.531+0000 7f4677e00700 1 -- 172.21.15.112:0/3926914936 learned_addr learned my addr 172.21.15.112:0/3926914936 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:43.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.531+0000 7f466f400700 1 -- 172.21.15.112:0/3926914936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4678109ad0 msgr2=0x7f46781a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:43.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.531+0000 7f466f400700 1 --2- 172.21.15.112:0/3926914936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4678109ad0 0x7f46781a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.531+0000 7f466f400700 1 -- 172.21.15.112:0/3926914936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f467810fb60 msgr2=0x7f46781a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:43.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.531+0000 7f466f400700 1 --2- 172.21.15.112:0/3926914936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f467810fb60 0x7f46781a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.531+0000 7f466f400700 1 -- 172.21.15.112:0/3926914936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46781a5230 con 0x7f467810a490 2024-05-08T05:52:43.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.531+0000 7f467d000700 1 --2- 172.21.15.112:0/3926914936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f467810fb60 0x7f46781a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:52:43.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.531+0000 7f466f400700 1 --2- 172.21.15.112:0/3926914936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f467810a490 0x7f46781a07b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f466000c010 tx=0x7f466000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:43.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.532+0000 7f4675600700 1 -- 172.21.15.112:0/3926914936 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46600050c0 con 0x7f467810a490 2024-05-08T05:52:43.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.532+0000 7f467f000700 1 -- 172.21.15.112:0/3926914936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46781a54c0 con 0x7f467810a490 2024-05-08T05:52:43.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.532+0000 7f4675600700 1 -- 172.21.15.112:0/3926914936 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f466000b560 con 0x7f467810a490 2024-05-08T05:52:43.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.532+0000 7f4675600700 1 -- 172.21.15.112:0/3926914936 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660003a00 con 0x7f467810a490 2024-05-08T05:52:43.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.532+0000 7f467f000700 1 -- 172.21.15.112:0/3926914936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46781acda0 con 0x7f467810a490 2024-05-08T05:52:43.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.533+0000 7f467f000700 1 -- 172.21.15.112:0/3926914936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4678063c30 con 0x7f467810a490 2024-05-08T05:52:43.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.536+0000 7f4675600700 1 -- 172.21.15.112:0/3926914936 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f466000b710 con 0x7f467810a490 2024-05-08T05:52:43.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.537+0000 7f4675600700 1 --2- 172.21.15.112:0/3926914936 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f465c0b2910 0x7f465c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:43.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.538+0000 7f4675600700 1 -- 172.21.15.112:0/3926914936 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7f46600e3680 con 0x7f467810a490 2024-05-08T05:52:43.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.538+0000 7f4677e00700 1 --2- 172.21.15.112:0/3926914936 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f465c0b2910 0x7f465c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:43.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.538+0000 7f4675600700 1 --2- 172.21.15.112:0/3926914936 >> [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755] conn(0x7f465c0bc4d0 0x7f465c0be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:43.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.538+0000 7f4675600700 1 -- 172.21.15.112:0/3926914936 --> [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f466001fe80 con 0x7f465c0bc4d0 2024-05-08T05:52:43.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.538+0000 7f4675600700 1 -- 172.21.15.112:0/3926914936 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f46600e3a30 con 0x7f467810a490 2024-05-08T05:52:43.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.538+0000 7f467d000700 1 --2- 172.21.15.112:0/3926914936 >> [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755] conn(0x7f465c0bc4d0 0x7f465c0be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:43.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.538+0000 7f4677e00700 1 --2- 172.21.15.112:0/3926914936 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f465c0b2910 0x7f465c0b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4668009c10 tx=0x7f466800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:43.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.539+0000 7f467d000700 1 --2- 172.21.15.112:0/3926914936 >> [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755] conn(0x7f465c0bc4d0 0x7f465c0be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:43.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.540+0000 7f4675600700 1 -- 172.21.15.112:0/3926914936 <== osd.3 v2:172.21.15.112:6810/3325718755 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f466001fe80 con 0x7f465c0bc4d0 2024-05-08T05:52:43.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.558+0000 7f467f000700 1 -- 172.21.15.112:0/3926914936 --> [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4678064030 con 0x7f465c0bc4d0 2024-05-08T05:52:43.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.558+0000 7f4675600700 1 -- 172.21.15.112:0/3926914936 <== osd.3 v2:172.21.15.112:6810/3325718755 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4678064030 con 0x7f465c0bc4d0 2024-05-08T05:52:43.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.559+0000 7f467f000700 1 -- 172.21.15.112:0/3926914936 >> [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755] conn(0x7f465c0bc4d0 msgr2=0x7f465c0be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:43.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.559+0000 7f467f000700 1 --2- 172.21.15.112:0/3926914936 >> [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755] conn(0x7f465c0bc4d0 0x7f465c0be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.559+0000 7f467f000700 1 -- 172.21.15.112:0/3926914936 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f465c0b2910 msgr2=0x7f465c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:43.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.559+0000 7f467f000700 1 --2- 172.21.15.112:0/3926914936 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f465c0b2910 0x7f465c0b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4668009c10 tx=0x7f466800b040 comp rx=0 tx=0).stop 2024-05-08T05:52:43.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.559+0000 7f467f000700 1 -- 172.21.15.112:0/3926914936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f467810a490 msgr2=0x7f46781a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:43.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.559+0000 7f467f000700 1 --2- 172.21.15.112:0/3926914936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f467810a490 0x7f46781a07b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f466000c010 tx=0x7f466000fa00 comp rx=0 tx=0).stop 2024-05-08T05:52:43.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.559+0000 7f467d000700 1 -- 172.21.15.112:0/3926914936 reap_dead start 2024-05-08T05:52:43.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.559+0000 7f467f000700 1 -- 172.21.15.112:0/3926914936 shutdown_connections 2024-05-08T05:52:43.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.559+0000 7f467f000700 1 -- 172.21.15.112:0/3926914936 >> 172.21.15.112:0/3926914936 conn(0x7f4678076c70 msgr2=0x7f46780774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:43.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.559+0000 7f467f000700 1 -- 172.21.15.112:0/3926914936 shutdown_connections 2024-05-08T05:52:43.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:43.560+0000 7f467f000700 1 -- 172.21.15.112:0/3926914936 wait complete. 2024-05-08T05:52:44.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.205+0000 7fef4f200700 1 -- 172.21.15.112:0/1368348439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef5010e8b0 msgr2=0x7fef5010ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:44.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.205+0000 7fef4f200700 1 --2- 172.21.15.112:0/1368348439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef5010e8b0 0x7fef5010ec90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fef40009990 tx=0x7fef40009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:44.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.206+0000 7fef4f200700 1 -- 172.21.15.112:0/1368348439 shutdown_connections 2024-05-08T05:52:44.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.206+0000 7fef4f200700 1 --2- 172.21.15.112:0/1368348439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef5010a3a0 0x7fef50116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:44.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.206+0000 7fef4f200700 1 --2- 172.21.15.112:0/1368348439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef5010e8b0 0x7fef5010ec90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:44.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.206+0000 7fef4f200700 1 --2- 172.21.15.112:0/1368348439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef501099e0 0x7fef50109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:44.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.206+0000 7fef4f200700 1 -- 172.21.15.112:0/1368348439 >> 172.21.15.112:0/1368348439 conn(0x7fef50079230 msgr2=0x7fef501082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:44.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.206+0000 7fef4f200700 1 -- 172.21.15.112:0/1368348439 shutdown_connections 2024-05-08T05:52:44.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.206+0000 7fef4f200700 1 -- 172.21.15.112:0/1368348439 wait complete. 2024-05-08T05:52:44.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.207+0000 7fef4f200700 1 Processor -- start 2024-05-08T05:52:44.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.207+0000 7fef4f200700 1 -- start start 2024-05-08T05:52:44.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.207+0000 7fef4f200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef501099e0 0x7fef501a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:44.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.207+0000 7fef4f200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef5010a3a0 0x7fef501a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:44.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.207+0000 7fef4f200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef5010e8b0 0x7fef501a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.207+0000 7fef4f200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef50119bd0 con 0x7fef501099e0 2024-05-08T05:52:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.207+0000 7fef4f200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef50119a50 con 0x7fef5010a3a0 2024-05-08T05:52:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.207+0000 7fef4f200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fef50119d50 con 0x7fef5010e8b0 2024-05-08T05:52:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.208+0000 7fef4de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef501099e0 0x7fef501a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.208+0000 7fef4de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef501099e0 0x7fef501a0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59726/0 (socket says 172.21.15.112:59726) 2024-05-08T05:52:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.208+0000 7fef4de00700 1 -- 172.21.15.112:0/3895550859 learned_addr learned my addr 172.21.15.112:0/3895550859 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.208+0000 7fef4d400700 1 --2- 172.21.15.112:0/3895550859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef5010a3a0 0x7fef501a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.208+0000 7fef4e800700 1 --2- 172.21.15.112:0/3895550859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef5010e8b0 0x7fef501a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.208+0000 7fef4de00700 1 -- 172.21.15.112:0/3895550859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef5010e8b0 msgr2=0x7fef501a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.208+0000 7fef4de00700 1 --2- 172.21.15.112:0/3895550859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef5010e8b0 0x7fef501a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.208+0000 7fef4de00700 1 -- 172.21.15.112:0/3895550859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef5010a3a0 msgr2=0x7fef501a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.208+0000 7fef4de00700 1 --2- 172.21.15.112:0/3895550859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef5010a3a0 0x7fef501a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.208+0000 7fef4de00700 1 -- 172.21.15.112:0/3895550859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef501a5110 con 0x7fef501099e0 2024-05-08T05:52:44.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.208+0000 7fef4d400700 1 --2- 172.21.15.112:0/3895550859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef5010a3a0 0x7fef501a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:52:44.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.208+0000 7fef4de00700 1 --2- 172.21.15.112:0/3895550859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef501099e0 0x7fef501a0100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fef40000c00 tx=0x7fef400057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:44.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.209+0000 7fef46a00700 1 -- 172.21.15.112:0/3895550859 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40027070 con 0x7fef501099e0 2024-05-08T05:52:44.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.209+0000 7fef46a00700 1 -- 172.21.15.112:0/3895550859 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef40022470 con 0x7fef501099e0 2024-05-08T05:52:44.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.209+0000 7fef4f200700 1 -- 172.21.15.112:0/3895550859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef501a53a0 con 0x7fef501099e0 2024-05-08T05:52:44.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.209+0000 7fef46a00700 1 -- 172.21.15.112:0/3895550859 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40017650 con 0x7fef501099e0 2024-05-08T05:52:44.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.209+0000 7fef4f200700 1 -- 172.21.15.112:0/3895550859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef501acc80 con 0x7fef501099e0 2024-05-08T05:52:44.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.210+0000 7fef4f200700 1 -- 172.21.15.112:0/3895550859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef50063bf0 con 0x7fef501099e0 2024-05-08T05:52:44.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.212+0000 7fef46a00700 1 -- 172.21.15.112:0/3895550859 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fef4002a070 con 0x7fef501099e0 2024-05-08T05:52:44.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.214+0000 7fef46a00700 1 --2- 172.21.15.112:0/3895550859 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fef340b2990 0x7fef340b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:44.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.214+0000 7fef46a00700 1 -- 172.21.15.112:0/3895550859 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7fef400e3630 con 0x7fef501099e0 2024-05-08T05:52:44.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.214+0000 7fef4d400700 1 --2- 172.21.15.112:0/3895550859 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fef340b2990 0x7fef340b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:44.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.214+0000 7fef46a00700 1 --2- 172.21.15.112:0/3895550859 >> [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331] conn(0x7fef340bc420 0x7fef340be8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:44.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.214+0000 7fef46a00700 1 -- 172.21.15.112:0/3895550859 --> [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef40007e80 con 0x7fef340bc420 2024-05-08T05:52:44.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.214+0000 7fef46a00700 1 -- 172.21.15.112:0/3895550859 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fef400e39e0 con 0x7fef501099e0 2024-05-08T05:52:44.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.215+0000 7fef4e800700 1 --2- 172.21.15.112:0/3895550859 >> [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331] conn(0x7fef340bc420 0x7fef340be8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:44.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.215+0000 7fef4d400700 1 --2- 172.21.15.112:0/3895550859 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fef340b2990 0x7fef340b4e50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009680 tx=0x7fef3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:44.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.215+0000 7fef4e800700 1 --2- 172.21.15.112:0/3895550859 >> [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331] conn(0x7fef340bc420 0x7fef340be8c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:44.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.216+0000 7fef46a00700 1 -- 172.21.15.112:0/3895550859 <== osd.11 v2:172.21.15.162:6824/3702614331 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef40007e80 con 0x7fef340bc420 2024-05-08T05:52:44.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.234+0000 7fef4f200700 1 -- 172.21.15.112:0/3895550859 --> [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fef50064020 con 0x7fef340bc420 2024-05-08T05:52:44.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.234+0000 7fef46a00700 1 -- 172.21.15.112:0/3895550859 <== osd.11 v2:172.21.15.162:6824/3702614331 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fef50064020 con 0x7fef340bc420 2024-05-08T05:52:44.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.235+0000 7fef4f200700 1 -- 172.21.15.112:0/3895550859 >> [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331] conn(0x7fef340bc420 msgr2=0x7fef340be8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:44.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.235+0000 7fef4f200700 1 --2- 172.21.15.112:0/3895550859 >> [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331] conn(0x7fef340bc420 0x7fef340be8c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:44.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.235+0000 7fef4f200700 1 -- 172.21.15.112:0/3895550859 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fef340b2990 msgr2=0x7fef340b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:44.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.235+0000 7fef4f200700 1 --2- 172.21.15.112:0/3895550859 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fef340b2990 0x7fef340b4e50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009680 tx=0x7fef3c009260 comp rx=0 tx=0).stop 2024-05-08T05:52:44.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.235+0000 7fef4f200700 1 -- 172.21.15.112:0/3895550859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef501099e0 msgr2=0x7fef501a0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:44.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.235+0000 7fef4f200700 1 --2- 172.21.15.112:0/3895550859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef501099e0 0x7fef501a0100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fef40000c00 tx=0x7fef400057c0 comp rx=0 tx=0).stop 2024-05-08T05:52:44.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.235+0000 7fef4e800700 1 -- 172.21.15.112:0/3895550859 reap_dead start 2024-05-08T05:52:44.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.235+0000 7fef4f200700 1 -- 172.21.15.112:0/3895550859 shutdown_connections 2024-05-08T05:52:44.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.235+0000 7fef4f200700 1 -- 172.21.15.112:0/3895550859 >> 172.21.15.112:0/3895550859 conn(0x7fef50079230 msgr2=0x7fef500798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:44.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.236+0000 7fef4f200700 1 -- 172.21.15.112:0/3895550859 shutdown_connections 2024-05-08T05:52:44.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:44.236+0000 7fef4f200700 1 -- 172.21.15.112:0/3895550859 wait complete. 2024-05-08T05:52:44.869 INFO:teuthology.orchestra.run.smithi112.stdout:158913790022 2024-05-08T05:52:44.869 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-08T05:52:45.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.087+0000 7f8a41c00700 1 -- 172.21.15.112:0/290150916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a3c075ff0 msgr2=0x7f8a3c10f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.087+0000 7f8a41c00700 1 --2- 172.21.15.112:0/290150916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a3c075ff0 0x7f8a3c10f960 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8a240098f0 tx=0x7f8a24009c00 comp rx=0 tx=0).stop 2024-05-08T05:52:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.087+0000 7f8a41c00700 1 -- 172.21.15.112:0/290150916 shutdown_connections 2024-05-08T05:52:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.087+0000 7f8a41c00700 1 --2- 172.21.15.112:0/290150916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a3c10fea0 0x7f8a3c112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.087+0000 7f8a41c00700 1 --2- 172.21.15.112:0/290150916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c0756d0 0x7f8a3c075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.087+0000 7f8a41c00700 1 --2- 172.21.15.112:0/290150916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a3c075ff0 0x7f8a3c10f960 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.087+0000 7f8a41c00700 1 -- 172.21.15.112:0/290150916 >> 172.21.15.112:0/290150916 conn(0x7f8a3c07a600 msgr2=0x7f8a3c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:45.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.087+0000 7f8a41c00700 1 -- 172.21.15.112:0/290150916 shutdown_connections 2024-05-08T05:52:45.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.087+0000 7f8a41c00700 1 -- 172.21.15.112:0/290150916 wait complete. 2024-05-08T05:52:45.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.088+0000 7f8a41c00700 1 Processor -- start 2024-05-08T05:52:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.089+0000 7f8a41c00700 1 -- start start 2024-05-08T05:52:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.089+0000 7f8a41c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a3c0756d0 0x7f8a3c19be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.089+0000 7f8a41c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c075ff0 0x7f8a3c19c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.089+0000 7f8a41c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a3c10fea0 0x7f8a3c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.089+0000 7f8a41c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1158d0 con 0x7f8a3c075ff0 2024-05-08T05:52:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.089+0000 7f8a41c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c115750 con 0x7f8a3c10fea0 2024-05-08T05:52:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.089+0000 7f8a41c00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c115a50 con 0x7f8a3c0756d0 2024-05-08T05:52:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.089+0000 7f8a33e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c075ff0 0x7f8a3c19c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.089+0000 7f8a3aa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a3c0756d0 0x7f8a3c19be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.089+0000 7f8a3aa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a3c0756d0 0x7f8a3c19be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50630/0 (socket says 172.21.15.112:50630) 2024-05-08T05:52:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.089+0000 7f8a3aa00700 1 -- 172.21.15.112:0/1746571644 learned_addr learned my addr 172.21.15.112:0/1746571644 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.090+0000 7f8a3b400700 1 --2- 172.21.15.112:0/1746571644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a3c10fea0 0x7f8a3c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.090+0000 7f8a33e00700 1 -- 172.21.15.112:0/1746571644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a3c0756d0 msgr2=0x7f8a3c19be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.090+0000 7f8a33e00700 1 --2- 172.21.15.112:0/1746571644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a3c0756d0 0x7f8a3c19be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.090+0000 7f8a33e00700 1 -- 172.21.15.112:0/1746571644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a3c10fea0 msgr2=0x7f8a3c1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.090+0000 7f8a33e00700 1 --2- 172.21.15.112:0/1746571644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a3c10fea0 0x7f8a3c1a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.090+0000 7f8a33e00700 1 -- 172.21.15.112:0/1746571644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c1a0e90 con 0x7f8a3c075ff0 2024-05-08T05:52:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.090+0000 7f8a3b400700 1 --2- 172.21.15.112:0/1746571644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a3c10fea0 0x7f8a3c1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:52:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.090+0000 7f8a33e00700 1 --2- 172.21.15.112:0/1746571644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c075ff0 0x7f8a3c19c3c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400c010 tx=0x7f8a2400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.091+0000 7f8a33400700 1 -- 172.21.15.112:0/1746571644 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a240051c0 con 0x7f8a3c075ff0 2024-05-08T05:52:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.091+0000 7f8a41c00700 1 -- 172.21.15.112:0/1746571644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3c1a1120 con 0x7f8a3c075ff0 2024-05-08T05:52:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.091+0000 7f8a33400700 1 -- 172.21.15.112:0/1746571644 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a2400b560 con 0x7f8a3c075ff0 2024-05-08T05:52:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.091+0000 7f8a33400700 1 -- 172.21.15.112:0/1746571644 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a240039e0 con 0x7f8a3c075ff0 2024-05-08T05:52:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.091+0000 7f8a41c00700 1 -- 172.21.15.112:0/1746571644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3c1a8a00 con 0x7f8a3c075ff0 2024-05-08T05:52:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.092+0000 7f8a41c00700 1 -- 172.21.15.112:0/1746571644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8a3c063bb0 con 0x7f8a3c075ff0 2024-05-08T05:52:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.095+0000 7f8a33400700 1 -- 172.21.15.112:0/1746571644 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8a2400b930 con 0x7f8a3c075ff0 2024-05-08T05:52:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.096+0000 7f8a33400700 1 --2- 172.21.15.112:0/1746571644 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8a200b2910 0x7f8a200b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.096+0000 7f8a33400700 1 -- 172.21.15.112:0/1746571644 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7f8a240e3640 con 0x7f8a3c075ff0 2024-05-08T05:52:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.096+0000 7f8a3aa00700 1 --2- 172.21.15.112:0/1746571644 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8a200b2910 0x7f8a200b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.097+0000 7f8a33400700 1 --2- 172.21.15.112:0/1746571644 >> [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683] conn(0x7f8a200bc4d0 0x7f8a200be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.097+0000 7f8a33400700 1 -- 172.21.15.112:0/1746571644 --> [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a2402fe50 con 0x7f8a200bc4d0 2024-05-08T05:52:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.097+0000 7f8a33400700 1 -- 172.21.15.112:0/1746571644 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f8a240e39f0 con 0x7f8a3c075ff0 2024-05-08T05:52:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.097+0000 7f8a3b400700 1 --2- 172.21.15.112:0/1746571644 >> [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683] conn(0x7f8a200bc4d0 0x7f8a200be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.097+0000 7f8a3aa00700 1 --2- 172.21.15.112:0/1746571644 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8a200b2910 0x7f8a200b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c005950 tx=0x7f8a2c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.098+0000 7f8a3b400700 1 --2- 172.21.15.112:0/1746571644 >> [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683] conn(0x7f8a200bc4d0 0x7f8a200be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.099+0000 7f8a33400700 1 -- 172.21.15.112:0/1746571644 <== osd.4 v2:172.21.15.150:6808/1548914683 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8a2402fe50 con 0x7f8a200bc4d0 2024-05-08T05:52:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.116+0000 7f8a41c00700 1 -- 172.21.15.112:0/1746571644 --> [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8a3c064030 con 0x7f8a200bc4d0 2024-05-08T05:52:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.117+0000 7f8a33400700 1 -- 172.21.15.112:0/1746571644 <== osd.4 v2:172.21.15.150:6808/1548914683 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8a3c064030 con 0x7f8a200bc4d0 2024-05-08T05:52:45.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.117+0000 7f8a41c00700 1 -- 172.21.15.112:0/1746571644 >> [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683] conn(0x7f8a200bc4d0 msgr2=0x7f8a200be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:45.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.117+0000 7f8a41c00700 1 --2- 172.21.15.112:0/1746571644 >> [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683] conn(0x7f8a200bc4d0 0x7f8a200be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:45.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.118+0000 7f8a41c00700 1 -- 172.21.15.112:0/1746571644 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8a200b2910 msgr2=0x7f8a200b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:45.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.118+0000 7f8a41c00700 1 --2- 172.21.15.112:0/1746571644 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8a200b2910 0x7f8a200b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c005950 tx=0x7f8a2c009310 comp rx=0 tx=0).stop 2024-05-08T05:52:45.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.118+0000 7f8a41c00700 1 -- 172.21.15.112:0/1746571644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c075ff0 msgr2=0x7f8a3c19c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:45.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.118+0000 7f8a41c00700 1 --2- 172.21.15.112:0/1746571644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c075ff0 0x7f8a3c19c3c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400c010 tx=0x7f8a2400fa00 comp rx=0 tx=0).stop 2024-05-08T05:52:45.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.118+0000 7f8a3b400700 1 -- 172.21.15.112:0/1746571644 reap_dead start 2024-05-08T05:52:45.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.118+0000 7f8a41c00700 1 -- 172.21.15.112:0/1746571644 shutdown_connections 2024-05-08T05:52:45.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.118+0000 7f8a41c00700 1 -- 172.21.15.112:0/1746571644 >> 172.21.15.112:0/1746571644 conn(0x7f8a3c07a600 msgr2=0x7f8a3c101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:45.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.118+0000 7f8a41c00700 1 -- 172.21.15.112:0/1746571644 shutdown_connections 2024-05-08T05:52:45.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.118+0000 7f8a41c00700 1 -- 172.21.15.112:0/1746571644 wait complete. 2024-05-08T05:52:45.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:45 smithi112 ceph-mon[30003]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:52:45.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:45 smithi150 ceph-mon[34315]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:52:45.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:45 smithi162 ceph-mon[34800]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:52:46.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.381+0000 7fb40ac00700 1 -- 172.21.15.112:0/1651974501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb404075ff0 msgr2=0x7fb40410f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:46.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.381+0000 7fb40ac00700 1 --2- 172.21.15.112:0/1651974501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb404075ff0 0x7fb40410f960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009990 tx=0x7fb3f0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.381+0000 7fb40ac00700 1 -- 172.21.15.112:0/1651974501 shutdown_connections 2024-05-08T05:52:46.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.381+0000 7fb40ac00700 1 --2- 172.21.15.112:0/1651974501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb40410fea0 0x7fb404112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.381+0000 7fb40ac00700 1 --2- 172.21.15.112:0/1651974501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4040756d0 0x7fb404075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.381+0000 7fb40ac00700 1 --2- 172.21.15.112:0/1651974501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb404075ff0 0x7fb40410f960 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.381+0000 7fb40ac00700 1 -- 172.21.15.112:0/1651974501 >> 172.21.15.112:0/1651974501 conn(0x7fb40407a600 msgr2=0x7fb40407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.382+0000 7fb40ac00700 1 -- 172.21.15.112:0/1651974501 shutdown_connections 2024-05-08T05:52:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.382+0000 7fb40ac00700 1 -- 172.21.15.112:0/1651974501 wait complete. 2024-05-08T05:52:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.383+0000 7fb40ac00700 1 Processor -- start 2024-05-08T05:52:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.383+0000 7fb40ac00700 1 -- start start 2024-05-08T05:52:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.383+0000 7fb40ac00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4040756d0 0x7fb40419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.383+0000 7fb40ac00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb404075ff0 0x7fb40419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.383+0000 7fb40ac00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40410fea0 0x7fb4041a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.383+0000 7fb40ac00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041158b0 con 0x7fb404075ff0 2024-05-08T05:52:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.383+0000 7fb40ac00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb404115730 con 0x7fb40410fea0 2024-05-08T05:52:46.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.383+0000 7fb40ac00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb404115a30 con 0x7fb4040756d0 2024-05-08T05:52:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.384+0000 7fb403e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4040756d0 0x7fb40419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.384+0000 7fb403e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4040756d0 0x7fb40419bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50688/0 (socket says 172.21.15.112:50688) 2024-05-08T05:52:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.384+0000 7fb403e00700 1 -- 172.21.15.112:0/3237567243 learned_addr learned my addr 172.21.15.112:0/3237567243 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.384+0000 7fb403400700 1 --2- 172.21.15.112:0/3237567243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb404075ff0 0x7fb40419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.384+0000 7fb408c00700 1 --2- 172.21.15.112:0/3237567243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40410fea0 0x7fb4041a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.384+0000 7fb403e00700 1 -- 172.21.15.112:0/3237567243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40410fea0 msgr2=0x7fb4041a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.384+0000 7fb403e00700 1 --2- 172.21.15.112:0/3237567243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40410fea0 0x7fb4041a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.384+0000 7fb403e00700 1 -- 172.21.15.112:0/3237567243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb404075ff0 msgr2=0x7fb40419c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.384+0000 7fb403e00700 1 --2- 172.21.15.112:0/3237567243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb404075ff0 0x7fb40419c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.384+0000 7fb403e00700 1 -- 172.21.15.112:0/3237567243 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4041a0d90 con 0x7fb4040756d0 2024-05-08T05:52:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.384+0000 7fb408c00700 1 --2- 172.21.15.112:0/3237567243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40410fea0 0x7fb4041a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:52:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.384+0000 7fb403e00700 1 --2- 172.21.15.112:0/3237567243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4040756d0 0x7fb40419bd80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400e9a0 tx=0x7fb3f400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.385+0000 7fb400c00700 1 -- 172.21.15.112:0/3237567243 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f400cbd0 con 0x7fb4040756d0 2024-05-08T05:52:46.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.385+0000 7fb400c00700 1 -- 172.21.15.112:0/3237567243 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f400cd70 con 0x7fb4040756d0 2024-05-08T05:52:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.385+0000 7fb40ac00700 1 -- 172.21.15.112:0/3237567243 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4041a1080 con 0x7fb4040756d0 2024-05-08T05:52:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.385+0000 7fb400c00700 1 -- 172.21.15.112:0/3237567243 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4018680 con 0x7fb4040756d0 2024-05-08T05:52:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.385+0000 7fb40ac00700 1 -- 172.21.15.112:0/3237567243 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4041a8900 con 0x7fb4040756d0 2024-05-08T05:52:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.386+0000 7fb40ac00700 1 -- 172.21.15.112:0/3237567243 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb404063c30 con 0x7fb4040756d0 2024-05-08T05:52:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.388+0000 7fb400c00700 1 -- 172.21.15.112:0/3237567243 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb3f4010430 con 0x7fb4040756d0 2024-05-08T05:52:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.389+0000 7fb400c00700 1 --2- 172.21.15.112:0/3237567243 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb3e80b2990 0x7fb3e80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.390+0000 7fb400c00700 1 -- 172.21.15.112:0/3237567243 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7fb3f40e25c0 con 0x7fb4040756d0 2024-05-08T05:52:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.390+0000 7fb403400700 1 --2- 172.21.15.112:0/3237567243 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb3e80b2990 0x7fb3e80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.390+0000 7fb400c00700 1 --2- 172.21.15.112:0/3237567243 >> [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215] conn(0x7fb3e80bc550 0x7fb3e80be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.390+0000 7fb400c00700 1 -- 172.21.15.112:0/3237567243 --> [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb3e80bf0c0 con 0x7fb3e80bc550 2024-05-08T05:52:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.390+0000 7fb400c00700 1 -- 172.21.15.112:0/3237567243 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fb3f40e9050 con 0x7fb4040756d0 2024-05-08T05:52:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.390+0000 7fb403400700 1 --2- 172.21.15.112:0/3237567243 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb3e80b2990 0x7fb3e80b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb404102fd0 tx=0x7fb3f000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:46.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.390+0000 7fb408c00700 1 --2- 172.21.15.112:0/3237567243 >> [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215] conn(0x7fb3e80bc550 0x7fb3e80be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.391+0000 7fb408c00700 1 --2- 172.21.15.112:0/3237567243 >> [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215] conn(0x7fb3e80bc550 0x7fb3e80be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.392+0000 7fb400c00700 1 -- 172.21.15.112:0/3237567243 <== osd.2 v2:172.21.15.162:6800/1556203215 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3e80bf0c0 con 0x7fb3e80bc550 2024-05-08T05:52:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.410+0000 7fb40ac00700 1 -- 172.21.15.112:0/3237567243 --> [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb404064060 con 0x7fb3e80bc550 2024-05-08T05:52:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.410+0000 7fb400c00700 1 -- 172.21.15.112:0/3237567243 <== osd.2 v2:172.21.15.162:6800/1556203215 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb404064060 con 0x7fb3e80bc550 2024-05-08T05:52:46.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.411+0000 7fb40ac00700 1 -- 172.21.15.112:0/3237567243 >> [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215] conn(0x7fb3e80bc550 msgr2=0x7fb3e80be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:46.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.411+0000 7fb40ac00700 1 --2- 172.21.15.112:0/3237567243 >> [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215] conn(0x7fb3e80bc550 0x7fb3e80be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.411+0000 7fb40ac00700 1 -- 172.21.15.112:0/3237567243 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb3e80b2990 msgr2=0x7fb3e80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:46.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.411+0000 7fb40ac00700 1 --2- 172.21.15.112:0/3237567243 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb3e80b2990 0x7fb3e80b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb404102fd0 tx=0x7fb3f000b260 comp rx=0 tx=0).stop 2024-05-08T05:52:46.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.411+0000 7fb40ac00700 1 -- 172.21.15.112:0/3237567243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4040756d0 msgr2=0x7fb40419bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:46.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.411+0000 7fb40ac00700 1 --2- 172.21.15.112:0/3237567243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4040756d0 0x7fb40419bd80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400e9a0 tx=0x7fb3f400ecb0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.411+0000 7fb408c00700 1 -- 172.21.15.112:0/3237567243 reap_dead start 2024-05-08T05:52:46.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.411+0000 7fb40ac00700 1 -- 172.21.15.112:0/3237567243 shutdown_connections 2024-05-08T05:52:46.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.411+0000 7fb40ac00700 1 -- 172.21.15.112:0/3237567243 >> 172.21.15.112:0/3237567243 conn(0x7fb40407a600 msgr2=0x7fb4041006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:46.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.411+0000 7fb40ac00700 1 -- 172.21.15.112:0/3237567243 shutdown_connections 2024-05-08T05:52:46.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:46.411+0000 7fb40ac00700 1 -- 172.21.15.112:0/3237567243 wait complete. 2024-05-08T05:52:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.895+0000 7f2edea00700 1 -- 172.21.15.112:0/220370114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ed8109600 msgr2=0x7f2ed81099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.895+0000 7f2edea00700 1 --2- 172.21.15.112:0/220370114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ed8109600 0x7f2ed81099e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009990 tx=0x7f2ec8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.895+0000 7f2edea00700 1 -- 172.21.15.112:0/220370114 shutdown_connections 2024-05-08T05:52:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.895+0000 7f2edea00700 1 --2- 172.21.15.112:0/220370114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ed8103eb0 0x7f2ed8110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.895+0000 7f2edea00700 1 --2- 172.21.15.112:0/220370114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ed8109600 0x7f2ed81099e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.895+0000 7f2edea00700 1 --2- 172.21.15.112:0/220370114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed81034f0 0x7f2ed8103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.895+0000 7f2edea00700 1 -- 172.21.15.112:0/220370114 >> 172.21.15.112:0/220370114 conn(0x7f2ed807a650 msgr2=0x7f2ed807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.896+0000 7f2edea00700 1 -- 172.21.15.112:0/220370114 shutdown_connections 2024-05-08T05:52:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.896+0000 7f2edea00700 1 -- 172.21.15.112:0/220370114 wait complete. 2024-05-08T05:52:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.897+0000 7f2edea00700 1 Processor -- start 2024-05-08T05:52:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.897+0000 7f2edea00700 1 -- start start 2024-05-08T05:52:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.897+0000 7f2edea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed81034f0 0x7f2ed819bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.897+0000 7f2edea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ed8103eb0 0x7f2ed819c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.897+0000 7f2edea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ed8109600 0x7f2ed81a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.897+0000 7f2edea00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81136c0 con 0x7f2ed8109600 2024-05-08T05:52:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.897+0000 7f2edea00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8113540 con 0x7f2ed8103eb0 2024-05-08T05:52:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.897+0000 7f2edea00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8113840 con 0x7f2ed81034f0 2024-05-08T05:52:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.898+0000 7f2ed7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed81034f0 0x7f2ed819bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.898+0000 7f2ed7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed81034f0 0x7f2ed819bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50662/0 (socket says 172.21.15.112:50662) 2024-05-08T05:52:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.898+0000 7f2ed7e00700 1 -- 172.21.15.112:0/1410996349 learned_addr learned my addr 172.21.15.112:0/1410996349 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.898+0000 7f2edca00700 1 --2- 172.21.15.112:0/1410996349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ed8109600 0x7f2ed81a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.898+0000 7f2ed7400700 1 --2- 172.21.15.112:0/1410996349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ed8103eb0 0x7f2ed819c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.898+0000 7f2ed7e00700 1 -- 172.21.15.112:0/1410996349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ed8103eb0 msgr2=0x7f2ed819c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.898+0000 7f2ed7e00700 1 --2- 172.21.15.112:0/1410996349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ed8103eb0 0x7f2ed819c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.898+0000 7f2ed7e00700 1 -- 172.21.15.112:0/1410996349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ed8109600 msgr2=0x7f2ed81a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.898+0000 7f2ed7e00700 1 --2- 172.21.15.112:0/1410996349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ed8109600 0x7f2ed81a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.898+0000 7f2ed7e00700 1 -- 172.21.15.112:0/1410996349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed81a0d70 con 0x7f2ed81034f0 2024-05-08T05:52:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.898+0000 7f2ed7400700 1 --2- 172.21.15.112:0/1410996349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ed8103eb0 0x7f2ed819c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:52:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.898+0000 7f2ed7e00700 1 --2- 172.21.15.112:0/1410996349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed81034f0 0x7f2ed819bd60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8000c00 tx=0x7f2ec80057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.899+0000 7f2ed4c00700 1 -- 172.21.15.112:0/1410996349 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8027070 con 0x7f2ed81034f0 2024-05-08T05:52:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.899+0000 7f2edea00700 1 -- 172.21.15.112:0/1410996349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed81a1000 con 0x7f2ed81034f0 2024-05-08T05:52:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.899+0000 7f2ed4c00700 1 -- 172.21.15.112:0/1410996349 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec8022470 con 0x7f2ed81034f0 2024-05-08T05:52:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.899+0000 7f2ed4c00700 1 -- 172.21.15.112:0/1410996349 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8017650 con 0x7f2ed81034f0 2024-05-08T05:52:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.899+0000 7f2edea00700 1 -- 172.21.15.112:0/1410996349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed81a88e0 con 0x7f2ed81034f0 2024-05-08T05:52:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.900+0000 7f2edea00700 1 -- 172.21.15.112:0/1410996349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2ed81103d0 con 0x7f2ed81034f0 2024-05-08T05:52:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.902+0000 7f2ed4c00700 1 -- 172.21.15.112:0/1410996349 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f2ec802a070 con 0x7f2ed81034f0 2024-05-08T05:52:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.904+0000 7f2ed4c00700 1 --2- 172.21.15.112:0/1410996349 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2ebc0b2910 0x7f2ebc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.904+0000 7f2ed4c00700 1 -- 172.21.15.112:0/1410996349 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7f2ec80e3370 con 0x7f2ed81034f0 2024-05-08T05:52:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.904+0000 7f2ed7400700 1 --2- 172.21.15.112:0/1410996349 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2ebc0b2910 0x7f2ebc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.904+0000 7f2ed4c00700 1 --2- 172.21.15.112:0/1410996349 >> [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474] conn(0x7f2ebc0bc4d0 0x7f2ebc0be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.904+0000 7f2ed4c00700 1 -- 172.21.15.112:0/1410996349 --> [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ec8018e50 con 0x7f2ebc0bc4d0 2024-05-08T05:52:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.904+0000 7f2ed4c00700 1 -- 172.21.15.112:0/1410996349 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f2ec80e3720 con 0x7f2ed81034f0 2024-05-08T05:52:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.905+0000 7f2edca00700 1 --2- 172.21.15.112:0/1410996349 >> [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474] conn(0x7f2ebc0bc4d0 0x7f2ebc0be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.905+0000 7f2ed7400700 1 --2- 172.21.15.112:0/1410996349 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2ebc0b2910 0x7f2ebc0b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8101af0 tx=0x7f2ec4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.905+0000 7f2edca00700 1 --2- 172.21.15.112:0/1410996349 >> [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474] conn(0x7f2ebc0bc4d0 0x7f2ebc0be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.906+0000 7f2ed4c00700 1 -- 172.21.15.112:0/1410996349 <== osd.10 v2:172.21.15.150:6824/2838995474 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ec8018e50 con 0x7f2ebc0bc4d0 2024-05-08T05:52:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.925+0000 7f2edea00700 1 -- 172.21.15.112:0/1410996349 --> [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2ed8104c10 con 0x7f2ebc0bc4d0 2024-05-08T05:52:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.926+0000 7f2ed4c00700 1 -- 172.21.15.112:0/1410996349 <== osd.10 v2:172.21.15.150:6824/2838995474 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2ed8104c10 con 0x7f2ebc0bc4d0 2024-05-08T05:52:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.926+0000 7f2edea00700 1 -- 172.21.15.112:0/1410996349 >> [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474] conn(0x7f2ebc0bc4d0 msgr2=0x7f2ebc0be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.926+0000 7f2edea00700 1 --2- 172.21.15.112:0/1410996349 >> [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474] conn(0x7f2ebc0bc4d0 0x7f2ebc0be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.926+0000 7f2edea00700 1 -- 172.21.15.112:0/1410996349 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2ebc0b2910 msgr2=0x7f2ebc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.926+0000 7f2edea00700 1 --2- 172.21.15.112:0/1410996349 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2ebc0b2910 0x7f2ebc0b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8101af0 tx=0x7f2ec4009260 comp rx=0 tx=0).stop 2024-05-08T05:52:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.926+0000 7f2edea00700 1 -- 172.21.15.112:0/1410996349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed81034f0 msgr2=0x7f2ed819bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.926+0000 7f2edea00700 1 --2- 172.21.15.112:0/1410996349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed81034f0 0x7f2ed819bd60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8000c00 tx=0x7f2ec80057c0 comp rx=0 tx=0).stop 2024-05-08T05:52:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.926+0000 7f2edca00700 1 -- 172.21.15.112:0/1410996349 reap_dead start 2024-05-08T05:52:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.927+0000 7f2edea00700 1 -- 172.21.15.112:0/1410996349 shutdown_connections 2024-05-08T05:52:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.927+0000 7f2edea00700 1 -- 172.21.15.112:0/1410996349 >> 172.21.15.112:0/1410996349 conn(0x7f2ed807a650 msgr2=0x7f2ed810e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.927+0000 7f2edea00700 1 -- 172.21.15.112:0/1410996349 shutdown_connections 2024-05-08T05:52:47.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:45.927+0000 7f2edea00700 1 -- 172.21.15.112:0/1410996349 wait complete. 2024-05-08T05:52:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:47 smithi150 ceph-mon[34315]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:47.459 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:47 smithi112 ceph-mon[30003]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:47.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:47 smithi162 ceph-mon[34800]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:48.046 INFO:teuthology.orchestra.run.smithi112.stdout:283467841577 2024-05-08T05:52:48.046 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-08T05:52:48.816 INFO:teuthology.orchestra.run.smithi112.stdout:107374182479 2024-05-08T05:52:48.816 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-08T05:52:49.189 INFO:teuthology.orchestra.run.smithi112.stdout:133143986251 2024-05-08T05:52:49.190 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-08T05:52:49.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:49 smithi150 ceph-mon[34315]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-05-08T05:52:49.467 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:49 smithi112 ceph-mon[30003]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-05-08T05:52:49.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.889+0000 7fc128200700 1 -- 172.21.15.112:0/1162923278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1201056c0 msgr2=0x7fc120105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:49.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.889+0000 7fc128200700 1 --2- 172.21.15.112:0/1162923278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1201056c0 0x7fc120105b40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009990 tx=0x7fc10c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.889+0000 7fc128200700 1 -- 172.21.15.112:0/1162923278 shutdown_connections 2024-05-08T05:52:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.889+0000 7fc128200700 1 --2- 172.21.15.112:0/1162923278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc120106080 0x7fc120112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.889+0000 7fc128200700 1 --2- 172.21.15.112:0/1162923278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc12010b6c0 0x7fc12010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.889+0000 7fc128200700 1 --2- 172.21.15.112:0/1162923278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1201056c0 0x7fc120105b40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.889+0000 7fc128200700 1 -- 172.21.15.112:0/1162923278 >> 172.21.15.112:0/1162923278 conn(0x7fc1201011b0 msgr2=0x7fc1201035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.889+0000 7fc128200700 1 -- 172.21.15.112:0/1162923278 shutdown_connections 2024-05-08T05:52:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.889+0000 7fc128200700 1 -- 172.21.15.112:0/1162923278 wait complete. 2024-05-08T05:52:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.890+0000 7fc128200700 1 Processor -- start 2024-05-08T05:52:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.890+0000 7fc128200700 1 -- start start 2024-05-08T05:52:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.891+0000 7fc128200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1201056c0 0x7fc12019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.891+0000 7fc128200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc120106080 0x7fc12019c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.891+0000 7fc128200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc12010b6c0 0x7fc1201a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.891+0000 7fc128200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc120115900 con 0x7fc120106080 2024-05-08T05:52:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.891+0000 7fc128200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc120115780 con 0x7fc12010b6c0 2024-05-08T05:52:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.891+0000 7fc128200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc120115a80 con 0x7fc1201056c0 2024-05-08T05:52:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.891+0000 7fc125800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1201056c0 0x7fc12019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.891+0000 7fc124e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc120106080 0x7fc12019c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.891+0000 7fc125800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1201056c0 0x7fc12019bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50700/0 (socket says 172.21.15.112:50700) 2024-05-08T05:52:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.891+0000 7fc125800700 1 -- 172.21.15.112:0/3447228752 learned_addr learned my addr 172.21.15.112:0/3447228752 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.891+0000 7fc126200700 1 --2- 172.21.15.112:0/3447228752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc12010b6c0 0x7fc1201a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.892+0000 7fc125800700 1 -- 172.21.15.112:0/3447228752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc12010b6c0 msgr2=0x7fc1201a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.892+0000 7fc125800700 1 --2- 172.21.15.112:0/3447228752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc12010b6c0 0x7fc1201a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.892+0000 7fc125800700 1 -- 172.21.15.112:0/3447228752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc120106080 msgr2=0x7fc12019c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.892+0000 7fc125800700 1 --2- 172.21.15.112:0/3447228752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc120106080 0x7fc12019c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.892+0000 7fc125800700 1 -- 172.21.15.112:0/3447228752 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1201a0dc0 con 0x7fc1201056c0 2024-05-08T05:52:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.892+0000 7fc125800700 1 --2- 172.21.15.112:0/3447228752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1201056c0 0x7fc12019bdb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc11400e9a0 tx=0x7fc11400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.892+0000 7fc11e000700 1 -- 172.21.15.112:0/3447228752 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11400ca10 con 0x7fc1201056c0 2024-05-08T05:52:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.892+0000 7fc128200700 1 -- 172.21.15.112:0/3447228752 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1201a10b0 con 0x7fc1201056c0 2024-05-08T05:52:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.892+0000 7fc11e000700 1 -- 172.21.15.112:0/3447228752 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc11400cbb0 con 0x7fc1201056c0 2024-05-08T05:52:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.892+0000 7fc11e000700 1 -- 172.21.15.112:0/3447228752 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114018680 con 0x7fc1201056c0 2024-05-08T05:52:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.893+0000 7fc128200700 1 -- 172.21.15.112:0/3447228752 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1201a8930 con 0x7fc1201056c0 2024-05-08T05:52:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.893+0000 7fc128200700 1 -- 172.21.15.112:0/3447228752 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc120063c30 con 0x7fc1201056c0 2024-05-08T05:52:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.896+0000 7fc11e000700 1 -- 172.21.15.112:0/3447228752 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc114010430 con 0x7fc1201056c0 2024-05-08T05:52:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.897+0000 7fc11e000700 1 --2- 172.21.15.112:0/3447228752 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc1080b2910 0x7fc1080b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.898+0000 7fc11e000700 1 -- 172.21.15.112:0/3447228752 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7fc1140e2ae0 con 0x7fc1201056c0 2024-05-08T05:52:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.898+0000 7fc124e00700 1 --2- 172.21.15.112:0/3447228752 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc1080b2910 0x7fc1080b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.898+0000 7fc11e000700 1 --2- 172.21.15.112:0/3447228752 >> [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606] conn(0x7fc1080bc4d0 0x7fc1080be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.898+0000 7fc11e000700 1 -- 172.21.15.112:0/3447228752 --> [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc114024e70 con 0x7fc1080bc4d0 2024-05-08T05:52:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.898+0000 7fc11e000700 1 -- 172.21.15.112:0/3447228752 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fc1140e2e90 con 0x7fc1201056c0 2024-05-08T05:52:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.898+0000 7fc126200700 1 --2- 172.21.15.112:0/3447228752 >> [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606] conn(0x7fc1080bc4d0 0x7fc1080be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.898+0000 7fc124e00700 1 --2- 172.21.15.112:0/3447228752 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc1080b2910 0x7fc1080b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc1201037a0 tx=0x7fc10c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.899+0000 7fc126200700 1 --2- 172.21.15.112:0/3447228752 >> [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606] conn(0x7fc1080bc4d0 0x7fc1080be970 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.900+0000 7fc11e000700 1 -- 172.21.15.112:0/3447228752 <== osd.8 v2:172.21.15.162:6816/852791606 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc114024e70 con 0x7fc1080bc4d0 2024-05-08T05:52:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.918+0000 7fc128200700 1 -- 172.21.15.112:0/3447228752 --> [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc120064030 con 0x7fc1080bc4d0 2024-05-08T05:52:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.919+0000 7fc11e000700 1 -- 172.21.15.112:0/3447228752 <== osd.8 v2:172.21.15.162:6816/852791606 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc120064030 con 0x7fc1080bc4d0 2024-05-08T05:52:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.919+0000 7fc128200700 1 -- 172.21.15.112:0/3447228752 >> [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606] conn(0x7fc1080bc4d0 msgr2=0x7fc1080be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.919+0000 7fc128200700 1 --2- 172.21.15.112:0/3447228752 >> [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606] conn(0x7fc1080bc4d0 0x7fc1080be970 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.919+0000 7fc128200700 1 -- 172.21.15.112:0/3447228752 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc1080b2910 msgr2=0x7fc1080b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.919+0000 7fc128200700 1 --2- 172.21.15.112:0/3447228752 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc1080b2910 0x7fc1080b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc1201037a0 tx=0x7fc10c00b260 comp rx=0 tx=0).stop 2024-05-08T05:52:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.919+0000 7fc128200700 1 -- 172.21.15.112:0/3447228752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1201056c0 msgr2=0x7fc12019bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.919+0000 7fc128200700 1 --2- 172.21.15.112:0/3447228752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1201056c0 0x7fc12019bdb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc11400e9a0 tx=0x7fc11400ecb0 comp rx=0 tx=0).stop 2024-05-08T05:52:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.919+0000 7fc126200700 1 -- 172.21.15.112:0/3447228752 reap_dead start 2024-05-08T05:52:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.920+0000 7fc128200700 1 -- 172.21.15.112:0/3447228752 shutdown_connections 2024-05-08T05:52:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.920+0000 7fc128200700 1 -- 172.21.15.112:0/3447228752 >> 172.21.15.112:0/3447228752 conn(0x7fc1201011b0 msgr2=0x7fc1201018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.920+0000 7fc128200700 1 -- 172.21.15.112:0/3447228752 shutdown_connections 2024-05-08T05:52:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:47.920+0000 7fc128200700 1 -- 172.21.15.112:0/3447228752 wait complete. 2024-05-08T05:52:49.571 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:49 smithi162 ceph-mon[34800]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-05-08T05:52:50.560 INFO:teuthology.orchestra.run.smithi112.stdout:184683593790 2024-05-08T05:52:50.561 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-08T05:52:50.856 INFO:teuthology.orchestra.run.smithi112.stdout:77309411414 2024-05-08T05:52:50.856 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-08T05:52:50.948 INFO:teuthology.orchestra.run.smithi112.stdout:206158430264 2024-05-08T05:52:50.948 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-08T05:52:51.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:51 smithi112 ceph-mon[30003]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:51.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:51 smithi162 ceph-mon[34800]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:51.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:51 smithi150 ceph-mon[34315]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:51.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.771+0000 7f8a01600700 1 -- 172.21.15.112:0/3242655238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89fc10b680 msgr2=0x7f89fc10ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:51.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.771+0000 7f8a01600700 1 --2- 172.21.15.112:0/3242655238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89fc10b680 0x7f89fc10ba60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0099c0 tx=0x7f89ec009cd0 comp rx=0 tx=0).stop 2024-05-08T05:52:51.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.771+0000 7f8a01600700 1 -- 172.21.15.112:0/3242655238 shutdown_connections 2024-05-08T05:52:51.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.771+0000 7f8a01600700 1 --2- 172.21.15.112:0/3242655238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89fc105f90 0x7f89fc112450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:51.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.771+0000 7f8a01600700 1 --2- 172.21.15.112:0/3242655238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89fc10b680 0x7f89fc10ba60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:51.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.771+0000 7f8a01600700 1 --2- 172.21.15.112:0/3242655238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89fc1055d0 0x7f89fc105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:51.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.771+0000 7f8a01600700 1 -- 172.21.15.112:0/3242655238 >> 172.21.15.112:0/3242655238 conn(0x7f89fc101080 msgr2=0x7f89fc1034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:51.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.771+0000 7f8a01600700 1 -- 172.21.15.112:0/3242655238 shutdown_connections 2024-05-08T05:52:51.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.771+0000 7f8a01600700 1 -- 172.21.15.112:0/3242655238 wait complete. 2024-05-08T05:52:51.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.772+0000 7f8a01600700 1 Processor -- start 2024-05-08T05:52:51.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.772+0000 7f8a01600700 1 -- start start 2024-05-08T05:52:51.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f8a01600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89fc1055d0 0x7f89fc1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:51.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f8a01600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89fc105f90 0x7f89fc1a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:51.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f8a01600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89fc10b680 0x7f89fc19ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:51.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f8a01600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1157f0 con 0x7f89fc105f90 2024-05-08T05:52:51.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f8a01600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc115670 con 0x7f89fc10b680 2024-05-08T05:52:51.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f8a01600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc115970 con 0x7f89fc1055d0 2024-05-08T05:52:51.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f89fb400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89fc105f90 0x7f89fc1a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:51.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f89fb400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89fc105f90 0x7f89fc1a1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59810/0 (socket says 172.21.15.112:59810) 2024-05-08T05:52:51.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f89fb400700 1 -- 172.21.15.112:0/1309076379 learned_addr learned my addr 172.21.15.112:0/1309076379 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:52:51.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f89fbe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89fc1055d0 0x7f89fc1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:51.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f8a00c00700 1 --2- 172.21.15.112:0/1309076379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89fc10b680 0x7f89fc19ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:51.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f89fbe00700 1 -- 172.21.15.112:0/1309076379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89fc10b680 msgr2=0x7f89fc19ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:51.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f89fbe00700 1 --2- 172.21.15.112:0/1309076379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89fc10b680 0x7f89fc19ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:51.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f89fbe00700 1 -- 172.21.15.112:0/1309076379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89fc105f90 msgr2=0x7f89fc1a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:51.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f89fbe00700 1 --2- 172.21.15.112:0/1309076379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89fc105f90 0x7f89fc1a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:51.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f89fbe00700 1 -- 172.21.15.112:0/1309076379 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89fc19c170 con 0x7f89fc1055d0 2024-05-08T05:52:51.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.773+0000 7f89fb400700 1 --2- 172.21.15.112:0/1309076379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89fc105f90 0x7f89fc1a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:52:51.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.774+0000 7f89fbe00700 1 --2- 172.21.15.112:0/1309076379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89fc1055d0 0x7f89fc1a1850 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0098e0 tx=0x7f89ec005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:51.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.774+0000 7f89f8c00700 1 -- 172.21.15.112:0/1309076379 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec026070 con 0x7f89fc1055d0 2024-05-08T05:52:51.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.774+0000 7f89f8c00700 1 -- 172.21.15.112:0/1309076379 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89ec0056c0 con 0x7f89fc1055d0 2024-05-08T05:52:51.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.774+0000 7f8a01600700 1 -- 172.21.15.112:0/1309076379 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89fc19c400 con 0x7f89fc1055d0 2024-05-08T05:52:51.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.774+0000 7f89f8c00700 1 -- 172.21.15.112:0/1309076379 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec00f990 con 0x7f89fc1055d0 2024-05-08T05:52:51.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.774+0000 7f8a01600700 1 -- 172.21.15.112:0/1309076379 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89fc1a8720 con 0x7f89fc1055d0 2024-05-08T05:52:51.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.775+0000 7f8a01600700 1 -- 172.21.15.112:0/1309076379 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f89fc063c30 con 0x7f89fc1055d0 2024-05-08T05:52:51.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.778+0000 7f89f8c00700 1 -- 172.21.15.112:0/1309076379 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f89ec029070 con 0x7f89fc1055d0 2024-05-08T05:52:51.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.779+0000 7f89f8c00700 1 --2- 172.21.15.112:0/1309076379 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f89e00b2910 0x7f89e00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:51.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.779+0000 7f89f8c00700 1 -- 172.21.15.112:0/1309076379 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7f89ec0e2ff0 con 0x7f89fc1055d0 2024-05-08T05:52:51.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.779+0000 7f89fb400700 1 --2- 172.21.15.112:0/1309076379 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f89e00b2910 0x7f89e00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:51.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.780+0000 7f89f8c00700 1 --2- 172.21.15.112:0/1309076379 >> [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797] conn(0x7f89e00bc4d0 0x7f89e00be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:51.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.780+0000 7f89f8c00700 1 -- 172.21.15.112:0/1309076379 --> [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f89ec009030 con 0x7f89e00bc4d0 2024-05-08T05:52:51.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.780+0000 7f89f8c00700 1 -- 172.21.15.112:0/1309076379 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f89ec0e3400 con 0x7f89fc1055d0 2024-05-08T05:52:51.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.780+0000 7f8a00c00700 1 --2- 172.21.15.112:0/1309076379 >> [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797] conn(0x7f89e00bc4d0 0x7f89e00be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:51.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.780+0000 7f89fb400700 1 --2- 172.21.15.112:0/1309076379 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f89e00b2910 0x7f89e00b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f89fc10ac60 tx=0x7f89e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:51.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.781+0000 7f8a00c00700 1 --2- 172.21.15.112:0/1309076379 >> [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797] conn(0x7f89e00bc4d0 0x7f89e00be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:51.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.782+0000 7f89f8c00700 1 -- 172.21.15.112:0/1309076379 <== osd.1 v2:172.21.15.150:6800/2728419797 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f89ec009030 con 0x7f89e00bc4d0 2024-05-08T05:52:51.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.800+0000 7f8a01600700 1 -- 172.21.15.112:0/1309076379 --> [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f89fc0640b0 con 0x7f89e00bc4d0 2024-05-08T05:52:51.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.800+0000 7f89f8c00700 1 -- 172.21.15.112:0/1309076379 <== osd.1 v2:172.21.15.150:6800/2728419797 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f89fc0640b0 con 0x7f89e00bc4d0 2024-05-08T05:52:51.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.801+0000 7f8a01600700 1 -- 172.21.15.112:0/1309076379 >> [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797] conn(0x7f89e00bc4d0 msgr2=0x7f89e00be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:51.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.801+0000 7f8a01600700 1 --2- 172.21.15.112:0/1309076379 >> [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797] conn(0x7f89e00bc4d0 0x7f89e00be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:51.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.801+0000 7f8a01600700 1 -- 172.21.15.112:0/1309076379 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f89e00b2910 msgr2=0x7f89e00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:51.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.801+0000 7f8a01600700 1 --2- 172.21.15.112:0/1309076379 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f89e00b2910 0x7f89e00b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f89fc10ac60 tx=0x7f89e8009260 comp rx=0 tx=0).stop 2024-05-08T05:52:51.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.801+0000 7f8a01600700 1 -- 172.21.15.112:0/1309076379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89fc1055d0 msgr2=0x7f89fc1a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:51.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.801+0000 7f8a01600700 1 --2- 172.21.15.112:0/1309076379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89fc1055d0 0x7f89fc1a1850 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0098e0 tx=0x7f89ec005280 comp rx=0 tx=0).stop 2024-05-08T05:52:51.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.801+0000 7f8a00c00700 1 -- 172.21.15.112:0/1309076379 reap_dead start 2024-05-08T05:52:51.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.801+0000 7f8a01600700 1 -- 172.21.15.112:0/1309076379 shutdown_connections 2024-05-08T05:52:51.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.801+0000 7f8a01600700 1 -- 172.21.15.112:0/1309076379 >> 172.21.15.112:0/1309076379 conn(0x7f89fc101080 msgr2=0x7f89fc102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:51.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.801+0000 7f8a01600700 1 -- 172.21.15.112:0/1309076379 shutdown_connections 2024-05-08T05:52:51.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:51.801+0000 7f8a01600700 1 -- 172.21.15.112:0/1309076379 wait complete. 2024-05-08T05:52:52.120 INFO:teuthology.orchestra.run.smithi112.stdout:236223201333 2024-05-08T05:52:52.121 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-08T05:52:52.288 INFO:teuthology.orchestra.run.smithi112.stdout:309237645348 2024-05-08T05:52:52.289 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-08T05:52:52.538 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:52.651 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:52.652 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:52.794 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:52.796 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:52.906 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:52.907 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:53.297 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:53 smithi112 ceph-mon[30003]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T05:52:53.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:53 smithi162 ceph-mon[34800]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T05:52:53.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:53 smithi150 ceph-mon[34315]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T05:52:55.229 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:55.237 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:55 smithi112 ceph-mon[30003]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:52:55.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:55 smithi150 ceph-mon[34315]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:52:55.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:55 smithi162 ceph-mon[34800]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:52:56.498 INFO:teuthology.orchestra.run.smithi112.stdout:55834574941 2024-05-08T05:52:56.498 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-08T05:52:57.548 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:57 smithi112 ceph-mon[30003]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:57.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:57 smithi162 ceph-mon[34800]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:57.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:57 smithi150 ceph-mon[34315]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:52:59.394 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:52:59 smithi112 ceph-mon[30003]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:59.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:52:59 smithi162 ceph-mon[34800]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:52:59.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:52:59 smithi150 ceph-mon[34315]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:00.417 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:00 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/11511171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T05:53:00.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.544+0000 7f5474a00700 1 -- 172.21.15.112:0/2883447995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5470114670 msgr2=0x7f5470116a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:00.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.544+0000 7f5474a00700 1 --2- 172.21.15.112:0/2883447995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5470114670 0x7f5470116a60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f546800b0a0 tx=0x7f546800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:53:00.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.544+0000 7f5474a00700 1 -- 172.21.15.112:0/2883447995 shutdown_connections 2024-05-08T05:53:00.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.544+0000 7f5474a00700 1 --2- 172.21.15.112:0/2883447995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5470114670 0x7f5470116a60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:00.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.544+0000 7f5474a00700 1 --2- 172.21.15.112:0/2883447995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54700756d0 0x7f5470075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:00.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.544+0000 7f5474a00700 1 --2- 172.21.15.112:0/2883447995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5470075ff0 0x7f5470114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:00.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.544+0000 7f5474a00700 1 -- 172.21.15.112:0/2883447995 >> 172.21.15.112:0/2883447995 conn(0x7f547007a600 msgr2=0x7f547007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:00.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.544+0000 7f5474a00700 1 -- 172.21.15.112:0/2883447995 shutdown_connections 2024-05-08T05:53:00.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.544+0000 7f5474a00700 1 -- 172.21.15.112:0/2883447995 wait complete. 2024-05-08T05:53:00.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.545+0000 7f5474a00700 1 Processor -- start 2024-05-08T05:53:00.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.546+0000 7f5474a00700 1 -- start start 2024-05-08T05:53:00.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.546+0000 7f5474a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54700756d0 0x7f54701a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:00.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.546+0000 7f5474a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5470075ff0 0x7f54701a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:00.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.546+0000 7f5474a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5470114670 0x7f54701a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:00.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.546+0000 7f5474a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5470119ca0 con 0x7f5470114670 2024-05-08T05:53:00.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.546+0000 7f5474a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5470119b20 con 0x7f5470075ff0 2024-05-08T05:53:00.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.546+0000 7f5474a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5470119e20 con 0x7f54700756d0 2024-05-08T05:53:00.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.546+0000 7f546e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5470114670 0x7f54701a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:00.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.546+0000 7f546de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54700756d0 0x7f54701a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:00.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.546+0000 7f546de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54700756d0 0x7f54701a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:52098/0 (socket says 172.21.15.112:52098) 2024-05-08T05:53:00.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.546+0000 7f546de00700 1 -- 172.21.15.112:0/11511171 learned_addr learned my addr 172.21.15.112:0/11511171 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:00.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.546+0000 7f546d400700 1 --2- 172.21.15.112:0/11511171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5470075ff0 0x7f54701a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:00.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.547+0000 7f546e800700 1 -- 172.21.15.112:0/11511171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54700756d0 msgr2=0x7f54701a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:00.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.547+0000 7f546e800700 1 --2- 172.21.15.112:0/11511171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54700756d0 0x7f54701a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:00.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.547+0000 7f546e800700 1 -- 172.21.15.112:0/11511171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5470075ff0 msgr2=0x7f54701a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:00.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.547+0000 7f546e800700 1 --2- 172.21.15.112:0/11511171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5470075ff0 0x7f54701a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:00.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.547+0000 7f546e800700 1 -- 172.21.15.112:0/11511171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54701a5260 con 0x7f5470114670 2024-05-08T05:53:00.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.547+0000 7f546e800700 1 --2- 172.21.15.112:0/11511171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5470114670 0x7f54701a4b20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f546800b070 tx=0x7f5468004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:00.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.547+0000 7f5466a00700 1 -- 172.21.15.112:0/11511171 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468021070 con 0x7f5470114670 2024-05-08T05:53:00.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.547+0000 7f5466a00700 1 -- 172.21.15.112:0/11511171 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5468004e30 con 0x7f5470114670 2024-05-08T05:53:00.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.547+0000 7f5466a00700 1 -- 172.21.15.112:0/11511171 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468004620 con 0x7f5470114670 2024-05-08T05:53:00.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.547+0000 7f5474a00700 1 -- 172.21.15.112:0/11511171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54701a54f0 con 0x7f5470114670 2024-05-08T05:53:00.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.548+0000 7f5474a00700 1 -- 172.21.15.112:0/11511171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54701acdd0 con 0x7f5470114670 2024-05-08T05:53:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.549+0000 7f5474a00700 1 -- 172.21.15.112:0/11511171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5470076dc0 con 0x7f5470114670 2024-05-08T05:53:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.551+0000 7f5466a00700 1 -- 172.21.15.112:0/11511171 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f5468007980 con 0x7f5470114670 2024-05-08T05:53:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.553+0000 7f5466a00700 1 --2- 172.21.15.112:0/11511171 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f54540b2990 0x7f54540b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.553+0000 7f5466a00700 1 -- 172.21.15.112:0/11511171 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7f54680e3220 con 0x7f5470114670 2024-05-08T05:53:00.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.553+0000 7f546de00700 1 --2- 172.21.15.112:0/11511171 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f54540b2990 0x7f54540b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.554+0000 7f546de00700 1 --2- 172.21.15.112:0/11511171 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f54540b2990 0x7f54540b4e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5460007c90 tx=0x7f5460007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.556+0000 7f5466a00700 1 -- 172.21.15.112:0/11511171 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f54680aae00 con 0x7f5470114670 2024-05-08T05:53:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.794+0000 7f5474a00700 1 -- 172.21.15.112:0/11511171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f547006a190 con 0x7f5470114670 2024-05-08T05:53:00.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.795+0000 7f5466a00700 1 -- 172.21.15.112:0/11511171 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f547006a190 con 0x7f5470114670 2024-05-08T05:53:00.455 INFO:teuthology.orchestra.run.smithi112.stdout:107374182482 2024-05-08T05:53:00.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.798+0000 7f5474a00700 1 -- 172.21.15.112:0/11511171 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f54540b2990 msgr2=0x7f54540b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:00.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.798+0000 7f5474a00700 1 --2- 172.21.15.112:0/11511171 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f54540b2990 0x7f54540b4e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5460007c90 tx=0x7f5460007480 comp rx=0 tx=0).stop 2024-05-08T05:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.798+0000 7f5474a00700 1 -- 172.21.15.112:0/11511171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5470114670 msgr2=0x7f54701a4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.798+0000 7f5474a00700 1 --2- 172.21.15.112:0/11511171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5470114670 0x7f54701a4b20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f546800b070 tx=0x7f5468004c80 comp rx=0 tx=0).stop 2024-05-08T05:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.798+0000 7f5474a00700 1 -- 172.21.15.112:0/11511171 shutdown_connections 2024-05-08T05:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.798+0000 7f5474a00700 1 --2- 172.21.15.112:0/11511171 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f54540b2990 0x7f54540b4e50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.798+0000 7f5474a00700 1 --2- 172.21.15.112:0/11511171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5470114670 0x7f54701a4b20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:00.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.798+0000 7f5474a00700 1 --2- 172.21.15.112:0/11511171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54700756d0 0x7f54701a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:00.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.798+0000 7f5474a00700 1 --2- 172.21.15.112:0/11511171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5470075ff0 0x7f54701a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:00.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.798+0000 7f5474a00700 1 -- 172.21.15.112:0/11511171 >> 172.21.15.112:0/11511171 conn(0x7f547007a600 msgr2=0x7f54701134a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:00.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.798+0000 7f5474a00700 1 -- 172.21.15.112:0/11511171 shutdown_connections 2024-05-08T05:53:00.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:52:59.798+0000 7f5474a00700 1 -- 172.21.15.112:0/11511171 wait complete. 2024-05-08T05:53:00.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:00 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/11511171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T05:53:00.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:00 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/11511171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T05:53:01.440 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:01 smithi112 ceph-mon[30003]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-05-08T05:53:01.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:01 smithi162 ceph-mon[34800]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-05-08T05:53:01.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:01 smithi150 ceph-mon[34315]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-05-08T05:53:02.387 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:02 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:53:02.528 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:02 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:53:02.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:02 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:53:02.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.902+0000 7f8e6b400700 1 -- 172.21.15.112:0/4209202132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e6406bad0 msgr2=0x7f8e6410fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:02.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.902+0000 7f8e6b400700 1 --2- 172.21.15.112:0/4209202132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e6406bad0 0x7f8e6410fa10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009990 tx=0x7f8e50009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.903+0000 7f8e6b400700 1 -- 172.21.15.112:0/4209202132 shutdown_connections 2024-05-08T05:53:02.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.903+0000 7f8e6b400700 1 --2- 172.21.15.112:0/4209202132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e6410ff50 0x7f8e64112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.903+0000 7f8e6b400700 1 --2- 172.21.15.112:0/4209202132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6406b1b0 0x7f8e6406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.903+0000 7f8e6b400700 1 --2- 172.21.15.112:0/4209202132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e6406bad0 0x7f8e6410fa10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.903+0000 7f8e6b400700 1 -- 172.21.15.112:0/4209202132 >> 172.21.15.112:0/4209202132 conn(0x7f8e64079350 msgr2=0x7f8e64079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:02.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.903+0000 7f8e6b400700 1 -- 172.21.15.112:0/4209202132 shutdown_connections 2024-05-08T05:53:02.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.903+0000 7f8e6b400700 1 -- 172.21.15.112:0/4209202132 wait complete. 2024-05-08T05:53:02.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.904+0000 7f8e6b400700 1 Processor -- start 2024-05-08T05:53:02.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.904+0000 7f8e6b400700 1 -- start start 2024-05-08T05:53:02.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e6b400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e6406b1b0 0x7f8e6419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:02.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e6b400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e6406bad0 0x7f8e6419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:02.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e6b400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6410ff50 0x7f8e641a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:02.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e6b400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8e641158b0 con 0x7f8e6410ff50 2024-05-08T05:53:02.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e6b400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8e64115730 con 0x7f8e6406bad0 2024-05-08T05:53:02.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e6b400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8e64115a30 con 0x7f8e6406b1b0 2024-05-08T05:53:02.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e68a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e6406b1b0 0x7f8e6419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:02.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e63e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e6406bad0 0x7f8e6419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:02.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e68a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e6406b1b0 0x7f8e6419be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:52106/0 (socket says 172.21.15.112:52106) 2024-05-08T05:53:02.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e68a00700 1 -- 172.21.15.112:0/3865043104 learned_addr learned my addr 172.21.15.112:0/3865043104 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e69400700 1 --2- 172.21.15.112:0/3865043104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6410ff50 0x7f8e641a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e68a00700 1 -- 172.21.15.112:0/3865043104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e6406bad0 msgr2=0x7f8e6419c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e68a00700 1 --2- 172.21.15.112:0/3865043104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e6406bad0 0x7f8e6419c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e68a00700 1 -- 172.21.15.112:0/3865043104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6410ff50 msgr2=0x7f8e641a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e68a00700 1 --2- 172.21.15.112:0/3865043104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6410ff50 0x7f8e641a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.905+0000 7f8e68a00700 1 -- 172.21.15.112:0/3865043104 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e641a0e00 con 0x7f8e6406b1b0 2024-05-08T05:53:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.906+0000 7f8e63e00700 1 --2- 172.21.15.112:0/3865043104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e6406bad0 0x7f8e6419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.906+0000 7f8e68a00700 1 --2- 172.21.15.112:0/3865043104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e6406b1b0 0x7f8e6419be40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800eb30 tx=0x7f8e5800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.906+0000 7f8e61600700 1 -- 172.21.15.112:0/3865043104 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5800cbd0 con 0x7f8e6406b1b0 2024-05-08T05:53:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.906+0000 7f8e6b400700 1 -- 172.21.15.112:0/3865043104 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e641a10f0 con 0x7f8e6406b1b0 2024-05-08T05:53:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.906+0000 7f8e61600700 1 -- 172.21.15.112:0/3865043104 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e580044d0 con 0x7f8e6406b1b0 2024-05-08T05:53:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.906+0000 7f8e61600700 1 -- 172.21.15.112:0/3865043104 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58003e30 con 0x7f8e6406b1b0 2024-05-08T05:53:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.906+0000 7f8e6b400700 1 -- 172.21.15.112:0/3865043104 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e641a8970 con 0x7f8e6406b1b0 2024-05-08T05:53:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.907+0000 7f8e6b400700 1 -- 172.21.15.112:0/3865043104 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e6406a190 con 0x7f8e6406b1b0 2024-05-08T05:53:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.909+0000 7f8e61600700 1 -- 172.21.15.112:0/3865043104 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8e58004000 con 0x7f8e6406b1b0 2024-05-08T05:53:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.911+0000 7f8e61600700 1 --2- 172.21.15.112:0/3865043104 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8e4c0b2910 0x7f8e4c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.911+0000 7f8e61600700 1 -- 172.21.15.112:0/3865043104 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8248+0+0 (secure 0 0 0) 0x7f8e580e27a0 con 0x7f8e6406b1b0 2024-05-08T05:53:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.911+0000 7f8e63e00700 1 --2- 172.21.15.112:0/3865043104 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8e4c0b2910 0x7f8e4c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.912+0000 7f8e63e00700 1 --2- 172.21.15.112:0/3865043104 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8e4c0b2910 0x7f8e4c0b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8e64103900 tx=0x7f8e5000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:02.918+0000 7f8e61600700 1 -- 172.21.15.112:0/3865043104 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8e580042f0 con 0x7f8e6406b1b0 2024-05-08T05:53:03.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:03.167+0000 7f8e6b400700 1 -- 172.21.15.112:0/3865043104 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8e6402e510 con 0x7f8e6406b1b0 2024-05-08T05:53:03.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:03.168+0000 7f8e61600700 1 -- 172.21.15.112:0/3865043104 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8e580aa2f0 con 0x7f8e6406b1b0 2024-05-08T05:53:03.169 INFO:teuthology.orchestra.run.smithi112.stdout:34359738467 2024-05-08T05:53:03.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:03.170+0000 7f8e6b400700 1 -- 172.21.15.112:0/3865043104 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8e4c0b2910 msgr2=0x7f8e4c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:03.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:03.170+0000 7f8e6b400700 1 --2- 172.21.15.112:0/3865043104 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8e4c0b2910 0x7f8e4c0b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8e64103900 tx=0x7f8e5000b260 comp rx=0 tx=0).stop 2024-05-08T05:53:03.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:03.170+0000 7f8e6b400700 1 -- 172.21.15.112:0/3865043104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e6406b1b0 msgr2=0x7f8e6419be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:03.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:03.170+0000 7f8e6b400700 1 --2- 172.21.15.112:0/3865043104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e6406b1b0 0x7f8e6419be40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800eb30 tx=0x7f8e5800ee40 comp rx=0 tx=0).stop 2024-05-08T05:53:03.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:03.171+0000 7f8e6b400700 1 -- 172.21.15.112:0/3865043104 shutdown_connections 2024-05-08T05:53:03.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:03.171+0000 7f8e6b400700 1 --2- 172.21.15.112:0/3865043104 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8e4c0b2910 0x7f8e4c0b4dd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:03.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:03.171+0000 7f8e6b400700 1 --2- 172.21.15.112:0/3865043104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e6410ff50 0x7f8e641a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:03.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:03.171+0000 7f8e6b400700 1 --2- 172.21.15.112:0/3865043104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e6406b1b0 0x7f8e6419be40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:03.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:03.171+0000 7f8e6b400700 1 --2- 172.21.15.112:0/3865043104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e6406bad0 0x7f8e6419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:03.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:03.171+0000 7f8e6b400700 1 -- 172.21.15.112:0/3865043104 >> 172.21.15.112:0/3865043104 conn(0x7f8e64079350 msgr2=0x7f8e641114a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:03.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:03.171+0000 7f8e6b400700 1 -- 172.21.15.112:0/3865043104 shutdown_connections 2024-05-08T05:53:03.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:03.171+0000 7f8e6b400700 1 -- 172.21.15.112:0/3865043104 wait complete. 2024-05-08T05:53:03.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:03 smithi112 ceph-mon[30003]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:03.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:03 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:03.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:03 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:03.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:03 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3865043104' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T05:53:03.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:03 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:03.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:03 smithi162 ceph-mon[34800]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:03.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:03 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:03.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:03 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:03.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:03 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3865043104' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T05:53:03.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:03 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:03.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:03 smithi150 ceph-mon[34315]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:03.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:03 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:03.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:03 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:03.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:03 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3865043104' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T05:53:03.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:03 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:04.451 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:04.451 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:04.451 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:04.451 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:04 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-08T05:53:04.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:04.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:04.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:04.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:04 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-08T05:53:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:04.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:04 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-08T05:53:04.695 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:04.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.852+0000 7f1865a00700 1 -- 172.21.15.112:0/741645719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18601056e0 msgr2=0x7f1860105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:04.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.852+0000 7f1865a00700 1 --2- 172.21.15.112:0/741645719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18601056e0 0x7f1860105b60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f184c0098f0 tx=0x7f184c009c00 comp rx=0 tx=0).stop 2024-05-08T05:53:04.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.852+0000 7f1865a00700 1 -- 172.21.15.112:0/741645719 shutdown_connections 2024-05-08T05:53:04.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.852+0000 7f1865a00700 1 --2- 172.21.15.112:0/741645719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18601060a0 0x7f18601125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:04.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.852+0000 7f1865a00700 1 --2- 172.21.15.112:0/741645719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f186010b6e0 0x7f186010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.852+0000 7f1865a00700 1 --2- 172.21.15.112:0/741645719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18601056e0 0x7f1860105b60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.852+0000 7f1865a00700 1 -- 172.21.15.112:0/741645719 >> 172.21.15.112:0/741645719 conn(0x7f18601011d0 msgr2=0x7f18601035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.852+0000 7f1865a00700 1 -- 172.21.15.112:0/741645719 shutdown_connections 2024-05-08T05:53:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.852+0000 7f1865a00700 1 -- 172.21.15.112:0/741645719 wait complete. 2024-05-08T05:53:04.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.854+0000 7f1865a00700 1 Processor -- start 2024-05-08T05:53:04.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.854+0000 7f1865a00700 1 -- start start 2024-05-08T05:53:04.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.854+0000 7f1865a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18601056e0 0x7f186019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:04.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.854+0000 7f1865a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18601060a0 0x7f186019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:04.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.854+0000 7f1865a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f186010b6e0 0x7f18601a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:04.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.854+0000 7f1865a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1860115920 con 0x7f186010b6e0 2024-05-08T05:53:04.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.854+0000 7f1865a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f18601157a0 con 0x7f18601056e0 2024-05-08T05:53:04.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.854+0000 7f1865a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1860115aa0 con 0x7f18601060a0 2024-05-08T05:53:04.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.855+0000 7f185ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18601056e0 0x7f186019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:04.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.855+0000 7f185ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18601056e0 0x7f186019bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:42986/0 (socket says 172.21.15.112:42986) 2024-05-08T05:53:04.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.855+0000 7f185ea00700 1 -- 172.21.15.112:0/1892121149 learned_addr learned my addr 172.21.15.112:0/1892121149 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:04.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.855+0000 7f185e000700 1 --2- 172.21.15.112:0/1892121149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18601060a0 0x7f186019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:04.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.855+0000 7f185f400700 1 --2- 172.21.15.112:0/1892121149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f186010b6e0 0x7f18601a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:04.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.855+0000 7f185ea00700 1 -- 172.21.15.112:0/1892121149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18601060a0 msgr2=0x7f186019c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:04.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.855+0000 7f185ea00700 1 --2- 172.21.15.112:0/1892121149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18601060a0 0x7f186019c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:04.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.855+0000 7f185ea00700 1 -- 172.21.15.112:0/1892121149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f186010b6e0 msgr2=0x7f18601a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:04.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.855+0000 7f185ea00700 1 --2- 172.21.15.112:0/1892121149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f186010b6e0 0x7f18601a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:04.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.855+0000 7f185ea00700 1 -- 172.21.15.112:0/1892121149 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18601a0e00 con 0x7f18601056e0 2024-05-08T05:53:04.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.855+0000 7f185f400700 1 --2- 172.21.15.112:0/1892121149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f186010b6e0 0x7f18601a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:53:04.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.855+0000 7f185ea00700 1 --2- 172.21.15.112:0/1892121149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18601056e0 0x7f186019bdf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f185000eb30 tx=0x7f185000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:04.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.856+0000 7f1857400700 1 -- 172.21.15.112:0/1892121149 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185000cc90 con 0x7f18601056e0 2024-05-08T05:53:04.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.856+0000 7f1865a00700 1 -- 172.21.15.112:0/1892121149 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18601a10f0 con 0x7f18601056e0 2024-05-08T05:53:04.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.856+0000 7f1857400700 1 -- 172.21.15.112:0/1892121149 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18500044d0 con 0x7f18601056e0 2024-05-08T05:53:04.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.856+0000 7f1857400700 1 -- 172.21.15.112:0/1892121149 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850010430 con 0x7f18601056e0 2024-05-08T05:53:04.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.856+0000 7f1865a00700 1 -- 172.21.15.112:0/1892121149 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18601a8970 con 0x7f18601056e0 2024-05-08T05:53:04.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.857+0000 7f1865a00700 1 -- 172.21.15.112:0/1892121149 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f186006a190 con 0x7f18601056e0 2024-05-08T05:53:04.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.859+0000 7f1857400700 1 -- 172.21.15.112:0/1892121149 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1850003ea0 con 0x7f18601056e0 2024-05-08T05:53:04.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.861+0000 7f1857400700 1 --2- 172.21.15.112:0/1892121149 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f18440b2880 0x7f18440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:04.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.861+0000 7f1857400700 1 -- 172.21.15.112:0/1892121149 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8277+0+0 (secure 0 0 0) 0x7f18500e24a0 con 0x7f18601056e0 2024-05-08T05:53:04.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.861+0000 7f185e000700 1 --2- 172.21.15.112:0/1892121149 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f18440b2880 0x7f18440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:04.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.862+0000 7f185e000700 1 --2- 172.21.15.112:0/1892121149 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f18440b2880 0x7f18440b4d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f184c0098c0 tx=0x7f184c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:04.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:04.865+0000 7f1857400700 1 -- 172.21.15.112:0/1892121149 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f18500aa130 con 0x7f18601056e0 2024-05-08T05:53:05.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.108+0000 7f1865a00700 1 -- 172.21.15.112:0/1892121149 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f186019d150 con 0x7f18601056e0 2024-05-08T05:53:05.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.108+0000 7f1857400700 1 -- 172.21.15.112:0/1892121149 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f18500a9a30 con 0x7f18601056e0 2024-05-08T05:53:05.109 INFO:teuthology.orchestra.run.smithi112.stdout:77309411418 2024-05-08T05:53:05.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.112+0000 7f1865a00700 1 -- 172.21.15.112:0/1892121149 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f18440b2880 msgr2=0x7f18440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.112+0000 7f1865a00700 1 --2- 172.21.15.112:0/1892121149 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f18440b2880 0x7f18440b4d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f184c0098c0 tx=0x7f184c019040 comp rx=0 tx=0).stop 2024-05-08T05:53:05.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.112+0000 7f1865a00700 1 -- 172.21.15.112:0/1892121149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18601056e0 msgr2=0x7f186019bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.112+0000 7f1865a00700 1 --2- 172.21.15.112:0/1892121149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18601056e0 0x7f186019bdf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f185000eb30 tx=0x7f185000ee40 comp rx=0 tx=0).stop 2024-05-08T05:53:05.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.112+0000 7f1865a00700 1 -- 172.21.15.112:0/1892121149 shutdown_connections 2024-05-08T05:53:05.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.112+0000 7f1865a00700 1 --2- 172.21.15.112:0/1892121149 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f18440b2880 0x7f18440b4d40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.112+0000 7f1865a00700 1 --2- 172.21.15.112:0/1892121149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f186010b6e0 0x7f18601a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.112+0000 7f1865a00700 1 --2- 172.21.15.112:0/1892121149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18601056e0 0x7f186019bdf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.112+0000 7f1865a00700 1 --2- 172.21.15.112:0/1892121149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18601060a0 0x7f186019c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.112+0000 7f1865a00700 1 -- 172.21.15.112:0/1892121149 >> 172.21.15.112:0/1892121149 conn(0x7f18601011d0 msgr2=0x7f1860101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:05.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.112+0000 7f1865a00700 1 -- 172.21.15.112:0/1892121149 shutdown_connections 2024-05-08T05:53:05.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.112+0000 7f1865a00700 1 -- 172.21.15.112:0/1892121149 wait complete. 2024-05-08T05:53:05.463 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:05 smithi112 ceph-mon[30003]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:53:05.463 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:05 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-08T05:53:05.463 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:05 smithi112 ceph-mon[30003]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-08T05:53:05.463 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:05 smithi112 ceph-mon[30003]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T05:53:05.463 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:05 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1892121149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T05:53:05.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:05 smithi162 ceph-mon[34800]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:53:05.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:05 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-08T05:53:05.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:05 smithi162 ceph-mon[34800]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-08T05:53:05.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:05 smithi162 ceph-mon[34800]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T05:53:05.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:05 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1892121149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T05:53:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.627+0000 7f254dc00700 1 -- 172.21.15.112:0/21886592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25481033d0 msgr2=0x7f2548103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.627+0000 7f254dc00700 1 --2- 172.21.15.112:0/21886592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25481033d0 0x7f2548103850 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f253000b0a0 tx=0x7f253000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.627+0000 7f254dc00700 1 -- 172.21.15.112:0/21886592 shutdown_connections 2024-05-08T05:53:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.627+0000 7f254dc00700 1 --2- 172.21.15.112:0/21886592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2548103d90 0x7f25481102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.627+0000 7f254dc00700 1 --2- 172.21.15.112:0/21886592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2548109460 0x7f2548109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.627+0000 7f254dc00700 1 --2- 172.21.15.112:0/21886592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25481033d0 0x7f2548103850 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.627+0000 7f254dc00700 1 -- 172.21.15.112:0/21886592 >> 172.21.15.112:0/21886592 conn(0x7f25480fef20 msgr2=0x7f2548101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.627+0000 7f254dc00700 1 -- 172.21.15.112:0/21886592 shutdown_connections 2024-05-08T05:53:05.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.627+0000 7f254dc00700 1 -- 172.21.15.112:0/21886592 wait complete. 2024-05-08T05:53:05.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.628+0000 7f254dc00700 1 Processor -- start 2024-05-08T05:53:05.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.628+0000 7f254dc00700 1 -- start start 2024-05-08T05:53:05.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.629+0000 7f254dc00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25481033d0 0x7f25481a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.629+0000 7f254dc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2548103d90 0x7f25481a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.629+0000 7f254dc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2548109460 0x7f254819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.629+0000 7f254dc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2548113570 con 0x7f2548109460 2024-05-08T05:53:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.629+0000 7f254dc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f25481133f0 con 0x7f2548103d90 2024-05-08T05:53:05.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.629+0000 7f254dc00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f25481136f0 con 0x7f25481033d0 2024-05-08T05:53:05.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.629+0000 7f254d200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2548109460 0x7f254819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.629+0000 7f2547e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25481033d0 0x7f25481a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.629+0000 7f254d200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2548109460 0x7f254819ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51608/0 (socket says 172.21.15.112:51608) 2024-05-08T05:53:05.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.629+0000 7f254d200700 1 -- 172.21.15.112:0/3434821375 learned_addr learned my addr 172.21.15.112:0/3434821375 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:05.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.629+0000 7f253f400700 1 --2- 172.21.15.112:0/3434821375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2548103d90 0x7f25481a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.630+0000 7f254d200700 1 -- 172.21.15.112:0/3434821375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25481033d0 msgr2=0x7f25481a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.630+0000 7f254d200700 1 --2- 172.21.15.112:0/3434821375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25481033d0 0x7f25481a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.630+0000 7f254d200700 1 -- 172.21.15.112:0/3434821375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2548103d90 msgr2=0x7f25481a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.630+0000 7f254d200700 1 --2- 172.21.15.112:0/3434821375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2548103d90 0x7f25481a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.630+0000 7f254d200700 1 -- 172.21.15.112:0/3434821375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254819c1f0 con 0x7f2548109460 2024-05-08T05:53:05.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.630+0000 7f2547e00700 1 --2- 172.21.15.112:0/3434821375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25481033d0 0x7f25481a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:05.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.630+0000 7f254d200700 1 --2- 172.21.15.112:0/3434821375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2548109460 0x7f254819ba90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f254000d980 tx=0x7f254000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:05.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.630+0000 7f2545600700 1 -- 172.21.15.112:0/3434821375 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540017070 con 0x7f2548109460 2024-05-08T05:53:05.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.630+0000 7f2545600700 1 -- 172.21.15.112:0/3434821375 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2540005020 con 0x7f2548109460 2024-05-08T05:53:05.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.630+0000 7f2545600700 1 -- 172.21.15.112:0/3434821375 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254000fa10 con 0x7f2548109460 2024-05-08T05:53:05.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.630+0000 7f254dc00700 1 -- 172.21.15.112:0/3434821375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254819c4e0 con 0x7f2548109460 2024-05-08T05:53:05.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.631+0000 7f254dc00700 1 -- 172.21.15.112:0/3434821375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25481a87a0 con 0x7f2548109460 2024-05-08T05:53:05.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.632+0000 7f254dc00700 1 -- 172.21.15.112:0/3434821375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f254806a190 con 0x7f2548109460 2024-05-08T05:53:05.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.634+0000 7f2545600700 1 -- 172.21.15.112:0/3434821375 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2540004a30 con 0x7f2548109460 2024-05-08T05:53:05.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.635+0000 7f2545600700 1 --2- 172.21.15.112:0/3434821375 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f252c0b2900 0x7f252c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.635+0000 7f2545600700 1 -- 172.21.15.112:0/3434821375 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f25400e2070 con 0x7f2548109460 2024-05-08T05:53:05.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.636+0000 7f2547e00700 1 --2- 172.21.15.112:0/3434821375 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f252c0b2900 0x7f252c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.636+0000 7f2547e00700 1 --2- 172.21.15.112:0/3434821375 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f252c0b2900 0x7f252c0b4dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f254819cbd0 tx=0x7f25380073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:05.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.640+0000 7f2545600700 1 -- 172.21.15.112:0/3434821375 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2540020020 con 0x7f2548109460 2024-05-08T05:53:05.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:05 smithi150 ceph-mon[34315]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:53:05.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:05 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-08T05:53:05.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:05 smithi150 ceph-mon[34315]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-08T05:53:05.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:05 smithi150 ceph-mon[34315]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T05:53:05.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:05 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1892121149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T05:53:05.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.878+0000 7f254dc00700 1 -- 172.21.15.112:0/3434821375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f25481a89d0 con 0x7f2548109460 2024-05-08T05:53:05.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.879+0000 7f2545600700 1 -- 172.21.15.112:0/3434821375 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f25400a9d00 con 0x7f2548109460 2024-05-08T05:53:05.880 INFO:teuthology.orchestra.run.smithi112.stdout:133143986254 2024-05-08T05:53:05.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.884+0000 7f254dc00700 1 -- 172.21.15.112:0/3434821375 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f252c0b2900 msgr2=0x7f252c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.884+0000 7f254dc00700 1 --2- 172.21.15.112:0/3434821375 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f252c0b2900 0x7f252c0b4dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f254819cbd0 tx=0x7f25380073d0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.884+0000 7f254dc00700 1 -- 172.21.15.112:0/3434821375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2548109460 msgr2=0x7f254819ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.884+0000 7f254dc00700 1 --2- 172.21.15.112:0/3434821375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2548109460 0x7f254819ba90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f254000d980 tx=0x7f254000dc90 comp rx=0 tx=0).stop 2024-05-08T05:53:05.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.884+0000 7f254dc00700 1 -- 172.21.15.112:0/3434821375 shutdown_connections 2024-05-08T05:53:05.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.884+0000 7f254dc00700 1 --2- 172.21.15.112:0/3434821375 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f252c0b2900 0x7f252c0b4dc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.884+0000 7f254dc00700 1 --2- 172.21.15.112:0/3434821375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2548109460 0x7f254819ba90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.884+0000 7f254dc00700 1 --2- 172.21.15.112:0/3434821375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25481033d0 0x7f25481a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.884+0000 7f254dc00700 1 --2- 172.21.15.112:0/3434821375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2548103d90 0x7f25481a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.884+0000 7f254dc00700 1 -- 172.21.15.112:0/3434821375 >> 172.21.15.112:0/3434821375 conn(0x7f25480fef20 msgr2=0x7f254810eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:05.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.884+0000 7f254dc00700 1 -- 172.21.15.112:0/3434821375 shutdown_connections 2024-05-08T05:53:05.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:05.885+0000 7f254dc00700 1 -- 172.21.15.112:0/3434821375 wait complete. 2024-05-08T05:53:06.419 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:06 smithi112 ceph-mon[30003]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T05:53:06.419 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:06 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3434821375' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T05:53:06.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:06 smithi162 ceph-mon[34800]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T05:53:06.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:06 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3434821375' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T05:53:06.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:06 smithi150 ceph-mon[34315]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T05:53:06.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:06 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3434821375' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T05:53:07.217 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182479 got 107374182482 for osd.3 2024-05-08T05:53:07.217 DEBUG:teuthology.parallel:result is None 2024-05-08T05:53:07.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:07 smithi112 ceph-mon[30003]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-08T05:53:07.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:07 smithi162 ceph-mon[34800]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-08T05:53:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:07 smithi150 ceph-mon[34315]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-08T05:53:08.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.158+0000 7f43c4e00700 1 -- 172.21.15.112:0/2304513374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43c0109600 msgr2=0x7f43c01099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:08.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.158+0000 7f43c4e00700 1 --2- 172.21.15.112:0/2304513374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43c0109600 0x7f43c01099e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009990 tx=0x7f43b0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.158+0000 7f43c4e00700 1 -- 172.21.15.112:0/2304513374 shutdown_connections 2024-05-08T05:53:08.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.158+0000 7f43c4e00700 1 --2- 172.21.15.112:0/2304513374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43c0103eb0 0x7f43c0110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.158+0000 7f43c4e00700 1 --2- 172.21.15.112:0/2304513374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43c0109600 0x7f43c01099e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.158+0000 7f43c4e00700 1 --2- 172.21.15.112:0/2304513374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43c01034f0 0x7f43c0103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.158+0000 7f43c4e00700 1 -- 172.21.15.112:0/2304513374 >> 172.21.15.112:0/2304513374 conn(0x7f43c007a650 msgr2=0x7f43c007aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:08.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.158+0000 7f43c4e00700 1 -- 172.21.15.112:0/2304513374 shutdown_connections 2024-05-08T05:53:08.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.158+0000 7f43c4e00700 1 -- 172.21.15.112:0/2304513374 wait complete. 2024-05-08T05:53:08.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.160+0000 7f43c4e00700 1 Processor -- start 2024-05-08T05:53:08.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.160+0000 7f43c4e00700 1 -- start start 2024-05-08T05:53:08.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.160+0000 7f43c4e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43c01034f0 0x7f43c019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:08.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.161+0000 7f43c4e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43c0103eb0 0x7f43c019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:08.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.161+0000 7f43c4e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43c0109600 0x7f43c0199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:08.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.161+0000 7f43bde00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43c01034f0 0x7f43c019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:08.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.161+0000 7f43c4e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0113650 con 0x7f43c01034f0 2024-05-08T05:53:08.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.161+0000 7f43c4e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43c01134d0 con 0x7f43c0103eb0 2024-05-08T05:53:08.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.161+0000 7f43c4e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f43c01137d0 con 0x7f43c0109600 2024-05-08T05:53:08.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.161+0000 7f43bde00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43c01034f0 0x7f43c019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51644/0 (socket says 172.21.15.112:51644) 2024-05-08T05:53:08.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.161+0000 7f43bde00700 1 -- 172.21.15.112:0/2571004248 learned_addr learned my addr 172.21.15.112:0/2571004248 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:08.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.161+0000 7f43bd400700 1 --2- 172.21.15.112:0/2571004248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43c0103eb0 0x7f43c019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:08.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.161+0000 7f43be800700 1 --2- 172.21.15.112:0/2571004248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43c0109600 0x7f43c0199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:08.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.161+0000 7f43bd400700 1 -- 172.21.15.112:0/2571004248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43c0109600 msgr2=0x7f43c0199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:08.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.161+0000 7f43bd400700 1 --2- 172.21.15.112:0/2571004248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43c0109600 0x7f43c0199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.161+0000 7f43bd400700 1 -- 172.21.15.112:0/2571004248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43c01034f0 msgr2=0x7f43c019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:08.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.161+0000 7f43bd400700 1 --2- 172.21.15.112:0/2571004248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43c01034f0 0x7f43c019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.161+0000 7f43bd400700 1 -- 172.21.15.112:0/2571004248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c019a0b0 con 0x7f43c0103eb0 2024-05-08T05:53:08.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.162+0000 7f43bd400700 1 --2- 172.21.15.112:0/2571004248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43c0103eb0 0x7f43c019fc80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00e9f0 tx=0x7f43ac00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:08.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.162+0000 7f43b6a00700 1 -- 172.21.15.112:0/2571004248 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac00cb70 con 0x7f43c0103eb0 2024-05-08T05:53:08.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.162+0000 7f43c4e00700 1 -- 172.21.15.112:0/2571004248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c019a3a0 con 0x7f43c0103eb0 2024-05-08T05:53:08.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.162+0000 7f43b6a00700 1 -- 172.21.15.112:0/2571004248 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43ac00ee90 con 0x7f43c0103eb0 2024-05-08T05:53:08.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.162+0000 7f43b6a00700 1 -- 172.21.15.112:0/2571004248 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac003d40 con 0x7f43c0103eb0 2024-05-08T05:53:08.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.162+0000 7f43c4e00700 1 -- 172.21.15.112:0/2571004248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c01a6610 con 0x7f43c0103eb0 2024-05-08T05:53:08.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.163+0000 7f43c4e00700 1 -- 172.21.15.112:0/2571004248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c006a190 con 0x7f43c0103eb0 2024-05-08T05:53:08.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.167+0000 7f43b6a00700 1 -- 172.21.15.112:0/2571004248 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f43ac047960 con 0x7f43c0103eb0 2024-05-08T05:53:08.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.168+0000 7f43b6a00700 1 --2- 172.21.15.112:0/2571004248 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f43a40b2900 0x7f43a40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:08.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.168+0000 7f43b6a00700 1 -- 172.21.15.112:0/2571004248 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f43ac0e2470 con 0x7f43c0103eb0 2024-05-08T05:53:08.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.169+0000 7f43bde00700 1 --2- 172.21.15.112:0/2571004248 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f43a40b2900 0x7f43a40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:08.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.169+0000 7f43bde00700 1 --2- 172.21.15.112:0/2571004248 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f43a40b2900 0x7f43a40b4dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f43b0006010 tx=0x7f43b00050c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:08.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.170+0000 7f43b6a00700 1 -- 172.21.15.112:0/2571004248 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f43ac0aa100 con 0x7f43c0103eb0 2024-05-08T05:53:08.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.433+0000 7f43c4e00700 1 -- 172.21.15.112:0/2571004248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f43c002e510 con 0x7f43c0103eb0 2024-05-08T05:53:08.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.434+0000 7f43b6a00700 1 -- 172.21.15.112:0/2571004248 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f43ac0a9a00 con 0x7f43c0103eb0 2024-05-08T05:53:08.434 INFO:teuthology.orchestra.run.smithi112.stdout:261993005107 2024-05-08T05:53:08.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.437+0000 7f43c4e00700 1 -- 172.21.15.112:0/2571004248 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f43a40b2900 msgr2=0x7f43a40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:08.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.437+0000 7f43c4e00700 1 --2- 172.21.15.112:0/2571004248 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f43a40b2900 0x7f43a40b4dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f43b0006010 tx=0x7f43b00050c0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.437+0000 7f43c4e00700 1 -- 172.21.15.112:0/2571004248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43c0103eb0 msgr2=0x7f43c019fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:08.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.437+0000 7f43c4e00700 1 --2- 172.21.15.112:0/2571004248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43c0103eb0 0x7f43c019fc80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00e9f0 tx=0x7f43ac00ed00 comp rx=0 tx=0).stop 2024-05-08T05:53:08.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.437+0000 7f43c4e00700 1 -- 172.21.15.112:0/2571004248 shutdown_connections 2024-05-08T05:53:08.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.437+0000 7f43c4e00700 1 --2- 172.21.15.112:0/2571004248 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f43a40b2900 0x7f43a40b4dc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.437+0000 7f43c4e00700 1 --2- 172.21.15.112:0/2571004248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43c0109600 0x7f43c0199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.437+0000 7f43c4e00700 1 --2- 172.21.15.112:0/2571004248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43c01034f0 0x7f43c019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.437+0000 7f43c4e00700 1 --2- 172.21.15.112:0/2571004248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43c0103eb0 0x7f43c019fc80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.437+0000 7f43c4e00700 1 -- 172.21.15.112:0/2571004248 >> 172.21.15.112:0/2571004248 conn(0x7f43c007a650 msgr2=0x7f43c01021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:08.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.437+0000 7f43c4e00700 1 -- 172.21.15.112:0/2571004248 shutdown_connections 2024-05-08T05:53:08.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.438+0000 7f43c4e00700 1 -- 172.21.15.112:0/2571004248 wait complete. 2024-05-08T05:53:09.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.037+0000 7f3424a00700 1 -- 172.21.15.112:0/3649740043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f342006b180 msgr2=0x7f342006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:09.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.037+0000 7f3424a00700 1 --2- 172.21.15.112:0/3649740043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f342006b180 0x7f342006b600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f340c009990 tx=0x7f340c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:09.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.037+0000 7f3424a00700 1 -- 172.21.15.112:0/3649740043 shutdown_connections 2024-05-08T05:53:09.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.037+0000 7f3424a00700 1 --2- 172.21.15.112:0/3649740043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f342006bb40 0x7f3420110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:09.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.037+0000 7f3424a00700 1 --2- 172.21.15.112:0/3649740043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3420109510 0x7f34201098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:09.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.037+0000 7f3424a00700 1 --2- 172.21.15.112:0/3649740043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f342006b180 0x7f342006b600 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:09.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.037+0000 7f3424a00700 1 -- 172.21.15.112:0/3649740043 >> 172.21.15.112:0/3649740043 conn(0x7f3420079350 msgr2=0x7f3420079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:09.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.037+0000 7f3424a00700 1 -- 172.21.15.112:0/3649740043 shutdown_connections 2024-05-08T05:53:09.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.037+0000 7f3424a00700 1 -- 172.21.15.112:0/3649740043 wait complete. 2024-05-08T05:53:09.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.039+0000 7f3424a00700 1 Processor -- start 2024-05-08T05:53:09.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.039+0000 7f3424a00700 1 -- start start 2024-05-08T05:53:09.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.039+0000 7f3424a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f342006b180 0x7f342019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:09.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.039+0000 7f3424a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f342006bb40 0x7f342019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:09.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.039+0000 7f3424a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420109510 0x7f34201999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.039+0000 7f3424a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3420113690 con 0x7f3420109510 2024-05-08T05:53:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.039+0000 7f3424a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3420113510 con 0x7f342006bb40 2024-05-08T05:53:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.039+0000 7f3424a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3420113810 con 0x7f342006b180 2024-05-08T05:53:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.040+0000 7f341de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f342006b180 0x7f342019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.040+0000 7f341de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f342006b180 0x7f342019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38072/0 (socket says 172.21.15.112:38072) 2024-05-08T05:53:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.040+0000 7f341de00700 1 -- 172.21.15.112:0/207679142 learned_addr learned my addr 172.21.15.112:0/207679142 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.040+0000 7f341e800700 1 --2- 172.21.15.112:0/207679142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420109510 0x7f34201999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.040+0000 7f341d400700 1 --2- 172.21.15.112:0/207679142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f342006bb40 0x7f342019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.040+0000 7f341de00700 1 -- 172.21.15.112:0/207679142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f342006bb40 msgr2=0x7f342019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.040+0000 7f341de00700 1 --2- 172.21.15.112:0/207679142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f342006bb40 0x7f342019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.040+0000 7f341de00700 1 -- 172.21.15.112:0/207679142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420109510 msgr2=0x7f34201999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.040+0000 7f341de00700 1 --2- 172.21.15.112:0/207679142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420109510 0x7f34201999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.040+0000 7f341de00700 1 -- 172.21.15.112:0/207679142 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342019a130 con 0x7f342006b180 2024-05-08T05:53:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.040+0000 7f341e800700 1 --2- 172.21.15.112:0/207679142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420109510 0x7f34201999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.041+0000 7f341de00700 1 --2- 172.21.15.112:0/207679142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f342006b180 0x7f342019f810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f341000e9c0 tx=0x7f341000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:09.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.041+0000 7f3416a00700 1 -- 172.21.15.112:0/207679142 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341000cbd0 con 0x7f342006b180 2024-05-08T05:53:09.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.041+0000 7f3424a00700 1 -- 172.21.15.112:0/207679142 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342019a420 con 0x7f342006b180 2024-05-08T05:53:09.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.041+0000 7f3416a00700 1 -- 172.21.15.112:0/207679142 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f341000ee60 con 0x7f342006b180 2024-05-08T05:53:09.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.041+0000 7f3416a00700 1 -- 172.21.15.112:0/207679142 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34100055c0 con 0x7f342006b180 2024-05-08T05:53:09.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.041+0000 7f3424a00700 1 -- 172.21.15.112:0/207679142 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34201a66e0 con 0x7f342006b180 2024-05-08T05:53:09.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.043+0000 7f3424a00700 1 -- 172.21.15.112:0/207679142 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342006a190 con 0x7f342006b180 2024-05-08T05:53:09.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.045+0000 7f3416a00700 1 -- 172.21.15.112:0/207679142 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3410047960 con 0x7f342006b180 2024-05-08T05:53:09.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.046+0000 7f3416a00700 1 --2- 172.21.15.112:0/207679142 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f34040b2880 0x7f34040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:09.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.046+0000 7f3416a00700 1 -- 172.21.15.112:0/207679142 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f34100e2470 con 0x7f342006b180 2024-05-08T05:53:09.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.046+0000 7f341d400700 1 --2- 172.21.15.112:0/207679142 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f34040b2880 0x7f34040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:09.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.047+0000 7f341d400700 1 --2- 172.21.15.112:0/207679142 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f34040b2880 0x7f34040b4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3420079a30 tx=0x7f340c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:09.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.049+0000 7f3416a00700 1 -- 172.21.15.112:0/207679142 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f34100aa100 con 0x7f342006b180 2024-05-08T05:53:09.031 INFO:teuthology.orchestra.run.smithi112.stdout:184683593796 2024-05-08T05:53:09.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.314+0000 7f3424a00700 1 -- 172.21.15.112:0/207679142 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f342019af00 con 0x7f342006b180 2024-05-08T05:53:09.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.314+0000 7f3416a00700 1 -- 172.21.15.112:0/207679142 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f34100a9a00 con 0x7f342006b180 2024-05-08T05:53:09.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.317+0000 7f3424a00700 1 -- 172.21.15.112:0/207679142 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f34040b2880 msgr2=0x7f34040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:09.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.317+0000 7f3424a00700 1 --2- 172.21.15.112:0/207679142 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f34040b2880 0x7f34040b4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3420079a30 tx=0x7f340c005fd0 comp rx=0 tx=0).stop 2024-05-08T05:53:09.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.317+0000 7f3424a00700 1 -- 172.21.15.112:0/207679142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f342006b180 msgr2=0x7f342019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:09.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.317+0000 7f3424a00700 1 --2- 172.21.15.112:0/207679142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f342006b180 0x7f342019f810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f341000e9c0 tx=0x7f341000ecd0 comp rx=0 tx=0).stop 2024-05-08T05:53:09.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.317+0000 7f3424a00700 1 -- 172.21.15.112:0/207679142 shutdown_connections 2024-05-08T05:53:09.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.317+0000 7f3424a00700 1 --2- 172.21.15.112:0/207679142 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f34040b2880 0x7f34040b4d40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:09.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.317+0000 7f3424a00700 1 --2- 172.21.15.112:0/207679142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3420109510 0x7f34201999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:09.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.317+0000 7f3424a00700 1 --2- 172.21.15.112:0/207679142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f342006b180 0x7f342019f810 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:09.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.317+0000 7f3424a00700 1 --2- 172.21.15.112:0/207679142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f342006bb40 0x7f342019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:09.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.317+0000 7f3424a00700 1 -- 172.21.15.112:0/207679142 >> 172.21.15.112:0/207679142 conn(0x7f3420079350 msgr2=0x7f3420101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:09.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.318+0000 7f3424a00700 1 -- 172.21.15.112:0/207679142 shutdown_connections 2024-05-08T05:53:09.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.318+0000 7f3424a00700 1 -- 172.21.15.112:0/207679142 wait complete. 2024-05-08T05:53:09.387 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411414 got 77309411418 for osd.2 2024-05-08T05:53:09.387 DEBUG:teuthology.parallel:result is None 2024-05-08T05:53:09.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:09 smithi112 ceph-mon[30003]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 5 op/s 2024-05-08T05:53:09.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:09 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/207679142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T05:53:09.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:09 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2571004248' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T05:53:09.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:09 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3205805306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T05:53:09.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:09 smithi162 ceph-mon[34800]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 5 op/s 2024-05-08T05:53:09.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:09 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/207679142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T05:53:09.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:09 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/2571004248' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T05:53:09.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:09 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3205805306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T05:53:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:09 smithi150 ceph-mon[34315]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 5 op/s 2024-05-08T05:53:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:09 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/207679142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T05:53:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:09 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/2571004248' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T05:53:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:09 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3205805306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T05:53:10.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.565+0000 7f5b41400700 1 -- 172.21.15.112:0/789094154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b3c0756d0 msgr2=0x7f5b3c075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:10.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.565+0000 7f5b41400700 1 --2- 172.21.15.112:0/789094154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b3c0756d0 0x7f5b3c075ab0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009990 tx=0x7f5b2c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.566+0000 7f5b41400700 1 -- 172.21.15.112:0/789094154 shutdown_connections 2024-05-08T05:53:10.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.566+0000 7f5b41400700 1 --2- 172.21.15.112:0/789094154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b3c10fed0 0x7f5b3c1122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.566+0000 7f5b41400700 1 --2- 172.21.15.112:0/789094154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b3c0756d0 0x7f5b3c075ab0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.566+0000 7f5b41400700 1 --2- 172.21.15.112:0/789094154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b3c075ff0 0x7f5b3c10f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.566+0000 7f5b41400700 1 -- 172.21.15.112:0/789094154 >> 172.21.15.112:0/789094154 conn(0x7f5b3c07a650 msgr2=0x7f5b3c07aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:10.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.566+0000 7f5b41400700 1 -- 172.21.15.112:0/789094154 shutdown_connections 2024-05-08T05:53:10.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.566+0000 7f5b41400700 1 -- 172.21.15.112:0/789094154 wait complete. 2024-05-08T05:53:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.567+0000 7f5b41400700 1 Processor -- start 2024-05-08T05:53:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.567+0000 7f5b41400700 1 -- start start 2024-05-08T05:53:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.567+0000 7f5b41400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b3c0756d0 0x7f5b3c19bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.567+0000 7f5b41400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b3c075ff0 0x7f5b3c19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.568+0000 7f5b41400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b3c10fed0 0x7f5b3c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.568+0000 7f5b41400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1158f0 con 0x7f5b3c0756d0 2024-05-08T05:53:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.568+0000 7f5b41400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c115770 con 0x7f5b3c10fed0 2024-05-08T05:53:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.568+0000 7f5b41400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c115a70 con 0x7f5b3c075ff0 2024-05-08T05:53:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.568+0000 7f5b3a800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b3c0756d0 0x7f5b3c19bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.568+0000 7f5b3a800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b3c0756d0 0x7f5b3c19bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51698/0 (socket says 172.21.15.112:51698) 2024-05-08T05:53:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.568+0000 7f5b3a800700 1 -- 172.21.15.112:0/4121738269 learned_addr learned my addr 172.21.15.112:0/4121738269 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.568+0000 7f5b39e00700 1 --2- 172.21.15.112:0/4121738269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b3c075ff0 0x7f5b3c19c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.568+0000 7f5b3b200700 1 --2- 172.21.15.112:0/4121738269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b3c10fed0 0x7f5b3c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:10.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.568+0000 7f5b3a800700 1 -- 172.21.15.112:0/4121738269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b3c075ff0 msgr2=0x7f5b3c19c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:10.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.568+0000 7f5b3a800700 1 --2- 172.21.15.112:0/4121738269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b3c075ff0 0x7f5b3c19c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.568+0000 7f5b3a800700 1 -- 172.21.15.112:0/4121738269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b3c10fed0 msgr2=0x7f5b3c1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:10.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.568+0000 7f5b3a800700 1 --2- 172.21.15.112:0/4121738269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b3c10fed0 0x7f5b3c1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.568+0000 7f5b3a800700 1 -- 172.21.15.112:0/4121738269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3c1a0db0 con 0x7f5b3c0756d0 2024-05-08T05:53:10.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.569+0000 7f5b3a800700 1 --2- 172.21.15.112:0/4121738269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b3c0756d0 0x7f5b3c19bda0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c005910 tx=0x7f5b2c00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:10.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.569+0000 7f5b33400700 1 -- 172.21.15.112:0/4121738269 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c027070 con 0x7f5b3c0756d0 2024-05-08T05:53:10.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.569+0000 7f5b33400700 1 -- 172.21.15.112:0/4121738269 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b2c022470 con 0x7f5b3c0756d0 2024-05-08T05:53:10.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.569+0000 7f5b41400700 1 -- 172.21.15.112:0/4121738269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3c1a1040 con 0x7f5b3c0756d0 2024-05-08T05:53:10.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.569+0000 7f5b33400700 1 -- 172.21.15.112:0/4121738269 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c017650 con 0x7f5b3c0756d0 2024-05-08T05:53:10.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.569+0000 7f5b41400700 1 -- 172.21.15.112:0/4121738269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3c1a8920 con 0x7f5b3c0756d0 2024-05-08T05:53:10.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.570+0000 7f5b41400700 1 -- 172.21.15.112:0/4121738269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b3c06a190 con 0x7f5b3c0756d0 2024-05-08T05:53:10.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.572+0000 7f5b33400700 1 -- 172.21.15.112:0/4121738269 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5b2c02a070 con 0x7f5b3c0756d0 2024-05-08T05:53:10.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.574+0000 7f5b33400700 1 --2- 172.21.15.112:0/4121738269 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5b200b2900 0x7f5b200b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:10.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.574+0000 7f5b33400700 1 -- 172.21.15.112:0/4121738269 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f5b2c0e2ec0 con 0x7f5b3c0756d0 2024-05-08T05:53:10.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.574+0000 7f5b39e00700 1 --2- 172.21.15.112:0/4121738269 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5b200b2900 0x7f5b200b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:10.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.575+0000 7f5b39e00700 1 --2- 172.21.15.112:0/4121738269 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5b200b2900 0x7f5b200b4dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c111bd0 tx=0x7f5b28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:10.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.580+0000 7f5b33400700 1 -- 172.21.15.112:0/4121738269 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b2c013070 con 0x7f5b3c0756d0 2024-05-08T05:53:10.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.592+0000 7fd7eee00700 1 -- 172.21.15.112:0/46507808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e8105f90 msgr2=0x7fd7e81124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:10.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.592+0000 7fd7eee00700 1 --2- 172.21.15.112:0/46507808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e8105f90 0x7fd7e81124a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009f90 tx=0x7fd7e000b330 comp rx=0 tx=0).stop 2024-05-08T05:53:10.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.592+0000 7fd7eee00700 1 -- 172.21.15.112:0/46507808 shutdown_connections 2024-05-08T05:53:10.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.592+0000 7fd7eee00700 1 --2- 172.21.15.112:0/46507808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e8105f90 0x7fd7e81124a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.592+0000 7fd7eee00700 1 --2- 172.21.15.112:0/46507808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7e810b660 0x7fd7e810ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.592+0000 7fd7eee00700 1 --2- 172.21.15.112:0/46507808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e81055d0 0x7fd7e8105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.592+0000 7fd7eee00700 1 -- 172.21.15.112:0/46507808 >> 172.21.15.112:0/46507808 conn(0x7fd7e81010c0 msgr2=0x7fd7e81034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:10.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.593+0000 7fd7eee00700 1 -- 172.21.15.112:0/46507808 shutdown_connections 2024-05-08T05:53:10.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.593+0000 7fd7eee00700 1 -- 172.21.15.112:0/46507808 wait complete. 2024-05-08T05:53:10.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.594+0000 7fd7eee00700 1 Processor -- start 2024-05-08T05:53:10.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.594+0000 7fd7eee00700 1 -- start start 2024-05-08T05:53:10.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.594+0000 7fd7eee00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7e81055d0 0x7fd7e81a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:10.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.594+0000 7fd7eee00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e8105f90 0x7fd7e81a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:10.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.594+0000 7fd7eee00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e810b660 0x7fd7e819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:10.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.594+0000 7fd7eee00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e8115800 con 0x7fd7e810b660 2024-05-08T05:53:10.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.594+0000 7fd7eee00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e8115680 con 0x7fd7e81055d0 2024-05-08T05:53:10.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.594+0000 7fd7eee00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e8115980 con 0x7fd7e8105f90 2024-05-08T05:53:10.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.594+0000 7fd7ee400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e810b660 0x7fd7e819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:10.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.594+0000 7fd7ed000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e8105f90 0x7fd7e81a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:10.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.594+0000 7fd7ed000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e8105f90 0x7fd7e81a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38106/0 (socket says 172.21.15.112:38106) 2024-05-08T05:53:10.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.594+0000 7fd7ee400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e810b660 0x7fd7e819ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51670/0 (socket says 172.21.15.112:51670) 2024-05-08T05:53:10.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.594+0000 7fd7ed000700 1 -- 172.21.15.112:0/3205805306 learned_addr learned my addr 172.21.15.112:0/3205805306 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:10.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.594+0000 7fd7eda00700 1 --2- 172.21.15.112:0/3205805306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7e81055d0 0x7fd7e81a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:10.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.595+0000 7fd7ed000700 1 -- 172.21.15.112:0/3205805306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7e81055d0 msgr2=0x7fd7e81a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:10.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.595+0000 7fd7ed000700 1 --2- 172.21.15.112:0/3205805306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7e81055d0 0x7fd7e81a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.595+0000 7fd7ed000700 1 -- 172.21.15.112:0/3205805306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e810b660 msgr2=0x7fd7e819ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:10.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.595+0000 7fd7ed000700 1 --2- 172.21.15.112:0/3205805306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e810b660 0x7fd7e819ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.595+0000 7fd7ed000700 1 -- 172.21.15.112:0/3205805306 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e819c180 con 0x7fd7e8105f90 2024-05-08T05:53:10.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.595+0000 7fd7ee400700 1 --2- 172.21.15.112:0/3205805306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e810b660 0x7fd7e819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:10.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.595+0000 7fd7ed000700 1 --2- 172.21.15.112:0/3205805306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e8105f90 0x7fd7e81a1d50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd7d400e9c0 tx=0x7fd7d400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:10.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.595+0000 7fd7e6000700 1 -- 172.21.15.112:0/3205805306 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4004a30 con 0x7fd7e8105f90 2024-05-08T05:53:10.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.595+0000 7fd7e6000700 1 -- 172.21.15.112:0/3205805306 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7d400cc70 con 0x7fd7e8105f90 2024-05-08T05:53:10.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.595+0000 7fd7eee00700 1 -- 172.21.15.112:0/3205805306 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e819c470 con 0x7fd7e8105f90 2024-05-08T05:53:10.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.595+0000 7fd7e6000700 1 -- 172.21.15.112:0/3205805306 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4005490 con 0x7fd7e8105f90 2024-05-08T05:53:10.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.595+0000 7fd7eee00700 1 -- 172.21.15.112:0/3205805306 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e81a86e0 con 0x7fd7e8105f90 2024-05-08T05:53:10.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.597+0000 7fd7eee00700 1 -- 172.21.15.112:0/3205805306 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e806a190 con 0x7fd7e8105f90 2024-05-08T05:53:10.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.599+0000 7fd7e6000700 1 -- 172.21.15.112:0/3205805306 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd7d4004bd0 con 0x7fd7e8105f90 2024-05-08T05:53:10.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.600+0000 7fd7e6000700 1 --2- 172.21.15.112:0/3205805306 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd7d00b2880 0x7fd7d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:10.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.601+0000 7fd7e6000700 1 -- 172.21.15.112:0/3205805306 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd7d40e2340 con 0x7fd7e8105f90 2024-05-08T05:53:10.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.601+0000 7fd7eda00700 1 --2- 172.21.15.112:0/3205805306 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd7d00b2880 0x7fd7d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:10.716 INFO:teuthology.orchestra.run.smithi112.stdout:283467841581 2024-05-08T05:53:10.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.601+0000 7fd7eda00700 1 --2- 172.21.15.112:0/3205805306 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd7d00b2880 0x7fd7d00b4d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd7e810a6f0 tx=0x7fd7dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:10.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.603+0000 7fd7e6000700 1 -- 172.21.15.112:0/3205805306 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd7d40a9fd0 con 0x7fd7e8105f90 2024-05-08T05:53:10.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.829+0000 7fd7eee00700 1 -- 172.21.15.112:0/3205805306 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fd7e819cf00 con 0x7fd7e8105f90 2024-05-08T05:53:10.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.830+0000 7fd7e6000700 1 -- 172.21.15.112:0/3205805306 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd7d40a98d0 con 0x7fd7e8105f90 2024-05-08T05:53:10.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.833+0000 7fd7eee00700 1 -- 172.21.15.112:0/3205805306 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd7d00b2880 msgr2=0x7fd7d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:10.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.833+0000 7fd7eee00700 1 --2- 172.21.15.112:0/3205805306 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd7d00b2880 0x7fd7d00b4d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd7e810a6f0 tx=0x7fd7dc009260 comp rx=0 tx=0).stop 2024-05-08T05:53:10.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.833+0000 7fd7eee00700 1 -- 172.21.15.112:0/3205805306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e8105f90 msgr2=0x7fd7e81a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:10.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.833+0000 7fd7eee00700 1 --2- 172.21.15.112:0/3205805306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e8105f90 0x7fd7e81a1d50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd7d400e9c0 tx=0x7fd7d400ecd0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.833+0000 7fd7eee00700 1 -- 172.21.15.112:0/3205805306 shutdown_connections 2024-05-08T05:53:10.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.833+0000 7fd7eee00700 1 --2- 172.21.15.112:0/3205805306 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd7d00b2880 0x7fd7d00b4d40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.833+0000 7fd7eee00700 1 --2- 172.21.15.112:0/3205805306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e810b660 0x7fd7e819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.833+0000 7fd7eee00700 1 --2- 172.21.15.112:0/3205805306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7e81055d0 0x7fd7e81a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.833+0000 7fd7eee00700 1 --2- 172.21.15.112:0/3205805306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e8105f90 0x7fd7e81a1d50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.833+0000 7fd7eee00700 1 -- 172.21.15.112:0/3205805306 >> 172.21.15.112:0/3205805306 conn(0x7fd7e81010c0 msgr2=0x7fd7e8102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:10.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.833+0000 7fd7eee00700 1 -- 172.21.15.112:0/3205805306 shutdown_connections 2024-05-08T05:53:10.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:08.833+0000 7fd7eee00700 1 -- 172.21.15.112:0/3205805306 wait complete. 2024-05-08T05:53:10.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.818+0000 7f5b41400700 1 -- 172.21.15.112:0/4121738269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f5b3c02e510 con 0x7f5b3c0756d0 2024-05-08T05:53:10.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.819+0000 7f5b33400700 1 -- 172.21.15.112:0/4121738269 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5b2c0aabe0 con 0x7f5b3c0756d0 2024-05-08T05:53:10.820 INFO:teuthology.orchestra.run.smithi112.stdout:158913790029 2024-05-08T05:53:10.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.822+0000 7f5b41400700 1 -- 172.21.15.112:0/4121738269 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5b200b2900 msgr2=0x7f5b200b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:10.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.822+0000 7f5b41400700 1 --2- 172.21.15.112:0/4121738269 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5b200b2900 0x7f5b200b4dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c111bd0 tx=0x7f5b28009260 comp rx=0 tx=0).stop 2024-05-08T05:53:10.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.822+0000 7f5b41400700 1 -- 172.21.15.112:0/4121738269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b3c0756d0 msgr2=0x7f5b3c19bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:10.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.822+0000 7f5b41400700 1 --2- 172.21.15.112:0/4121738269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b3c0756d0 0x7f5b3c19bda0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c005910 tx=0x7f5b2c00b670 comp rx=0 tx=0).stop 2024-05-08T05:53:10.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.822+0000 7f5b41400700 1 -- 172.21.15.112:0/4121738269 shutdown_connections 2024-05-08T05:53:10.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.822+0000 7f5b41400700 1 --2- 172.21.15.112:0/4121738269 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5b200b2900 0x7f5b200b4dc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.822+0000 7f5b41400700 1 --2- 172.21.15.112:0/4121738269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b3c10fed0 0x7f5b3c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.822+0000 7f5b41400700 1 --2- 172.21.15.112:0/4121738269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b3c0756d0 0x7f5b3c19bda0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.822+0000 7f5b41400700 1 --2- 172.21.15.112:0/4121738269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b3c075ff0 0x7f5b3c19c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:10.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.822+0000 7f5b41400700 1 -- 172.21.15.112:0/4121738269 >> 172.21.15.112:0/4121738269 conn(0x7f5b3c07a650 msgr2=0x7f5b3c100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:10.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.823+0000 7f5b41400700 1 -- 172.21.15.112:0/4121738269 shutdown_connections 2024-05-08T05:53:10.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:10.823+0000 7f5b41400700 1 -- 172.21.15.112:0/4121738269 wait complete. 2024-05-08T05:53:11.370 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005101 got 261993005107 for osd.9 2024-05-08T05:53:11.370 DEBUG:teuthology.parallel:result is None 2024-05-08T05:53:11.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:11 smithi112 ceph-mon[30003]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-05-08T05:53:11.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:11 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/4121738269' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T05:53:11.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:11 smithi162 ceph-mon[34800]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-05-08T05:53:11.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:11 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/4121738269' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T05:53:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:11 smithi150 ceph-mon[34315]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-05-08T05:53:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:11 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/4121738269' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T05:53:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.950+0000 7f4058400700 1 -- 172.21.15.112:0/4196442706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050108850 msgr2=0x7f40501103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.950+0000 7f4058400700 1 --2- 172.21.15.112:0/4196442706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050108850 0x7f40501103d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4048009920 tx=0x7f4048009c30 comp rx=0 tx=0).stop 2024-05-08T05:53:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.950+0000 7f4058400700 1 -- 172.21.15.112:0/4196442706 shutdown_connections 2024-05-08T05:53:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.950+0000 7f4058400700 1 --2- 172.21.15.112:0/4196442706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050108850 0x7f40501103d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.950+0000 7f4058400700 1 --2- 172.21.15.112:0/4196442706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4050103750 0x7f4050103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.950+0000 7f4058400700 1 --2- 172.21.15.112:0/4196442706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4050104100 0x7f4050108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.950+0000 7f4058400700 1 -- 172.21.15.112:0/4196442706 >> 172.21.15.112:0/4196442706 conn(0x7f405007a600 msgr2=0x7f405007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.950+0000 7f4058400700 1 -- 172.21.15.112:0/4196442706 shutdown_connections 2024-05-08T05:53:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.950+0000 7f4058400700 1 -- 172.21.15.112:0/4196442706 wait complete. 2024-05-08T05:53:12.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.951+0000 7f4058400700 1 Processor -- start 2024-05-08T05:53:12.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.951+0000 7f4058400700 1 -- start start 2024-05-08T05:53:12.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.952+0000 7f4058400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4050103750 0x7f405019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:12.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.952+0000 7f4058400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4050104100 0x7f405019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:12.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.952+0000 7f4058400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050108850 0x7f40501a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:12.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.952+0000 7f4058400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4050113670 con 0x7f4050103750 2024-05-08T05:53:12.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.952+0000 7f4058400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f40501134f0 con 0x7f4050108850 2024-05-08T05:53:12.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.952+0000 7f4058400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f40501137f0 con 0x7f4050104100 2024-05-08T05:53:12.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.952+0000 7f4055a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4050103750 0x7f405019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:12.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.952+0000 7f4055a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4050103750 0x7f405019bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51710/0 (socket says 172.21.15.112:51710) 2024-05-08T05:53:12.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.952+0000 7f4055a00700 1 -- 172.21.15.112:0/4070092520 learned_addr learned my addr 172.21.15.112:0/4070092520 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:12.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.952+0000 7f4055000700 1 --2- 172.21.15.112:0/4070092520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4050104100 0x7f405019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:12.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.952+0000 7f4056400700 1 --2- 172.21.15.112:0/4070092520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050108850 0x7f40501a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:12.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.953+0000 7f4056400700 1 -- 172.21.15.112:0/4070092520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4050104100 msgr2=0x7f405019c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:12.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.953+0000 7f4056400700 1 --2- 172.21.15.112:0/4070092520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4050104100 0x7f405019c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:12.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.953+0000 7f4056400700 1 -- 172.21.15.112:0/4070092520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4050103750 msgr2=0x7f405019bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:12.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.953+0000 7f4056400700 1 --2- 172.21.15.112:0/4070092520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4050103750 0x7f405019bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:12.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.953+0000 7f4056400700 1 -- 172.21.15.112:0/4070092520 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40501a0d70 con 0x7f4050108850 2024-05-08T05:53:12.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.953+0000 7f4056400700 1 --2- 172.21.15.112:0/4070092520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050108850 0x7f40501a0630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4048009230 tx=0x7f404800f950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:12.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.953+0000 7f404e000700 1 -- 172.21.15.112:0/4070092520 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048027070 con 0x7f4050108850 2024-05-08T05:53:12.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.953+0000 7f404e000700 1 -- 172.21.15.112:0/4070092520 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40480054f0 con 0x7f4050108850 2024-05-08T05:53:12.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.953+0000 7f404e000700 1 -- 172.21.15.112:0/4070092520 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40480046d0 con 0x7f4050108850 2024-05-08T05:53:12.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.953+0000 7f4058400700 1 -- 172.21.15.112:0/4070092520 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40501a1000 con 0x7f4050108850 2024-05-08T05:53:12.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.953+0000 7f4058400700 1 -- 172.21.15.112:0/4070092520 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40501a88e0 con 0x7f4050108850 2024-05-08T05:53:12.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.955+0000 7f4058400700 1 -- 172.21.15.112:0/4070092520 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f405006a190 con 0x7f4050108850 2024-05-08T05:53:12.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.957+0000 7f404e000700 1 -- 172.21.15.112:0/4070092520 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f404802a070 con 0x7f4050108850 2024-05-08T05:53:12.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.958+0000 7f404e000700 1 --2- 172.21.15.112:0/4070092520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f40380b2880 0x7f40380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:12.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.958+0000 7f404e000700 1 -- 172.21.15.112:0/4070092520 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f40480e2c90 con 0x7f4050108850 2024-05-08T05:53:12.300 INFO:teuthology.orchestra.run.smithi112.stdout:236223201337 2024-05-08T05:53:12.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.959+0000 7f4055a00700 1 --2- 172.21.15.112:0/4070092520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f40380b2880 0x7f40380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:12.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.959+0000 7f4055a00700 1 --2- 172.21.15.112:0/4070092520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f40380b2880 0x7f40380b4d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4044009fd0 tx=0x7f404400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:12.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:11.961+0000 7f404e000700 1 -- 172.21.15.112:0/4070092520 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4048013030 con 0x7f4050108850 2024-05-08T05:53:12.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:12.198+0000 7f4058400700 1 -- 172.21.15.112:0/4070092520 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f405019d0c0 con 0x7f4050108850 2024-05-08T05:53:12.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:12.199+0000 7f404e000700 1 -- 172.21.15.112:0/4070092520 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f405019d0c0 con 0x7f4050108850 2024-05-08T05:53:12.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:12.202+0000 7f4058400700 1 -- 172.21.15.112:0/4070092520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f40380b2880 msgr2=0x7f40380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:12.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:12.202+0000 7f4058400700 1 --2- 172.21.15.112:0/4070092520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f40380b2880 0x7f40380b4d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4044009fd0 tx=0x7f404400b040 comp rx=0 tx=0).stop 2024-05-08T05:53:12.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:12.202+0000 7f4058400700 1 -- 172.21.15.112:0/4070092520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050108850 msgr2=0x7f40501a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:12.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:12.202+0000 7f4058400700 1 --2- 172.21.15.112:0/4070092520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050108850 0x7f40501a0630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4048009230 tx=0x7f404800f950 comp rx=0 tx=0).stop 2024-05-08T05:53:12.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:12.203+0000 7f4058400700 1 -- 172.21.15.112:0/4070092520 shutdown_connections 2024-05-08T05:53:12.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:12.203+0000 7f4058400700 1 --2- 172.21.15.112:0/4070092520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f40380b2880 0x7f40380b4d40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:12.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:12.203+0000 7f4058400700 1 --2- 172.21.15.112:0/4070092520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050108850 0x7f40501a0630 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:12.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:12.203+0000 7f4058400700 1 --2- 172.21.15.112:0/4070092520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4050103750 0x7f405019bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:12.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:12.203+0000 7f4058400700 1 --2- 172.21.15.112:0/4070092520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4050104100 0x7f405019c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:12.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:12.203+0000 7f4058400700 1 -- 172.21.15.112:0/4070092520 >> 172.21.15.112:0/4070092520 conn(0x7f405007a600 msgr2=0x7f4050101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:12.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:12.203+0000 7f4058400700 1 -- 172.21.15.112:0/4070092520 shutdown_connections 2024-05-08T05:53:12.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:12.203+0000 7f4058400700 1 -- 172.21.15.112:0/4070092520 wait complete. 2024-05-08T05:53:12.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:12 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/4070092520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T05:53:12.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:12 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/4070092520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T05:53:12.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:12 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/4070092520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T05:53:12.879 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986251 got 133143986254 for osd.4 2024-05-08T05:53:12.880 DEBUG:teuthology.parallel:result is None 2024-05-08T05:53:13.151 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841577 got 283467841581 for osd.10 2024-05-08T05:53:13.151 DEBUG:teuthology.parallel:result is None 2024-05-08T05:53:13.453 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:13.455 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:13.461 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593790 got 184683593796 for osd.6 2024-05-08T05:53:13.461 DEBUG:teuthology.parallel:result is None 2024-05-08T05:53:13.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:13 smithi112 ceph-mon[30003]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-05-08T05:53:13.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:13 smithi162 ceph-mon[34800]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-05-08T05:53:13.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:13 smithi150 ceph-mon[34315]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-05-08T05:53:13.794 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738461 got 34359738467 for osd.0 2024-05-08T05:53:13.794 DEBUG:teuthology.parallel:result is None 2024-05-08T05:53:14.646 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790022 got 158913790029 for osd.5 2024-05-08T05:53:14.646 DEBUG:teuthology.parallel:result is None 2024-05-08T05:53:14.747 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:14.804 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201333 got 236223201337 for osd.8 2024-05-08T05:53:14.804 DEBUG:teuthology.parallel:result is None 2024-05-08T05:53:15.516 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:15 smithi112 ceph-mon[30003]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 441 B/s wr, 6 op/s 2024-05-08T05:53:15.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:15 smithi162 ceph-mon[34800]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 441 B/s wr, 6 op/s 2024-05-08T05:53:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:15 smithi150 ceph-mon[34315]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 441 B/s wr, 6 op/s 2024-05-08T05:53:17.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:17 smithi112 ceph-mon[30003]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 99 B/s wr, 6 op/s 2024-05-08T05:53:17.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:17 smithi162 ceph-mon[34800]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 99 B/s wr, 6 op/s 2024-05-08T05:53:17.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:17 smithi150 ceph-mon[34315]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 99 B/s wr, 6 op/s 2024-05-08T05:53:19.471 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:19 smithi112 ceph-mon[30003]: pgmap v381: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T05:53:19.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:19 smithi162 ceph-mon[34800]: pgmap v381: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T05:53:19.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:19 smithi150 ceph-mon[34315]: pgmap v381: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T05:53:20.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.001+0000 7f6007000700 1 -- 172.21.15.112:0/2406625482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f600006b180 msgr2=0x7f600006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:20.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.001+0000 7f6007000700 1 --2- 172.21.15.112:0/2406625482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f600006b180 0x7f600006b600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009990 tx=0x7f5fec009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.001+0000 7f6007000700 1 -- 172.21.15.112:0/2406625482 shutdown_connections 2024-05-08T05:53:20.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.001+0000 7f6007000700 1 --2- 172.21.15.112:0/2406625482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f600006bb40 0x7f6000110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.001+0000 7f6007000700 1 --2- 172.21.15.112:0/2406625482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6000109510 0x7f60001098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.001+0000 7f6007000700 1 --2- 172.21.15.112:0/2406625482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f600006b180 0x7f600006b600 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.001+0000 7f6007000700 1 -- 172.21.15.112:0/2406625482 >> 172.21.15.112:0/2406625482 conn(0x7f6000079350 msgr2=0x7f6000079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:20.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.001+0000 7f6007000700 1 -- 172.21.15.112:0/2406625482 shutdown_connections 2024-05-08T05:53:20.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.001+0000 7f6007000700 1 -- 172.21.15.112:0/2406625482 wait complete. 2024-05-08T05:53:20.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.002+0000 7f6007000700 1 Processor -- start 2024-05-08T05:53:20.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.002+0000 7f6007000700 1 -- start start 2024-05-08T05:53:20.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.003+0000 7f6007000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f600006b180 0x7f6000102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:20.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.003+0000 7f6007000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f600006bb40 0x7f60001029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:20.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.003+0000 7f6007000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6000109510 0x7f6000102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:20.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.003+0000 7f6007000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6000113620 con 0x7f600006b180 2024-05-08T05:53:20.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.003+0000 7f6007000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f60001134a0 con 0x7f6000109510 2024-05-08T05:53:20.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.003+0000 7f6007000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f60001137a0 con 0x7f600006bb40 2024-05-08T05:53:20.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.003+0000 7f5fffe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f600006b180 0x7f6000102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:20.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.003+0000 7f5fffe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f600006b180 0x7f6000102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49904/0 (socket says 172.21.15.112:49904) 2024-05-08T05:53:20.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.003+0000 7f5fffe00700 1 -- 172.21.15.112:0/1158270269 learned_addr learned my addr 172.21.15.112:0/1158270269 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:20.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.003+0000 7f5fff400700 1 --2- 172.21.15.112:0/1158270269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f600006bb40 0x7f60001029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:20.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.003+0000 7f6005000700 1 --2- 172.21.15.112:0/1158270269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6000109510 0x7f6000102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:20.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.004+0000 7f5fff400700 1 -- 172.21.15.112:0/1158270269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6000109510 msgr2=0x7f6000102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:20.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.004+0000 7f5fff400700 1 --2- 172.21.15.112:0/1158270269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6000109510 0x7f6000102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.004+0000 7f5fff400700 1 -- 172.21.15.112:0/1158270269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f600006b180 msgr2=0x7f6000102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:20.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.004+0000 7f5fff400700 1 --2- 172.21.15.112:0/1158270269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f600006b180 0x7f6000102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.004+0000 7f5fff400700 1 -- 172.21.15.112:0/1158270269 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60001a80a0 con 0x7f600006bb40 2024-05-08T05:53:20.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.004+0000 7f5fffe00700 1 --2- 172.21.15.112:0/1158270269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f600006b180 0x7f6000102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:20.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.004+0000 7f5fff400700 1 --2- 172.21.15.112:0/1158270269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f600006bb40 0x7f60001029b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0059d0 tx=0x7f5fec0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:20.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.004+0000 7f5ffcc00700 1 -- 172.21.15.112:0/1158270269 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec027070 con 0x7f600006bb40 2024-05-08T05:53:20.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.005+0000 7f6007000700 1 -- 172.21.15.112:0/1158270269 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60001a8330 con 0x7f600006bb40 2024-05-08T05:53:20.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.005+0000 7f5ffcc00700 1 -- 172.21.15.112:0/1158270269 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fec022470 con 0x7f600006bb40 2024-05-08T05:53:20.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.005+0000 7f5ffcc00700 1 -- 172.21.15.112:0/1158270269 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec017630 con 0x7f600006bb40 2024-05-08T05:53:20.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.005+0000 7f6007000700 1 -- 172.21.15.112:0/1158270269 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60001a87e0 con 0x7f600006bb40 2024-05-08T05:53:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.006+0000 7f6007000700 1 -- 172.21.15.112:0/1158270269 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f600006a190 con 0x7f600006bb40 2024-05-08T05:53:20.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.009+0000 7f5ffcc00700 1 -- 172.21.15.112:0/1158270269 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5fec02a070 con 0x7f600006bb40 2024-05-08T05:53:20.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.010+0000 7f5ffcc00700 1 --2- 172.21.15.112:0/1158270269 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5fe40b2880 0x7f5fe40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:20.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.011+0000 7f5ffcc00700 1 -- 172.21.15.112:0/1158270269 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f5fec0e3370 con 0x7f600006bb40 2024-05-08T05:53:20.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.011+0000 7f5fffe00700 1 --2- 172.21.15.112:0/1158270269 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5fe40b2880 0x7f5fe40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:20.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.011+0000 7f5fffe00700 1 --2- 172.21.15.112:0/1158270269 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5fe40b2880 0x7f5fe40b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0011550 tx=0x7f5ff0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:20.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.012+0000 7f5ffcc00700 1 -- 172.21.15.112:0/1158270269 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5fec013070 con 0x7f600006bb40 2024-05-08T05:53:20.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.877+0000 7f3c1a800700 1 -- 172.21.15.112:0/21503608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c14108850 msgr2=0x7f3c141103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:20.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.877+0000 7f3c1a800700 1 --2- 172.21.15.112:0/21503608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c14108850 0x7f3c141103d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00b0d0 tx=0x7f3c0c00b3e0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.877+0000 7f3c1a800700 1 -- 172.21.15.112:0/21503608 shutdown_connections 2024-05-08T05:53:20.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.877+0000 7f3c1a800700 1 --2- 172.21.15.112:0/21503608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c14108850 0x7f3c141103d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.877+0000 7f3c1a800700 1 --2- 172.21.15.112:0/21503608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c14103750 0x7f3c14103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.877+0000 7f3c1a800700 1 --2- 172.21.15.112:0/21503608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c14104100 0x7f3c14108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.877+0000 7f3c1a800700 1 -- 172.21.15.112:0/21503608 >> 172.21.15.112:0/21503608 conn(0x7f3c1407a600 msgr2=0x7f3c1407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.877+0000 7f3c1a800700 1 -- 172.21.15.112:0/21503608 shutdown_connections 2024-05-08T05:53:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.877+0000 7f3c1a800700 1 -- 172.21.15.112:0/21503608 wait complete. 2024-05-08T05:53:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.878+0000 7f3c1a800700 1 Processor -- start 2024-05-08T05:53:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.878+0000 7f3c1a800700 1 -- start start 2024-05-08T05:53:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c1a800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c14103750 0x7f3c141a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c1a800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c14104100 0x7f3c141a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c1a800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c14108850 0x7f3c1419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c1a800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c14113690 con 0x7f3c14108850 2024-05-08T05:53:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c1a800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3c14113510 con 0x7f3c14104100 2024-05-08T05:53:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c1a800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3c14113810 con 0x7f3c14103750 2024-05-08T05:53:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c13e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c14108850 0x7f3c1419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:20.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c13400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c14103750 0x7f3c141a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c13400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c14103750 0x7f3c141a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:44554/0 (socket says 172.21.15.112:44554) 2024-05-08T05:53:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c13400700 1 -- 172.21.15.112:0/2448099263 learned_addr learned my addr 172.21.15.112:0/2448099263 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c12a00700 1 --2- 172.21.15.112:0/2448099263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c14104100 0x7f3c141a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c13e00700 1 -- 172.21.15.112:0/2448099263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c14103750 msgr2=0x7f3c141a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c13e00700 1 --2- 172.21.15.112:0/2448099263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c14103750 0x7f3c141a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c13e00700 1 -- 172.21.15.112:0/2448099263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c14104100 msgr2=0x7f3c141a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c13e00700 1 --2- 172.21.15.112:0/2448099263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c14104100 0x7f3c141a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c13e00700 1 -- 172.21.15.112:0/2448099263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1419c240 con 0x7f3c14108850 2024-05-08T05:53:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c12a00700 1 --2- 172.21.15.112:0/2448099263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c14104100 0x7f3c141a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:53:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.879+0000 7f3c13400700 1 --2- 172.21.15.112:0/2448099263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c14103750 0x7f3c141a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.880+0000 7f3c13e00700 1 --2- 172.21.15.112:0/2448099263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c14108850 0x7f3c1419bae0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00b770 tx=0x7f3c0c0079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.880+0000 7f3c0be00700 1 -- 172.21.15.112:0/2448099263 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c01f070 con 0x7f3c14108850 2024-05-08T05:53:20.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.880+0000 7f3c0be00700 1 -- 172.21.15.112:0/2448099263 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c0c004c50 con 0x7f3c14108850 2024-05-08T05:53:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.880+0000 7f3c1a800700 1 -- 172.21.15.112:0/2448099263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1419c4d0 con 0x7f3c14108850 2024-05-08T05:53:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.880+0000 7f3c0be00700 1 -- 172.21.15.112:0/2448099263 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c01b690 con 0x7f3c14108850 2024-05-08T05:53:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.880+0000 7f3c1a800700 1 -- 172.21.15.112:0/2448099263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c141a87f0 con 0x7f3c14108850 2024-05-08T05:53:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.881+0000 7f3c1a800700 1 -- 172.21.15.112:0/2448099263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c14104be0 con 0x7f3c14108850 2024-05-08T05:53:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.884+0000 7f3c0be00700 1 -- 172.21.15.112:0/2448099263 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3c0c022070 con 0x7f3c14108850 2024-05-08T05:53:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.885+0000 7f3c0be00700 1 --2- 172.21.15.112:0/2448099263 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3bf80b2880 0x7f3bf80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.885+0000 7f3c0be00700 1 -- 172.21.15.112:0/2448099263 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f3c0c00f020 con 0x7f3c14108850 2024-05-08T05:53:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.886+0000 7f3c13400700 1 --2- 172.21.15.112:0/2448099263 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3bf80b2880 0x7f3bf80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.886+0000 7f3c13400700 1 --2- 172.21.15.112:0/2448099263 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3bf80b2880 0x7f3bf80b4d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3c04011590 tx=0x7f3c04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:19.890+0000 7f3c0be00700 1 -- 172.21.15.112:0/2448099263 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3c0c029020 con 0x7f3c14108850 2024-05-08T05:53:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.133+0000 7f3c1a800700 1 -- 172.21.15.112:0/2448099263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f3c141a89c0 con 0x7f3c14108850 2024-05-08T05:53:20.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.134+0000 7f3c0be00700 1 -- 172.21.15.112:0/2448099263 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3c0c004e00 con 0x7f3c14108850 2024-05-08T05:53:20.264 INFO:teuthology.orchestra.run.smithi112.stdout:206158430272 2024-05-08T05:53:20.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.137+0000 7f3c1a800700 1 -- 172.21.15.112:0/2448099263 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3bf80b2880 msgr2=0x7f3bf80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:20.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.137+0000 7f3c1a800700 1 --2- 172.21.15.112:0/2448099263 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3bf80b2880 0x7f3bf80b4d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3c04011590 tx=0x7f3c04009260 comp rx=0 tx=0).stop 2024-05-08T05:53:20.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.137+0000 7f3c1a800700 1 -- 172.21.15.112:0/2448099263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c14108850 msgr2=0x7f3c1419bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:20.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.137+0000 7f3c1a800700 1 --2- 172.21.15.112:0/2448099263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c14108850 0x7f3c1419bae0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00b770 tx=0x7f3c0c0079a0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.137+0000 7f3c1a800700 1 -- 172.21.15.112:0/2448099263 shutdown_connections 2024-05-08T05:53:20.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.137+0000 7f3c1a800700 1 --2- 172.21.15.112:0/2448099263 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3bf80b2880 0x7f3bf80b4d40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.137+0000 7f3c1a800700 1 --2- 172.21.15.112:0/2448099263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c14108850 0x7f3c1419bae0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.137+0000 7f3c1a800700 1 --2- 172.21.15.112:0/2448099263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c14103750 0x7f3c141a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.137+0000 7f3c1a800700 1 --2- 172.21.15.112:0/2448099263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c14104100 0x7f3c141a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.137+0000 7f3c1a800700 1 -- 172.21.15.112:0/2448099263 >> 172.21.15.112:0/2448099263 conn(0x7f3c1407a600 msgr2=0x7f3c14101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:20.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.138+0000 7f3c1a800700 1 -- 172.21.15.112:0/2448099263 shutdown_connections 2024-05-08T05:53:20.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.138+0000 7f3c1a800700 1 -- 172.21.15.112:0/2448099263 wait complete. 2024-05-08T05:53:20.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.295+0000 7f6007000700 1 -- 172.21.15.112:0/1158270269 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f600002e510 con 0x7f600006bb40 2024-05-08T05:53:20.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.295+0000 7f5ffcc00700 1 -- 172.21.15.112:0/1158270269 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5fec0ab090 con 0x7f600006bb40 2024-05-08T05:53:20.295 INFO:teuthology.orchestra.run.smithi112.stdout:309237645355 2024-05-08T05:53:20.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.298+0000 7f6007000700 1 -- 172.21.15.112:0/1158270269 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5fe40b2880 msgr2=0x7f5fe40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:20.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.298+0000 7f6007000700 1 --2- 172.21.15.112:0/1158270269 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5fe40b2880 0x7f5fe40b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0011550 tx=0x7f5ff0009260 comp rx=0 tx=0).stop 2024-05-08T05:53:20.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.298+0000 7f6007000700 1 -- 172.21.15.112:0/1158270269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f600006bb40 msgr2=0x7f60001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:20.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.298+0000 7f6007000700 1 --2- 172.21.15.112:0/1158270269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f600006bb40 0x7f60001029b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0059d0 tx=0x7f5fec0047b0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.299+0000 7f6007000700 1 -- 172.21.15.112:0/1158270269 shutdown_connections 2024-05-08T05:53:20.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.299+0000 7f6007000700 1 --2- 172.21.15.112:0/1158270269 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f5fe40b2880 0x7f5fe40b4d40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.299+0000 7f6007000700 1 --2- 172.21.15.112:0/1158270269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6000109510 0x7f6000102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.299+0000 7f6007000700 1 --2- 172.21.15.112:0/1158270269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f600006b180 0x7f6000102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.299+0000 7f6007000700 1 --2- 172.21.15.112:0/1158270269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f600006bb40 0x7f60001029b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:20.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.299+0000 7f6007000700 1 -- 172.21.15.112:0/1158270269 >> 172.21.15.112:0/1158270269 conn(0x7f6000079350 msgr2=0x7f6000101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:20.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.299+0000 7f6007000700 1 -- 172.21.15.112:0/1158270269 shutdown_connections 2024-05-08T05:53:20.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:20.299+0000 7f6007000700 1 -- 172.21.15.112:0/1158270269 wait complete. 2024-05-08T05:53:20.486 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:20 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2448099263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T05:53:20.487 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:20 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1158270269' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T05:53:20.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:20 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/2448099263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T05:53:20.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:20 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1158270269' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T05:53:20.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:20 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/2448099263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T05:53:20.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:20 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1158270269' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T05:53:21.233 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645348 got 309237645355 for osd.11 2024-05-08T05:53:21.234 DEBUG:teuthology.parallel:result is None 2024-05-08T05:53:21.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:21 smithi112 ceph-mon[30003]: pgmap v382: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-08T05:53:21.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:21 smithi150 ceph-mon[34315]: pgmap v382: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-08T05:53:21.718 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:21 smithi162 ceph-mon[34800]: pgmap v382: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-08T05:53:21.726 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430264 got 206158430272 for osd.7 2024-05-08T05:53:21.726 DEBUG:teuthology.parallel:result is None 2024-05-08T05:53:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.462+0000 7fa403600700 1 -- 172.21.15.112:0/2241144683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc109510 msgr2=0x7fa3fc1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.462+0000 7fa403600700 1 --2- 172.21.15.112:0/2241144683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc109510 0x7fa3fc1098f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009990 tx=0x7fa3f0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.462+0000 7fa403600700 1 -- 172.21.15.112:0/2241144683 shutdown_connections 2024-05-08T05:53:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.462+0000 7fa403600700 1 --2- 172.21.15.112:0/2241144683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc06bb40 0x7fa3fc110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.462+0000 7fa403600700 1 --2- 172.21.15.112:0/2241144683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc109510 0x7fa3fc1098f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.462+0000 7fa403600700 1 --2- 172.21.15.112:0/2241144683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3fc06b180 0x7fa3fc06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.462+0000 7fa403600700 1 -- 172.21.15.112:0/2241144683 >> 172.21.15.112:0/2241144683 conn(0x7fa3fc079350 msgr2=0x7fa3fc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.462+0000 7fa403600700 1 -- 172.21.15.112:0/2241144683 shutdown_connections 2024-05-08T05:53:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.462+0000 7fa403600700 1 -- 172.21.15.112:0/2241144683 wait complete. 2024-05-08T05:53:22.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.464+0000 7fa403600700 1 Processor -- start 2024-05-08T05:53:22.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.464+0000 7fa403600700 1 -- start start 2024-05-08T05:53:22.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.465+0000 7fa403600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc06b180 0x7fa3fc19bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:22.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.465+0000 7fa403600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc06bb40 0x7fa3fc19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:22.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.465+0000 7fa403600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3fc109510 0x7fa3fc1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:22.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.465+0000 7fa403600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc113690 con 0x7fa3fc06b180 2024-05-08T05:53:22.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.465+0000 7fa403600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc113510 con 0x7fa3fc109510 2024-05-08T05:53:22.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.465+0000 7fa403600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc113810 con 0x7fa3fc06bb40 2024-05-08T05:53:22.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.465+0000 7fa400c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc06b180 0x7fa3fc19bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:22.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.465+0000 7fa400c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc06b180 0x7fa3fc19bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49916/0 (socket says 172.21.15.112:49916) 2024-05-08T05:53:22.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.465+0000 7fa400c00700 1 -- 172.21.15.112:0/36688993 learned_addr learned my addr 172.21.15.112:0/36688993 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:22.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.465+0000 7fa3fbe00700 1 --2- 172.21.15.112:0/36688993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc06bb40 0x7fa3fc19c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:22.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.465+0000 7fa401600700 1 --2- 172.21.15.112:0/36688993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3fc109510 0x7fa3fc1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:22.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.466+0000 7fa400c00700 1 -- 172.21.15.112:0/36688993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc06bb40 msgr2=0x7fa3fc19c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:22.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.466+0000 7fa400c00700 1 --2- 172.21.15.112:0/36688993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc06bb40 0x7fa3fc19c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.466+0000 7fa400c00700 1 -- 172.21.15.112:0/36688993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3fc109510 msgr2=0x7fa3fc1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:22.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.466+0000 7fa400c00700 1 --2- 172.21.15.112:0/36688993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3fc109510 0x7fa3fc1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.466+0000 7fa400c00700 1 -- 172.21.15.112:0/36688993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3fc1a0db0 con 0x7fa3fc06b180 2024-05-08T05:53:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.466+0000 7fa3fbe00700 1 --2- 172.21.15.112:0/36688993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc06bb40 0x7fa3fc19c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.466+0000 7fa400c00700 1 --2- 172.21.15.112:0/36688993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc06b180 0x7fa3fc19bda0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0000c00 tx=0x7fa3f00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.466+0000 7fa3f9600700 1 -- 172.21.15.112:0/36688993 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0027070 con 0x7fa3fc06b180 2024-05-08T05:53:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.466+0000 7fa3f9600700 1 -- 172.21.15.112:0/36688993 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3f0022470 con 0x7fa3fc06b180 2024-05-08T05:53:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.466+0000 7fa403600700 1 -- 172.21.15.112:0/36688993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3fc1a1040 con 0x7fa3fc06b180 2024-05-08T05:53:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.467+0000 7fa3f9600700 1 -- 172.21.15.112:0/36688993 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0017650 con 0x7fa3fc06b180 2024-05-08T05:53:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.467+0000 7fa403600700 1 -- 172.21.15.112:0/36688993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3fc1a8920 con 0x7fa3fc06b180 2024-05-08T05:53:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.468+0000 7fa403600700 1 -- 172.21.15.112:0/36688993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3fc06a190 con 0x7fa3fc06b180 2024-05-08T05:53:22.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.470+0000 7fa3f9600700 1 -- 172.21.15.112:0/36688993 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa3f002a070 con 0x7fa3fc06b180 2024-05-08T05:53:22.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.472+0000 7fa3f9600700 1 --2- 172.21.15.112:0/36688993 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3e40b2900 0x7fa3e40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:22.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.473+0000 7fa3f9600700 1 -- 172.21.15.112:0/36688993 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fa3f00e2e80 con 0x7fa3fc06b180 2024-05-08T05:53:22.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.473+0000 7fa3fbe00700 1 --2- 172.21.15.112:0/36688993 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3e40b2900 0x7fa3e40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:22.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.473+0000 7fa3fbe00700 1 --2- 172.21.15.112:0/36688993 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3e40b2900 0x7fa3e40b4dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc104490 tx=0x7fa3e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:22.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.476+0000 7fa3f9600700 1 -- 172.21.15.112:0/36688993 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3f0013070 con 0x7fa3fc06b180 2024-05-08T05:53:22.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.720+0000 7fa403600700 1 -- 172.21.15.112:0/36688993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa3fc1a8af0 con 0x7fa3fc06b180 2024-05-08T05:53:22.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.721+0000 7fa3f9600700 1 -- 172.21.15.112:0/36688993 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa3f00aaba0 con 0x7fa3fc06b180 2024-05-08T05:53:22.721 INFO:teuthology.orchestra.run.smithi112.stdout:55834574947 2024-05-08T05:53:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.724+0000 7fa403600700 1 -- 172.21.15.112:0/36688993 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3e40b2900 msgr2=0x7fa3e40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.724+0000 7fa403600700 1 --2- 172.21.15.112:0/36688993 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3e40b2900 0x7fa3e40b4dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc104490 tx=0x7fa3e8009260 comp rx=0 tx=0).stop 2024-05-08T05:53:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.724+0000 7fa403600700 1 -- 172.21.15.112:0/36688993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc06b180 msgr2=0x7fa3fc19bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.724+0000 7fa403600700 1 --2- 172.21.15.112:0/36688993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc06b180 0x7fa3fc19bda0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0000c00 tx=0x7fa3f00057c0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.725+0000 7fa403600700 1 -- 172.21.15.112:0/36688993 shutdown_connections 2024-05-08T05:53:22.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.725+0000 7fa403600700 1 --2- 172.21.15.112:0/36688993 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3e40b2900 0x7fa3e40b4dc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.725+0000 7fa403600700 1 --2- 172.21.15.112:0/36688993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3fc109510 0x7fa3fc1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.725+0000 7fa403600700 1 --2- 172.21.15.112:0/36688993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3fc06b180 0x7fa3fc19bda0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.725+0000 7fa403600700 1 --2- 172.21.15.112:0/36688993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc06bb40 0x7fa3fc19c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.725+0000 7fa403600700 1 -- 172.21.15.112:0/36688993 >> 172.21.15.112:0/36688993 conn(0x7fa3fc079350 msgr2=0x7fa3fc10e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:22.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.725+0000 7fa403600700 1 -- 172.21.15.112:0/36688993 shutdown_connections 2024-05-08T05:53:22.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:22.725+0000 7fa403600700 1 -- 172.21.15.112:0/36688993 wait complete. 2024-05-08T05:53:23.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:23 smithi150 ceph-mon[34315]: pgmap v383: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:23.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:23 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:23.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:23 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:23.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:23 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/36688993' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T05:53:23.509 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:23 smithi112 ceph-mon[30003]: pgmap v383: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:23.509 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:23 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:23.509 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:23 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:23.509 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:23 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/36688993' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T05:53:23.561 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574941 got 55834574947 for osd.1 2024-05-08T05:53:23.561 DEBUG:teuthology.parallel:result is None 2024-05-08T05:53:23.561 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T05:53:23.562 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:53:23.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:23 smithi162 ceph-mon[34800]: pgmap v383: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:23.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:23 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:23.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:23 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:23.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:23 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/36688993' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T05:53:23.990 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:24.338 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:53:24.339 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:53:24.339 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:24.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:53:24.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:53:24.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:24.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:53:24.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:53:24.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:53:25.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:25 smithi112 ceph-mon[30003]: pgmap v384: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:53:25.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:25 smithi150 ceph-mon[34315]: pgmap v384: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:53:25.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:25 smithi162 ceph-mon[34800]: pgmap v384: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:53:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.169+0000 7fc0f0c00700 1 -- 172.21.15.112:0/384590848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0ec109ad0 msgr2=0x7fc0ec109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.169+0000 7fc0f0c00700 1 --2- 172.21.15.112:0/384590848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0ec109ad0 0x7fc0ec109f50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009990 tx=0x7fc0d8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.169+0000 7fc0f0c00700 1 -- 172.21.15.112:0/384590848 shutdown_connections 2024-05-08T05:53:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.169+0000 7fc0f0c00700 1 --2- 172.21.15.112:0/384590848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0ec10a490 0x7fc0ec1169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.169+0000 7fc0f0c00700 1 --2- 172.21.15.112:0/384590848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0ec10fb60 0x7fc0ec10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.169+0000 7fc0f0c00700 1 --2- 172.21.15.112:0/384590848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0ec109ad0 0x7fc0ec109f50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.169+0000 7fc0f0c00700 1 -- 172.21.15.112:0/384590848 >> 172.21.15.112:0/384590848 conn(0x7fc0ec076c70 msgr2=0x7fc0ec1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.169+0000 7fc0f0c00700 1 -- 172.21.15.112:0/384590848 shutdown_connections 2024-05-08T05:53:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.169+0000 7fc0f0c00700 1 -- 172.21.15.112:0/384590848 wait complete. 2024-05-08T05:53:26.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.170+0000 7fc0f0c00700 1 Processor -- start 2024-05-08T05:53:26.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.170+0000 7fc0f0c00700 1 -- start start 2024-05-08T05:53:26.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.171+0000 7fc0f0c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0ec109ad0 0x7fc0ec1a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:26.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.171+0000 7fc0f0c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0ec10a490 0x7fc0ec1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:26.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.171+0000 7fc0f0c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0ec10fb60 0x7fc0ec1a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.171+0000 7fc0f0c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec119d20 con 0x7fc0ec10a490 2024-05-08T05:53:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.171+0000 7fc0f0c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec119ba0 con 0x7fc0ec10fb60 2024-05-08T05:53:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.171+0000 7fc0f0c00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec119ea0 con 0x7fc0ec109ad0 2024-05-08T05:53:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.171+0000 7fc0e9400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0ec10a490 0x7fc0ec1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.171+0000 7fc0e9400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0ec10a490 0x7fc0ec1a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36620/0 (socket says 172.21.15.112:36620) 2024-05-08T05:53:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.171+0000 7fc0e9400700 1 -- 172.21.15.112:0/1205803645 learned_addr learned my addr 172.21.15.112:0/1205803645 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.171+0000 7fc0e9e00700 1 --2- 172.21.15.112:0/1205803645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0ec109ad0 0x7fc0ec1a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.172+0000 7fc0ea800700 1 --2- 172.21.15.112:0/1205803645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0ec10fb60 0x7fc0ec1a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.172+0000 7fc0e9e00700 1 -- 172.21.15.112:0/1205803645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0ec10fb60 msgr2=0x7fc0ec1a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.172+0000 7fc0e9e00700 1 --2- 172.21.15.112:0/1205803645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0ec10fb60 0x7fc0ec1a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.172+0000 7fc0e9e00700 1 -- 172.21.15.112:0/1205803645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0ec10a490 msgr2=0x7fc0ec1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.172+0000 7fc0e9e00700 1 --2- 172.21.15.112:0/1205803645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0ec10a490 0x7fc0ec1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.172+0000 7fc0e9e00700 1 -- 172.21.15.112:0/1205803645 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec1a51c0 con 0x7fc0ec109ad0 2024-05-08T05:53:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.172+0000 7fc0e9400700 1 --2- 172.21.15.112:0/1205803645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0ec10a490 0x7fc0ec1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.172+0000 7fc0e9e00700 1 --2- 172.21.15.112:0/1205803645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0ec109ad0 0x7fc0ec1a01b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00e9f0 tx=0x7fc0dc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:26.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.172+0000 7fc0e2a00700 1 -- 172.21.15.112:0/1205803645 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc004220 con 0x7fc0ec109ad0 2024-05-08T05:53:26.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.172+0000 7fc0e2a00700 1 -- 172.21.15.112:0/1205803645 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0dc0043c0 con 0x7fc0ec109ad0 2024-05-08T05:53:26.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.172+0000 7fc0f0c00700 1 -- 172.21.15.112:0/1205803645 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ec1a54b0 con 0x7fc0ec109ad0 2024-05-08T05:53:26.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.173+0000 7fc0e2a00700 1 -- 172.21.15.112:0/1205803645 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc005020 con 0x7fc0ec109ad0 2024-05-08T05:53:26.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.173+0000 7fc0f0c00700 1 -- 172.21.15.112:0/1205803645 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ec1acd30 con 0x7fc0ec109ad0 2024-05-08T05:53:26.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.174+0000 7fc0f0c00700 1 -- 172.21.15.112:0/1205803645 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ec06a190 con 0x7fc0ec109ad0 2024-05-08T05:53:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.176+0000 7fc0e2a00700 1 -- 172.21.15.112:0/1205803645 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc0dc004750 con 0x7fc0ec109ad0 2024-05-08T05:53:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.177+0000 7fc0e2a00700 1 --2- 172.21.15.112:0/1205803645 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc0d00b2900 0x7fc0d00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.178+0000 7fc0e2a00700 1 -- 172.21.15.112:0/1205803645 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fc0dc0e2130 con 0x7fc0ec109ad0 2024-05-08T05:53:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.178+0000 7fc0e9400700 1 --2- 172.21.15.112:0/1205803645 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc0d00b2900 0x7fc0d00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.178+0000 7fc0e9400700 1 --2- 172.21.15.112:0/1205803645 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc0d00b2900 0x7fc0d00b4dc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec078b70 tx=0x7fc0d800b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:26.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.182+0000 7fc0e2a00700 1 -- 172.21.15.112:0/1205803645 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc0dc0a9dc0 con 0x7fc0ec109ad0 2024-05-08T05:53:26.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.412+0000 7fc0f0c00700 1 -- 172.21.15.112:0/1205803645 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0ec0638e0 con 0x7fc0d00b2900 2024-05-08T05:53:26.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.425+0000 7fc0e2a00700 1 -- 172.21.15.112:0/1205803645 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258828 (secure 0 0 0) 0x7fc0ec0638e0 con 0x7fc0d00b2900 2024-05-08T05:53:26.429 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:53:26.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.431+0000 7fc0f0c00700 1 -- 172.21.15.112:0/1205803645 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc0d00b2900 msgr2=0x7fc0d00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:26.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.431+0000 7fc0f0c00700 1 --2- 172.21.15.112:0/1205803645 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc0d00b2900 0x7fc0d00b4dc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec078b70 tx=0x7fc0d800b280 comp rx=0 tx=0).stop 2024-05-08T05:53:26.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.431+0000 7fc0f0c00700 1 -- 172.21.15.112:0/1205803645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0ec109ad0 msgr2=0x7fc0ec1a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:26.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.431+0000 7fc0f0c00700 1 --2- 172.21.15.112:0/1205803645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0ec109ad0 0x7fc0ec1a01b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00e9f0 tx=0x7fc0dc00ed00 comp rx=0 tx=0).stop 2024-05-08T05:53:26.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.431+0000 7fc0f0c00700 1 -- 172.21.15.112:0/1205803645 shutdown_connections 2024-05-08T05:53:26.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.431+0000 7fc0f0c00700 1 --2- 172.21.15.112:0/1205803645 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc0d00b2900 0x7fc0d00b4dc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:26.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.431+0000 7fc0f0c00700 1 --2- 172.21.15.112:0/1205803645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0ec10fb60 0x7fc0ec1a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:26.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.431+0000 7fc0f0c00700 1 --2- 172.21.15.112:0/1205803645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0ec109ad0 0x7fc0ec1a01b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:26.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.431+0000 7fc0f0c00700 1 --2- 172.21.15.112:0/1205803645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0ec10a490 0x7fc0ec1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:26.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.431+0000 7fc0f0c00700 1 -- 172.21.15.112:0/1205803645 >> 172.21.15.112:0/1205803645 conn(0x7fc0ec076c70 msgr2=0x7fc0ec077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:26.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.431+0000 7fc0f0c00700 1 -- 172.21.15.112:0/1205803645 shutdown_connections 2024-05-08T05:53:26.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:26.431+0000 7fc0f0c00700 1 -- 172.21.15.112:0/1205803645 wait complete. 2024-05-08T05:53:26.432 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-05-08T05:53:27.236 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-05-08T05:53:25.525047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656743,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":803,"num_read_kb":496,"num_write":297,"num_write_kb":2269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951974,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":740,"ondisk_log_size":740,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366404,"kb_used_data":13020,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575420,"statfs":{"total":1151940427776,"available":1151565230080,"internally_reserved":0,"allocated":13332480,"data_stored":7393437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004217"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.580018+0000","last_change":"2024-05-08T05:51:04.320319+0000","last_active":"2024-05-08T05:53:07.580018+0000","last_peered":"2024-05-08T05:53:07.580018+0000","last_clean":"2024-05-08T05:53:07.580018+0000","last_became_active":"2024-05-08T05:51:04.319961+0000","last_became_peered":"2024-05-08T05:51:04.319961+0000","last_unstale":"2024-05-08T05:53:07.580018+0000","last_undegraded":"2024-05-08T05:53:07.580018+0000","last_fullsized":"2024-05-08T05:53:07.580018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:32.463261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032466+0000","last_change":"2024-05-08T05:51:04.315531+0000","last_active":"2024-05-08T05:53:07.032466+0000","last_peered":"2024-05-08T05:53:07.032466+0000","last_clean":"2024-05-08T05:53:07.032466+0000","last_became_active":"2024-05-08T05:51:04.314116+0000","last_became_peered":"2024-05-08T05:51:04.314116+0000","last_unstale":"2024-05-08T05:53:07.032466+0000","last_undegraded":"2024-05-08T05:53:07.032466+0000","last_fullsized":"2024-05-08T05:53:07.032466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:12:03.505888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032236+0000","last_change":"2024-05-08T05:51:04.315826+0000","last_active":"2024-05-08T05:53:07.032236+0000","last_peered":"2024-05-08T05:53:07.032236+0000","last_clean":"2024-05-08T05:53:07.032236+0000","last_became_active":"2024-05-08T05:51:04.315460+0000","last_became_peered":"2024-05-08T05:51:04.315460+0000","last_unstale":"2024-05-08T05:53:07.032236+0000","last_undegraded":"2024-05-08T05:53:07.032236+0000","last_fullsized":"2024-05-08T05:53:07.032236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:22:20.412368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715439+0000","last_change":"2024-05-08T05:51:04.297401+0000","last_active":"2024-05-08T05:53:05.715439+0000","last_peered":"2024-05-08T05:53:05.715439+0000","last_clean":"2024-05-08T05:53:05.715439+0000","last_became_active":"2024-05-08T05:51:04.297061+0000","last_became_peered":"2024-05-08T05:51:04.297061+0000","last_unstale":"2024-05-08T05:53:05.715439+0000","last_undegraded":"2024-05-08T05:53:05.715439+0000","last_fullsized":"2024-05-08T05:53:05.715439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:27.967517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268733+0000","last_change":"2024-05-08T05:51:04.315069+0000","last_active":"2024-05-08T05:53:05.268733+0000","last_peered":"2024-05-08T05:53:05.268733+0000","last_clean":"2024-05-08T05:53:05.268733+0000","last_became_active":"2024-05-08T05:51:04.314513+0000","last_became_peered":"2024-05-08T05:51:04.314513+0000","last_unstale":"2024-05-08T05:53:05.268733+0000","last_undegraded":"2024-05-08T05:53:05.268733+0000","last_fullsized":"2024-05-08T05:53:05.268733+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:02.520130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265149+0000","last_change":"2024-05-08T05:51:04.302851+0000","last_active":"2024-05-08T05:53:05.265149+0000","last_peered":"2024-05-08T05:53:05.265149+0000","last_clean":"2024-05-08T05:53:05.265149+0000","last_became_active":"2024-05-08T05:51:04.302256+0000","last_became_peered":"2024-05-08T05:51:04.302256+0000","last_unstale":"2024-05-08T05:53:05.265149+0000","last_undegraded":"2024-05-08T05:53:05.265149+0000","last_fullsized":"2024-05-08T05:53:05.265149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:45.587319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139237+0000","last_change":"2024-05-08T05:51:04.308674+0000","last_active":"2024-05-08T05:53:07.139237+0000","last_peered":"2024-05-08T05:53:07.139237+0000","last_clean":"2024-05-08T05:53:07.139237+0000","last_became_active":"2024-05-08T05:51:04.308295+0000","last_became_peered":"2024-05-08T05:51:04.308295+0000","last_unstale":"2024-05-08T05:53:07.139237+0000","last_undegraded":"2024-05-08T05:53:07.139237+0000","last_fullsized":"2024-05-08T05:53:07.139237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:59:17.561655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571478+0000","last_change":"2024-05-08T05:51:04.314511+0000","last_active":"2024-05-08T05:53:05.571478+0000","last_peered":"2024-05-08T05:53:05.571478+0000","last_clean":"2024-05-08T05:53:05.571478+0000","last_became_active":"2024-05-08T05:51:04.314082+0000","last_became_peered":"2024-05-08T05:51:04.314082+0000","last_unstale":"2024-05-08T05:53:05.571478+0000","last_undegraded":"2024-05-08T05:53:05.571478+0000","last_fullsized":"2024-05-08T05:53:05.571478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:24.182391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269733+0000","last_change":"2024-05-08T05:51:04.293745+0000","last_active":"2024-05-08T05:53:05.269733+0000","last_peered":"2024-05-08T05:53:05.269733+0000","last_clean":"2024-05-08T05:53:05.269733+0000","last_became_active":"2024-05-08T05:51:04.293334+0000","last_became_peered":"2024-05-08T05:51:04.293334+0000","last_unstale":"2024-05-08T05:53:05.269733+0000","last_undegraded":"2024-05-08T05:53:05.269733+0000","last_fullsized":"2024-05-08T05:53:05.269733+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:16.576967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265435+0000","last_change":"2024-05-08T05:51:04.297800+0000","last_active":"2024-05-08T05:53:05.265435+0000","last_peered":"2024-05-08T05:53:05.265435+0000","last_clean":"2024-05-08T05:53:05.265435+0000","last_became_active":"2024-05-08T05:51:04.297349+0000","last_became_peered":"2024-05-08T05:51:04.297349+0000","last_unstale":"2024-05-08T05:53:05.265435+0000","last_undegraded":"2024-05-08T05:53:05.265435+0000","last_fullsized":"2024-05-08T05:53:05.265435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:55.784512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139575+0000","last_change":"2024-05-08T05:51:04.297997+0000","last_active":"2024-05-08T05:53:07.139575+0000","last_peered":"2024-05-08T05:53:07.139575+0000","last_clean":"2024-05-08T05:53:07.139575+0000","last_became_active":"2024-05-08T05:51:04.297516+0000","last_became_peered":"2024-05-08T05:51:04.297516+0000","last_unstale":"2024-05-08T05:53:07.139575+0000","last_undegraded":"2024-05-08T05:53:07.139575+0000","last_fullsized":"2024-05-08T05:53:07.139575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:25.991555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715431+0000","last_change":"2024-05-08T05:51:04.295610+0000","last_active":"2024-05-08T05:53:05.715431+0000","last_peered":"2024-05-08T05:53:05.715431+0000","last_clean":"2024-05-08T05:53:05.715431+0000","last_became_active":"2024-05-08T05:51:04.294553+0000","last_became_peered":"2024-05-08T05:51:04.294553+0000","last_unstale":"2024-05-08T05:53:05.715431+0000","last_undegraded":"2024-05-08T05:53:05.715431+0000","last_fullsized":"2024-05-08T05:53:05.715431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:40:14.492057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820576+0000","last_change":"2024-05-08T05:51:04.303879+0000","last_active":"2024-05-08T05:53:05.820576+0000","last_peered":"2024-05-08T05:53:05.820576+0000","last_clean":"2024-05-08T05:53:05.820576+0000","last_became_active":"2024-05-08T05:51:04.303423+0000","last_became_peered":"2024-05-08T05:51:04.303423+0000","last_unstale":"2024-05-08T05:53:05.820576+0000","last_undegraded":"2024-05-08T05:53:05.820576+0000","last_fullsized":"2024-05-08T05:53:05.820576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:50:55.905858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.031669+0000","last_change":"2024-05-08T05:51:04.309818+0000","last_active":"2024-05-08T05:53:07.031669+0000","last_peered":"2024-05-08T05:53:07.031669+0000","last_clean":"2024-05-08T05:53:07.031669+0000","last_became_active":"2024-05-08T05:51:04.309434+0000","last_became_peered":"2024-05-08T05:51:04.309434+0000","last_unstale":"2024-05-08T05:53:07.031669+0000","last_undegraded":"2024-05-08T05:53:07.031669+0000","last_fullsized":"2024-05-08T05:53:07.031669+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:55.832239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.031399+0000","last_change":"2024-05-08T05:51:04.316708+0000","last_active":"2024-05-08T05:53:07.031399+0000","last_peered":"2024-05-08T05:53:07.031399+0000","last_clean":"2024-05-08T05:53:07.031399+0000","last_became_active":"2024-05-08T05:51:04.316525+0000","last_became_peered":"2024-05-08T05:51:04.316525+0000","last_unstale":"2024-05-08T05:53:07.031399+0000","last_undegraded":"2024-05-08T05:53:07.031399+0000","last_fullsized":"2024-05-08T05:53:07.031399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:33:38.246611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269307+0000","last_change":"2024-05-08T05:51:04.314780+0000","last_active":"2024-05-08T05:53:05.269307+0000","last_peered":"2024-05-08T05:53:05.269307+0000","last_clean":"2024-05-08T05:53:05.269307+0000","last_became_active":"2024-05-08T05:51:04.314406+0000","last_became_peered":"2024-05-08T05:51:04.314406+0000","last_unstale":"2024-05-08T05:53:05.269307+0000","last_undegraded":"2024-05-08T05:53:05.269307+0000","last_fullsized":"2024-05-08T05:53:05.269307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:39:26.926238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579947+0000","last_change":"2024-05-08T05:51:04.319425+0000","last_active":"2024-05-08T05:53:07.579947+0000","last_peered":"2024-05-08T05:53:07.579947+0000","last_clean":"2024-05-08T05:53:07.579947+0000","last_became_active":"2024-05-08T05:51:04.319106+0000","last_became_peered":"2024-05-08T05:51:04.319106+0000","last_unstale":"2024-05-08T05:53:07.579947+0000","last_undegraded":"2024-05-08T05:53:07.579947+0000","last_fullsized":"2024-05-08T05:53:07.579947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:59:54.023162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267215+0000","last_change":"2024-05-08T05:51:04.303547+0000","last_active":"2024-05-08T05:53:05.267215+0000","last_peered":"2024-05-08T05:53:05.267215+0000","last_clean":"2024-05-08T05:53:05.267215+0000","last_became_active":"2024-05-08T05:51:04.303182+0000","last_became_peered":"2024-05-08T05:51:04.303182+0000","last_unstale":"2024-05-08T05:53:05.267215+0000","last_undegraded":"2024-05-08T05:53:05.267215+0000","last_fullsized":"2024-05-08T05:53:05.267215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:11:59.624329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505589+0000","last_change":"2024-05-08T05:51:04.315674+0000","last_active":"2024-05-08T05:53:05.505589+0000","last_peered":"2024-05-08T05:53:05.505589+0000","last_clean":"2024-05-08T05:53:05.505589+0000","last_became_active":"2024-05-08T05:51:04.314911+0000","last_became_peered":"2024-05-08T05:51:04.314911+0000","last_unstale":"2024-05-08T05:53:05.505589+0000","last_undegraded":"2024-05-08T05:53:05.505589+0000","last_fullsized":"2024-05-08T05:53:05.505589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:49.501508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267963+0000","last_change":"2024-05-08T05:51:00.273554+0000","last_active":"2024-05-08T05:53:05.267963+0000","last_peered":"2024-05-08T05:53:05.267963+0000","last_clean":"2024-05-08T05:53:05.267963+0000","last_became_active":"2024-05-08T05:51:00.272858+0000","last_became_peered":"2024-05-08T05:51:00.272858+0000","last_unstale":"2024-05-08T05:53:05.267963+0000","last_undegraded":"2024-05-08T05:53:05.267963+0000","last_fullsized":"2024-05-08T05:53:05.267963+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:47:37.418887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572441+0000","last_change":"2024-05-08T05:51:04.306392+0000","last_active":"2024-05-08T05:53:05.572441+0000","last_peered":"2024-05-08T05:53:05.572441+0000","last_clean":"2024-05-08T05:53:05.572441+0000","last_became_active":"2024-05-08T05:51:04.304248+0000","last_became_peered":"2024-05-08T05:51:04.304248+0000","last_unstale":"2024-05-08T05:53:05.572441+0000","last_undegraded":"2024-05-08T05:53:05.572441+0000","last_fullsized":"2024-05-08T05:53:05.572441+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:49:26.034883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572774+0000","last_change":"2024-05-08T05:51:01.187292+0000","last_active":"2024-05-08T05:53:05.572774+0000","last_peered":"2024-05-08T05:53:05.572774+0000","last_clean":"2024-05-08T05:53:05.572774+0000","last_became_active":"2024-05-08T05:51:01.185948+0000","last_became_peered":"2024-05-08T05:51:01.185948+0000","last_unstale":"2024-05-08T05:53:05.572774+0000","last_undegraded":"2024-05-08T05:53:05.572774+0000","last_fullsized":"2024-05-08T05:53:05.572774+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:02.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032756+0000","last_change":"2024-05-08T05:51:04.318439+0000","last_active":"2024-05-08T05:53:07.032756+0000","last_peered":"2024-05-08T05:53:07.032756+0000","last_clean":"2024-05-08T05:53:07.032756+0000","last_became_active":"2024-05-08T05:51:04.317722+0000","last_became_peered":"2024-05-08T05:51:04.317722+0000","last_unstale":"2024-05-08T05:53:07.032756+0000","last_undegraded":"2024-05-08T05:53:07.032756+0000","last_fullsized":"2024-05-08T05:53:07.032756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:36:43.648643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269243+0000","last_change":"2024-05-08T05:51:00.269684+0000","last_active":"2024-05-08T05:53:05.269243+0000","last_peered":"2024-05-08T05:53:05.269243+0000","last_clean":"2024-05-08T05:53:05.269243+0000","last_became_active":"2024-05-08T05:51:00.269329+0000","last_became_peered":"2024-05-08T05:51:00.269329+0000","last_unstale":"2024-05-08T05:53:05.269243+0000","last_undegraded":"2024-05-08T05:53:05.269243+0000","last_fullsized":"2024-05-08T05:53:05.269243+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:27:10.422994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269350+0000","last_change":"2024-05-08T05:51:04.308048+0000","last_active":"2024-05-08T05:53:05.269350+0000","last_peered":"2024-05-08T05:53:05.269350+0000","last_clean":"2024-05-08T05:53:05.269350+0000","last_became_active":"2024-05-08T05:51:04.307550+0000","last_became_peered":"2024-05-08T05:51:04.307550+0000","last_unstale":"2024-05-08T05:53:05.269350+0000","last_undegraded":"2024-05-08T05:53:05.269350+0000","last_fullsized":"2024-05-08T05:53:05.269350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:13.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268974+0000","last_change":"2024-05-08T05:51:00.268142+0000","last_active":"2024-05-08T05:53:05.268974+0000","last_peered":"2024-05-08T05:53:05.268974+0000","last_clean":"2024-05-08T05:53:05.268974+0000","last_became_active":"2024-05-08T05:51:00.267756+0000","last_became_peered":"2024-05-08T05:51:00.267756+0000","last_unstale":"2024-05-08T05:53:05.268974+0000","last_undegraded":"2024-05-08T05:53:05.268974+0000","last_fullsized":"2024-05-08T05:53:05.268974+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:01.439680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270137+0000","last_change":"2024-05-08T05:51:04.292341+0000","last_active":"2024-05-08T05:53:05.270137+0000","last_peered":"2024-05-08T05:53:05.270137+0000","last_clean":"2024-05-08T05:53:05.270137+0000","last_became_active":"2024-05-08T05:51:04.291086+0000","last_became_peered":"2024-05-08T05:51:04.291086+0000","last_unstale":"2024-05-08T05:53:05.270137+0000","last_undegraded":"2024-05-08T05:53:05.270137+0000","last_fullsized":"2024-05-08T05:53:05.270137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:47.108100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":154,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:22.820547+0000","last_change":"2024-05-08T05:51:00.268100+0000","last_active":"2024-05-08T05:53:22.820547+0000","last_peered":"2024-05-08T05:53:22.820547+0000","last_clean":"2024-05-08T05:53:22.820547+0000","last_became_active":"2024-05-08T05:51:00.267678+0000","last_became_peered":"2024-05-08T05:51:00.267678+0000","last_unstale":"2024-05-08T05:53:22.820547+0000","last_undegraded":"2024-05-08T05:53:22.820547+0000","last_fullsized":"2024-05-08T05:53:22.820547+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:11:31.731548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268695+0000","last_change":"2024-05-08T05:51:04.303872+0000","last_active":"2024-05-08T05:53:05.268695+0000","last_peered":"2024-05-08T05:53:05.268695+0000","last_clean":"2024-05-08T05:53:05.268695+0000","last_became_active":"2024-05-08T05:51:04.303456+0000","last_became_peered":"2024-05-08T05:51:04.303456+0000","last_unstale":"2024-05-08T05:53:05.268695+0000","last_undegraded":"2024-05-08T05:53:05.268695+0000","last_fullsized":"2024-05-08T05:53:05.268695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:42:05.098488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265098+0000","last_change":"2024-05-08T05:51:00.275657+0000","last_active":"2024-05-08T05:53:05.265098+0000","last_peered":"2024-05-08T05:53:05.265098+0000","last_clean":"2024-05-08T05:53:05.265098+0000","last_became_active":"2024-05-08T05:51:00.275254+0000","last_became_peered":"2024-05-08T05:51:00.275254+0000","last_unstale":"2024-05-08T05:53:05.265098+0000","last_undegraded":"2024-05-08T05:53:05.265098+0000","last_fullsized":"2024-05-08T05:53:05.265098+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:28.555698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270045+0000","last_change":"2024-05-08T05:51:04.293791+0000","last_active":"2024-05-08T05:53:05.270045+0000","last_peered":"2024-05-08T05:53:05.270045+0000","last_clean":"2024-05-08T05:53:05.270045+0000","last_became_active":"2024-05-08T05:51:04.293405+0000","last_became_peered":"2024-05-08T05:51:04.293405+0000","last_unstale":"2024-05-08T05:53:05.270045+0000","last_undegraded":"2024-05-08T05:53:05.270045+0000","last_fullsized":"2024-05-08T05:53:05.270045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:28.064452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820087+0000","last_change":"2024-05-08T05:51:00.280295+0000","last_active":"2024-05-08T05:53:05.820087+0000","last_peered":"2024-05-08T05:53:05.820087+0000","last_clean":"2024-05-08T05:53:05.820087+0000","last_became_active":"2024-05-08T05:51:00.279721+0000","last_became_peered":"2024-05-08T05:51:00.279721+0000","last_unstale":"2024-05-08T05:53:05.820087+0000","last_undegraded":"2024-05-08T05:53:05.820087+0000","last_fullsized":"2024-05-08T05:53:05.820087+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:06:38.064872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571660+0000","last_change":"2024-05-08T05:51:04.307546+0000","last_active":"2024-05-08T05:53:05.571660+0000","last_peered":"2024-05-08T05:53:05.571660+0000","last_clean":"2024-05-08T05:53:05.571660+0000","last_became_active":"2024-05-08T05:51:04.307164+0000","last_became_peered":"2024-05-08T05:51:04.307164+0000","last_unstale":"2024-05-08T05:53:05.571660+0000","last_undegraded":"2024-05-08T05:53:05.571660+0000","last_fullsized":"2024-05-08T05:53:05.571660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:36:02.269040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269777+0000","last_change":"2024-05-08T05:51:00.267987+0000","last_active":"2024-05-08T05:53:05.269777+0000","last_peered":"2024-05-08T05:53:05.269777+0000","last_clean":"2024-05-08T05:53:05.269777+0000","last_became_active":"2024-05-08T05:51:00.267633+0000","last_became_peered":"2024-05-08T05:51:00.267633+0000","last_unstale":"2024-05-08T05:53:05.269777+0000","last_undegraded":"2024-05-08T05:53:05.269777+0000","last_fullsized":"2024-05-08T05:53:05.269777+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:13.034300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.266338+0000","last_change":"2024-05-08T05:51:04.311422+0000","last_active":"2024-05-08T05:53:05.266338+0000","last_peered":"2024-05-08T05:53:05.266338+0000","last_clean":"2024-05-08T05:53:05.266338+0000","last_became_active":"2024-05-08T05:51:04.311052+0000","last_became_peered":"2024-05-08T05:51:04.311052+0000","last_unstale":"2024-05-08T05:53:05.266338+0000","last_undegraded":"2024-05-08T05:53:05.266338+0000","last_fullsized":"2024-05-08T05:53:05.266338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:47.790583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":150,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:23.821384+0000","last_change":"2024-05-08T05:51:00.274252+0000","last_active":"2024-05-08T05:53:23.821384+0000","last_peered":"2024-05-08T05:53:23.821384+0000","last_clean":"2024-05-08T05:53:23.821384+0000","last_became_active":"2024-05-08T05:51:00.273942+0000","last_became_peered":"2024-05-08T05:51:00.273942+0000","last_unstale":"2024-05-08T05:53:23.821384+0000","last_undegraded":"2024-05-08T05:53:23.821384+0000","last_fullsized":"2024-05-08T05:53:23.821384+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:36.778032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269640+0000","last_change":"2024-05-08T05:51:04.296414+0000","last_active":"2024-05-08T05:53:05.269640+0000","last_peered":"2024-05-08T05:53:05.269640+0000","last_clean":"2024-05-08T05:53:05.269640+0000","last_became_active":"2024-05-08T05:51:04.296015+0000","last_became_peered":"2024-05-08T05:51:04.296015+0000","last_unstale":"2024-05-08T05:53:05.269640+0000","last_undegraded":"2024-05-08T05:53:05.269640+0000","last_fullsized":"2024-05-08T05:53:05.269640+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:48:15.609385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139516+0000","last_change":"2024-05-08T05:51:00.268234+0000","last_active":"2024-05-08T05:53:07.139516+0000","last_peered":"2024-05-08T05:53:07.139516+0000","last_clean":"2024-05-08T05:53:07.139516+0000","last_became_active":"2024-05-08T05:51:00.267846+0000","last_became_peered":"2024-05-08T05:51:00.267846+0000","last_unstale":"2024-05-08T05:53:07.139516+0000","last_undegraded":"2024-05-08T05:53:07.139516+0000","last_fullsized":"2024-05-08T05:53:07.139516+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:26.919004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032562+0000","last_change":"2024-05-08T05:51:04.318215+0000","last_active":"2024-05-08T05:53:07.032562+0000","last_peered":"2024-05-08T05:53:07.032562+0000","last_clean":"2024-05-08T05:53:07.032562+0000","last_became_active":"2024-05-08T05:51:04.317309+0000","last_became_peered":"2024-05-08T05:51:04.317309+0000","last_unstale":"2024-05-08T05:53:07.032562+0000","last_undegraded":"2024-05-08T05:53:07.032562+0000","last_fullsized":"2024-05-08T05:53:07.032562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:11.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270484+0000","last_change":"2024-05-08T05:51:01.186438+0000","last_active":"2024-05-08T05:53:05.270484+0000","last_peered":"2024-05-08T05:53:05.270484+0000","last_clean":"2024-05-08T05:53:05.270484+0000","last_became_active":"2024-05-08T05:51:01.186039+0000","last_became_peered":"2024-05-08T05:51:01.186039+0000","last_unstale":"2024-05-08T05:53:05.270484+0000","last_undegraded":"2024-05-08T05:53:05.270484+0000","last_fullsized":"2024-05-08T05:53:05.270484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:34.652184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269557+0000","last_change":"2024-05-08T05:52:05.575531+0000","last_active":"2024-05-08T05:53:05.269557+0000","last_peered":"2024-05-08T05:53:05.269557+0000","last_clean":"2024-05-08T05:53:05.269557+0000","last_became_active":"2024-05-08T05:52:05.575042+0000","last_became_peered":"2024-05-08T05:52:05.575042+0000","last_unstale":"2024-05-08T05:53:05.269557+0000","last_undegraded":"2024-05-08T05:53:05.269557+0000","last_fullsized":"2024-05-08T05:53:05.269557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:56.096772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'33","reported_seq":307,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:23.821375+0000","last_change":"2024-05-08T05:51:00.263385+0000","last_active":"2024-05-08T05:53:23.821375+0000","last_peered":"2024-05-08T05:53:23.821375+0000","last_clean":"2024-05-08T05:53:23.821375+0000","last_became_active":"2024-05-08T05:51:00.262997+0000","last_became_peered":"2024-05-08T05:51:00.262997+0000","last_unstale":"2024-05-08T05:53:23.821375+0000","last_undegraded":"2024-05-08T05:53:23.821375+0000","last_fullsized":"2024-05-08T05:53:23.821375+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:22:09.727481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61999,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":244,"num_read_kb":182,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269032+0000","last_change":"2024-05-08T05:51:04.294790+0000","last_active":"2024-05-08T05:53:05.269032+0000","last_peered":"2024-05-08T05:53:05.269032+0000","last_clean":"2024-05-08T05:53:05.269032+0000","last_became_active":"2024-05-08T05:51:04.294418+0000","last_became_peered":"2024-05-08T05:51:04.294418+0000","last_unstale":"2024-05-08T05:53:05.269032+0000","last_undegraded":"2024-05-08T05:53:05.269032+0000","last_fullsized":"2024-05-08T05:53:05.269032+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:11:10.624434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505362+0000","last_change":"2024-05-08T05:51:00.258507+0000","last_active":"2024-05-08T05:53:05.505362+0000","last_peered":"2024-05-08T05:53:05.505362+0000","last_clean":"2024-05-08T05:53:05.505362+0000","last_became_active":"2024-05-08T05:51:00.258179+0000","last_became_peered":"2024-05-08T05:51:00.258179+0000","last_unstale":"2024-05-08T05:53:05.505362+0000","last_undegraded":"2024-05-08T05:53:05.505362+0000","last_fullsized":"2024-05-08T05:53:05.505362+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:22.376499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820104+0000","last_change":"2024-05-08T05:51:04.310947+0000","last_active":"2024-05-08T05:53:05.820104+0000","last_peered":"2024-05-08T05:53:05.820104+0000","last_clean":"2024-05-08T05:53:05.820104+0000","last_became_active":"2024-05-08T05:51:04.310378+0000","last_became_peered":"2024-05-08T05:51:04.310378+0000","last_unstale":"2024-05-08T05:53:05.820104+0000","last_undegraded":"2024-05-08T05:53:05.820104+0000","last_fullsized":"2024-05-08T05:53:05.820104+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:19.367660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270609+0000","last_change":"2024-05-08T05:51:00.271104+0000","last_active":"2024-05-08T05:53:05.270609+0000","last_peered":"2024-05-08T05:53:05.270609+0000","last_clean":"2024-05-08T05:53:05.270609+0000","last_became_active":"2024-05-08T05:51:00.270497+0000","last_became_peered":"2024-05-08T05:51:00.270497+0000","last_unstale":"2024-05-08T05:53:05.270609+0000","last_undegraded":"2024-05-08T05:53:05.270609+0000","last_fullsized":"2024-05-08T05:53:05.270609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:40.959423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139512+0000","last_change":"2024-05-08T05:51:04.310243+0000","last_active":"2024-05-08T05:53:07.139512+0000","last_peered":"2024-05-08T05:53:07.139512+0000","last_clean":"2024-05-08T05:53:07.139512+0000","last_became_active":"2024-05-08T05:51:04.309871+0000","last_became_peered":"2024-05-08T05:51:04.309871+0000","last_unstale":"2024-05-08T05:53:07.139512+0000","last_undegraded":"2024-05-08T05:53:07.139512+0000","last_fullsized":"2024-05-08T05:53:07.139512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:18:21.045621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":156,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:22.864241+0000","last_change":"2024-05-08T05:51:00.266096+0000","last_active":"2024-05-08T05:53:22.864241+0000","last_peered":"2024-05-08T05:53:22.864241+0000","last_clean":"2024-05-08T05:53:22.864241+0000","last_became_active":"2024-05-08T05:51:00.265706+0000","last_became_peered":"2024-05-08T05:51:00.265706+0000","last_unstale":"2024-05-08T05:53:22.864241+0000","last_undegraded":"2024-05-08T05:53:22.864241+0000","last_fullsized":"2024-05-08T05:53:22.864241+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:47.388732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139267+0000","last_change":"2024-05-08T05:51:04.293141+0000","last_active":"2024-05-08T05:53:07.139267+0000","last_peered":"2024-05-08T05:53:07.139267+0000","last_clean":"2024-05-08T05:53:07.139267+0000","last_became_active":"2024-05-08T05:51:04.292772+0000","last_became_peered":"2024-05-08T05:51:04.292772+0000","last_unstale":"2024-05-08T05:53:07.139267+0000","last_undegraded":"2024-05-08T05:53:07.139267+0000","last_fullsized":"2024-05-08T05:53:07.139267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:25:32.801209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268837+0000","last_change":"2024-05-08T05:52:05.042102+0000","last_active":"2024-05-08T05:53:05.268837+0000","last_peered":"2024-05-08T05:53:05.268837+0000","last_clean":"2024-05-08T05:53:05.268837+0000","last_became_active":"2024-05-08T05:52:05.027787+0000","last_became_peered":"2024-05-08T05:52:05.027787+0000","last_unstale":"2024-05-08T05:53:05.268837+0000","last_undegraded":"2024-05-08T05:53:05.268837+0000","last_fullsized":"2024-05-08T05:53:05.268837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:05.580309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269940+0000","last_change":"2024-05-08T05:51:04.290096+0000","last_active":"2024-05-08T05:53:05.269940+0000","last_peered":"2024-05-08T05:53:05.269940+0000","last_clean":"2024-05-08T05:53:05.269940+0000","last_became_active":"2024-05-08T05:51:04.289684+0000","last_became_peered":"2024-05-08T05:51:04.289684+0000","last_unstale":"2024-05-08T05:53:05.269940+0000","last_undegraded":"2024-05-08T05:53:05.269940+0000","last_fullsized":"2024-05-08T05:53:05.269940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:02.400348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579740+0000","last_change":"2024-05-08T05:52:05.034004+0000","last_active":"2024-05-08T05:53:07.579740+0000","last_peered":"2024-05-08T05:53:07.579740+0000","last_clean":"2024-05-08T05:53:07.579740+0000","last_became_active":"2024-05-08T05:52:05.023497+0000","last_became_peered":"2024-05-08T05:52:05.023497+0000","last_unstale":"2024-05-08T05:53:07.579740+0000","last_undegraded":"2024-05-08T05:53:07.579740+0000","last_fullsized":"2024-05-08T05:53:07.579740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:06.462940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579850+0000","last_change":"2024-05-08T05:52:05.027648+0000","last_active":"2024-05-08T05:53:07.579850+0000","last_peered":"2024-05-08T05:53:07.579850+0000","last_clean":"2024-05-08T05:53:07.579850+0000","last_became_active":"2024-05-08T05:52:05.027262+0000","last_became_peered":"2024-05-08T05:52:05.027262+0000","last_unstale":"2024-05-08T05:53:07.579850+0000","last_undegraded":"2024-05-08T05:53:07.579850+0000","last_fullsized":"2024-05-08T05:53:07.579850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:36:18.588967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267116+0000","last_change":"2024-05-08T05:51:01.189324+0000","last_active":"2024-05-08T05:53:05.267116+0000","last_peered":"2024-05-08T05:53:05.267116+0000","last_clean":"2024-05-08T05:53:05.267116+0000","last_became_active":"2024-05-08T05:51:01.188651+0000","last_became_peered":"2024-05-08T05:51:01.188651+0000","last_unstale":"2024-05-08T05:53:05.267116+0000","last_undegraded":"2024-05-08T05:53:05.267116+0000","last_fullsized":"2024-05-08T05:53:05.267116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:45.149213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265294+0000","last_change":"2024-05-08T05:51:04.306811+0000","last_active":"2024-05-08T05:53:05.265294+0000","last_peered":"2024-05-08T05:53:05.265294+0000","last_clean":"2024-05-08T05:53:05.265294+0000","last_became_active":"2024-05-08T05:51:04.306423+0000","last_became_peered":"2024-05-08T05:51:04.306423+0000","last_unstale":"2024-05-08T05:53:05.265294+0000","last_undegraded":"2024-05-08T05:53:05.265294+0000","last_fullsized":"2024-05-08T05:53:05.265294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:08.830682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139525+0000","last_change":"2024-05-08T05:51:00.276543+0000","last_active":"2024-05-08T05:53:07.139525+0000","last_peered":"2024-05-08T05:53:07.139525+0000","last_clean":"2024-05-08T05:53:07.139525+0000","last_became_active":"2024-05-08T05:51:00.276233+0000","last_became_peered":"2024-05-08T05:51:00.276233+0000","last_unstale":"2024-05-08T05:53:07.139525+0000","last_undegraded":"2024-05-08T05:53:07.139525+0000","last_fullsized":"2024-05-08T05:53:07.139525+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:11.229942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265173+0000","last_change":"2024-05-08T05:51:04.322033+0000","last_active":"2024-05-08T05:53:05.265173+0000","last_peered":"2024-05-08T05:53:05.265173+0000","last_clean":"2024-05-08T05:53:05.265173+0000","last_became_active":"2024-05-08T05:51:04.321555+0000","last_became_peered":"2024-05-08T05:51:04.321555+0000","last_unstale":"2024-05-08T05:53:05.265173+0000","last_undegraded":"2024-05-08T05:53:05.265173+0000","last_fullsized":"2024-05-08T05:53:05.265173+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:21:50.820709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.266264+0000","last_change":"2024-05-08T05:52:05.027723+0000","last_active":"2024-05-08T05:53:05.266264+0000","last_peered":"2024-05-08T05:53:05.266264+0000","last_clean":"2024-05-08T05:53:05.266264+0000","last_became_active":"2024-05-08T05:52:05.026508+0000","last_became_peered":"2024-05-08T05:52:05.026508+0000","last_unstale":"2024-05-08T05:53:05.266264+0000","last_undegraded":"2024-05-08T05:53:05.266264+0000","last_fullsized":"2024-05-08T05:53:05.266264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:14:32.399433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":150,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:19.856550+0000","last_change":"2024-05-08T05:51:01.187556+0000","last_active":"2024-05-08T05:53:19.856550+0000","last_peered":"2024-05-08T05:53:19.856550+0000","last_clean":"2024-05-08T05:53:19.856550+0000","last_became_active":"2024-05-08T05:51:01.186584+0000","last_became_peered":"2024-05-08T05:51:01.186584+0000","last_unstale":"2024-05-08T05:53:19.856550+0000","last_undegraded":"2024-05-08T05:53:19.856550+0000","last_fullsized":"2024-05-08T05:53:19.856550+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:29.036633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.274091+0000","last_change":"2024-05-08T05:53:05.272875+0000","last_active":"2024-05-08T05:53:05.274091+0000","last_peered":"2024-05-08T05:53:05.274091+0000","last_clean":"2024-05-08T05:53:05.274091+0000","last_became_active":"2024-05-08T05:53:05.271672+0000","last_became_peered":"2024-05-08T05:53:05.271672+0000","last_unstale":"2024-05-08T05:53:05.274091+0000","last_undegraded":"2024-05-08T05:53:05.274091+0000","last_fullsized":"2024-05-08T05:53:05.274091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:08:28.776982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715396+0000","last_change":"2024-05-08T05:51:00.272501+0000","last_active":"2024-05-08T05:53:05.715396+0000","last_peered":"2024-05-08T05:53:05.715396+0000","last_clean":"2024-05-08T05:53:05.715396+0000","last_became_active":"2024-05-08T05:51:00.272166+0000","last_became_peered":"2024-05-08T05:51:00.272166+0000","last_unstale":"2024-05-08T05:53:05.715396+0000","last_undegraded":"2024-05-08T05:53:05.715396+0000","last_fullsized":"2024-05-08T05:53:05.715396+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:30.980047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":103,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139306+0000","last_change":"2024-05-08T05:49:31.820799+0000","last_active":"2024-05-08T05:53:07.139306+0000","last_peered":"2024-05-08T05:53:07.139306+0000","last_clean":"2024-05-08T05:53:07.139306+0000","last_became_active":"2024-05-08T05:49:31.815568+0000","last_became_peered":"2024-05-08T05:49:31.815568+0000","last_unstale":"2024-05-08T05:53:07.139306+0000","last_undegraded":"2024-05-08T05:53:07.139306+0000","last_fullsized":"2024-05-08T05:53:07.139306+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:48.170242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:48.170242+0000","last_clean_scrub_stamp":"2024-05-08T05:45:48.170242+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:39.527036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820012+0000","last_change":"2024-05-08T05:51:04.301375+0000","last_active":"2024-05-08T05:53:05.820012+0000","last_peered":"2024-05-08T05:53:05.820012+0000","last_clean":"2024-05-08T05:53:05.820012+0000","last_became_active":"2024-05-08T05:51:04.301027+0000","last_became_peered":"2024-05-08T05:51:04.301027+0000","last_unstale":"2024-05-08T05:53:05.820012+0000","last_undegraded":"2024-05-08T05:53:05.820012+0000","last_fullsized":"2024-05-08T05:53:05.820012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:21.393242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579548+0000","last_change":"2024-05-08T05:52:05.037427+0000","last_active":"2024-05-08T05:53:07.579548+0000","last_peered":"2024-05-08T05:53:07.579548+0000","last_clean":"2024-05-08T05:53:07.579548+0000","last_became_active":"2024-05-08T05:52:05.023644+0000","last_became_peered":"2024-05-08T05:52:05.023644+0000","last_unstale":"2024-05-08T05:53:07.579548+0000","last_undegraded":"2024-05-08T05:53:07.579548+0000","last_fullsized":"2024-05-08T05:53:07.579548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:49.193375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269166+0000","last_change":"2024-05-08T05:51:01.187171+0000","last_active":"2024-05-08T05:53:05.269166+0000","last_peered":"2024-05-08T05:53:05.269166+0000","last_clean":"2024-05-08T05:53:05.269166+0000","last_became_active":"2024-05-08T05:51:01.186142+0000","last_became_peered":"2024-05-08T05:51:01.186142+0000","last_unstale":"2024-05-08T05:53:05.269166+0000","last_undegraded":"2024-05-08T05:53:05.269166+0000","last_fullsized":"2024-05-08T05:53:05.269166+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:37:17.021905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820912+0000","last_change":"2024-05-08T05:51:04.311538+0000","last_active":"2024-05-08T05:53:05.820912+0000","last_peered":"2024-05-08T05:53:05.820912+0000","last_clean":"2024-05-08T05:53:05.820912+0000","last_became_active":"2024-05-08T05:51:04.310535+0000","last_became_peered":"2024-05-08T05:51:04.310535+0000","last_unstale":"2024-05-08T05:53:05.820912+0000","last_undegraded":"2024-05-08T05:53:05.820912+0000","last_fullsized":"2024-05-08T05:53:05.820912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:01:04.497986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'12","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267458+0000","last_change":"2024-05-08T05:51:00.277923+0000","last_active":"2024-05-08T05:53:05.267458+0000","last_peered":"2024-05-08T05:53:05.267458+0000","last_clean":"2024-05-08T05:53:05.267458+0000","last_became_active":"2024-05-08T05:51:00.274896+0000","last_became_peered":"2024-05-08T05:51:00.274896+0000","last_unstale":"2024-05-08T05:53:05.267458+0000","last_undegraded":"2024-05-08T05:53:05.267458+0000","last_fullsized":"2024-05-08T05:53:05.267458+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:04:32.543451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.573064+0000","last_change":"2024-05-08T05:51:04.306185+0000","last_active":"2024-05-08T05:53:05.573064+0000","last_peered":"2024-05-08T05:53:05.573064+0000","last_clean":"2024-05-08T05:53:05.573064+0000","last_became_active":"2024-05-08T05:51:04.305229+0000","last_became_peered":"2024-05-08T05:51:04.305229+0000","last_unstale":"2024-05-08T05:53:05.573064+0000","last_undegraded":"2024-05-08T05:53:05.573064+0000","last_fullsized":"2024-05-08T05:53:05.573064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:48.859978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715588+0000","last_change":"2024-05-08T05:51:01.140178+0000","last_active":"2024-05-08T05:53:05.715588+0000","last_peered":"2024-05-08T05:53:05.715588+0000","last_clean":"2024-05-08T05:53:05.715588+0000","last_became_active":"2024-05-08T05:51:01.139772+0000","last_became_peered":"2024-05-08T05:51:01.139772+0000","last_unstale":"2024-05-08T05:53:05.715588+0000","last_undegraded":"2024-05-08T05:53:05.715588+0000","last_fullsized":"2024-05-08T05:53:05.715588+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:54:40.132683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032654+0000","last_change":"2024-05-08T05:51:04.319015+0000","last_active":"2024-05-08T05:53:07.032654+0000","last_peered":"2024-05-08T05:53:07.032654+0000","last_clean":"2024-05-08T05:53:07.032654+0000","last_became_active":"2024-05-08T05:51:04.318465+0000","last_became_peered":"2024-05-08T05:51:04.318465+0000","last_unstale":"2024-05-08T05:53:07.032654+0000","last_undegraded":"2024-05-08T05:53:07.032654+0000","last_fullsized":"2024-05-08T05:53:07.032654+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:07:19.336656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269322+0000","last_change":"2024-05-08T05:51:01.185619+0000","last_active":"2024-05-08T05:53:05.269322+0000","last_peered":"2024-05-08T05:53:05.269322+0000","last_clean":"2024-05-08T05:53:05.269322+0000","last_became_active":"2024-05-08T05:51:01.185325+0000","last_became_peered":"2024-05-08T05:51:01.185325+0000","last_unstale":"2024-05-08T05:53:05.269322+0000","last_undegraded":"2024-05-08T05:53:05.269322+0000","last_fullsized":"2024-05-08T05:53:05.269322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:41.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268756+0000","last_change":"2024-05-08T05:51:04.295018+0000","last_active":"2024-05-08T05:53:05.268756+0000","last_peered":"2024-05-08T05:53:05.268756+0000","last_clean":"2024-05-08T05:53:05.268756+0000","last_became_active":"2024-05-08T05:51:04.294432+0000","last_became_peered":"2024-05-08T05:51:04.294432+0000","last_unstale":"2024-05-08T05:53:05.268756+0000","last_undegraded":"2024-05-08T05:53:05.268756+0000","last_fullsized":"2024-05-08T05:53:05.268756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:01.925456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571832+0000","last_change":"2024-05-08T05:51:00.280527+0000","last_active":"2024-05-08T05:53:05.571832+0000","last_peered":"2024-05-08T05:53:05.571832+0000","last_clean":"2024-05-08T05:53:05.571832+0000","last_became_active":"2024-05-08T05:51:00.278718+0000","last_became_peered":"2024-05-08T05:51:00.278718+0000","last_unstale":"2024-05-08T05:53:05.571832+0000","last_undegraded":"2024-05-08T05:53:05.571832+0000","last_fullsized":"2024-05-08T05:53:05.571832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:48:27.149713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579777+0000","last_change":"2024-05-08T05:51:04.314543+0000","last_active":"2024-05-08T05:53:07.579777+0000","last_peered":"2024-05-08T05:53:07.579777+0000","last_clean":"2024-05-08T05:53:07.579777+0000","last_became_active":"2024-05-08T05:51:04.314168+0000","last_became_peered":"2024-05-08T05:51:04.314168+0000","last_unstale":"2024-05-08T05:53:07.579777+0000","last_undegraded":"2024-05-08T05:53:07.579777+0000","last_fullsized":"2024-05-08T05:53:07.579777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:10:23.215322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505255+0000","last_change":"2024-05-08T05:51:00.268081+0000","last_active":"2024-05-08T05:53:05.505255+0000","last_peered":"2024-05-08T05:53:05.505255+0000","last_clean":"2024-05-08T05:53:05.505255+0000","last_became_active":"2024-05-08T05:51:00.267470+0000","last_became_peered":"2024-05-08T05:51:00.267470+0000","last_unstale":"2024-05-08T05:53:05.505255+0000","last_undegraded":"2024-05-08T05:53:05.505255+0000","last_fullsized":"2024-05-08T05:53:05.505255+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:45.796019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572395+0000","last_change":"2024-05-08T05:51:04.322010+0000","last_active":"2024-05-08T05:53:05.572395+0000","last_peered":"2024-05-08T05:53:05.572395+0000","last_clean":"2024-05-08T05:53:05.572395+0000","last_became_active":"2024-05-08T05:51:04.320867+0000","last_became_peered":"2024-05-08T05:51:04.320867+0000","last_unstale":"2024-05-08T05:53:05.572395+0000","last_undegraded":"2024-05-08T05:53:05.572395+0000","last_fullsized":"2024-05-08T05:53:05.572395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:49.853518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268990+0000","last_change":"2024-05-08T05:52:05.045290+0000","last_active":"2024-05-08T05:53:05.268990+0000","last_peered":"2024-05-08T05:53:05.268990+0000","last_clean":"2024-05-08T05:53:05.268990+0000","last_became_active":"2024-05-08T05:52:05.029284+0000","last_became_peered":"2024-05-08T05:52:05.029284+0000","last_unstale":"2024-05-08T05:53:05.268990+0000","last_undegraded":"2024-05-08T05:53:05.268990+0000","last_fullsized":"2024-05-08T05:53:05.268990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:20.491862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139647+0000","last_change":"2024-05-08T05:51:04.302813+0000","last_active":"2024-05-08T05:53:07.139647+0000","last_peered":"2024-05-08T05:53:07.139647+0000","last_clean":"2024-05-08T05:53:07.139647+0000","last_became_active":"2024-05-08T05:51:04.302424+0000","last_became_peered":"2024-05-08T05:51:04.302424+0000","last_unstale":"2024-05-08T05:53:07.139647+0000","last_undegraded":"2024-05-08T05:53:07.139647+0000","last_fullsized":"2024-05-08T05:53:07.139647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:52:17.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505505+0000","last_change":"2024-05-08T05:51:01.187513+0000","last_active":"2024-05-08T05:53:05.505505+0000","last_peered":"2024-05-08T05:53:05.505505+0000","last_clean":"2024-05-08T05:53:05.505505+0000","last_became_active":"2024-05-08T05:51:01.186780+0000","last_became_peered":"2024-05-08T05:51:01.186780+0000","last_unstale":"2024-05-08T05:53:05.505505+0000","last_undegraded":"2024-05-08T05:53:05.505505+0000","last_fullsized":"2024-05-08T05:53:05.505505+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:42:26.206351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139833+0000","last_change":"2024-05-08T05:51:04.295338+0000","last_active":"2024-05-08T05:53:07.139833+0000","last_peered":"2024-05-08T05:53:07.139833+0000","last_clean":"2024-05-08T05:53:07.139833+0000","last_became_active":"2024-05-08T05:51:04.294990+0000","last_became_peered":"2024-05-08T05:51:04.294990+0000","last_unstale":"2024-05-08T05:53:07.139833+0000","last_undegraded":"2024-05-08T05:53:07.139833+0000","last_fullsized":"2024-05-08T05:53:07.139833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:29.411662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270051+0000","last_change":"2024-05-08T05:51:00.273052+0000","last_active":"2024-05-08T05:53:05.270051+0000","last_peered":"2024-05-08T05:53:05.270051+0000","last_clean":"2024-05-08T05:53:05.270051+0000","last_became_active":"2024-05-08T05:51:00.272718+0000","last_became_peered":"2024-05-08T05:51:00.272718+0000","last_unstale":"2024-05-08T05:53:05.270051+0000","last_undegraded":"2024-05-08T05:53:05.270051+0000","last_fullsized":"2024-05-08T05:53:05.270051+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:44.494378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139455+0000","last_change":"2024-05-08T05:51:04.306209+0000","last_active":"2024-05-08T05:53:07.139455+0000","last_peered":"2024-05-08T05:53:07.139455+0000","last_clean":"2024-05-08T05:53:07.139455+0000","last_became_active":"2024-05-08T05:51:04.305752+0000","last_became_peered":"2024-05-08T05:51:04.305752+0000","last_unstale":"2024-05-08T05:53:07.139455+0000","last_undegraded":"2024-05-08T05:53:07.139455+0000","last_fullsized":"2024-05-08T05:53:07.139455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:24.586111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820689+0000","last_change":"2024-05-08T05:51:00.278426+0000","last_active":"2024-05-08T05:53:05.820689+0000","last_peered":"2024-05-08T05:53:05.820689+0000","last_clean":"2024-05-08T05:53:05.820689+0000","last_became_active":"2024-05-08T05:51:00.276764+0000","last_became_peered":"2024-05-08T05:51:00.276764+0000","last_unstale":"2024-05-08T05:53:05.820689+0000","last_undegraded":"2024-05-08T05:53:05.820689+0000","last_fullsized":"2024-05-08T05:53:05.820689+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:34.158628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139683+0000","last_change":"2024-05-08T05:51:04.294511+0000","last_active":"2024-05-08T05:53:07.139683+0000","last_peered":"2024-05-08T05:53:07.139683+0000","last_clean":"2024-05-08T05:53:07.139683+0000","last_became_active":"2024-05-08T05:51:04.294145+0000","last_became_peered":"2024-05-08T05:51:04.294145+0000","last_unstale":"2024-05-08T05:53:07.139683+0000","last_undegraded":"2024-05-08T05:53:07.139683+0000","last_fullsized":"2024-05-08T05:53:07.139683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:10.034060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579833+0000","last_change":"2024-05-08T05:51:04.298905+0000","last_active":"2024-05-08T05:53:07.579833+0000","last_peered":"2024-05-08T05:53:07.579833+0000","last_clean":"2024-05-08T05:53:07.579833+0000","last_became_active":"2024-05-08T05:51:04.298586+0000","last_became_peered":"2024-05-08T05:51:04.298586+0000","last_unstale":"2024-05-08T05:53:07.579833+0000","last_undegraded":"2024-05-08T05:53:07.579833+0000","last_fullsized":"2024-05-08T05:53:07.579833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:43:24.899015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505323+0000","last_change":"2024-05-08T05:51:04.300503+0000","last_active":"2024-05-08T05:53:05.505323+0000","last_peered":"2024-05-08T05:53:05.505323+0000","last_clean":"2024-05-08T05:53:05.505323+0000","last_became_active":"2024-05-08T05:51:04.299771+0000","last_became_peered":"2024-05-08T05:51:04.299771+0000","last_unstale":"2024-05-08T05:53:05.505323+0000","last_undegraded":"2024-05-08T05:53:05.505323+0000","last_fullsized":"2024-05-08T05:53:05.505323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:19.790501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505267+0000","last_change":"2024-05-08T05:52:05.027724+0000","last_active":"2024-05-08T05:53:05.505267+0000","last_peered":"2024-05-08T05:53:05.505267+0000","last_clean":"2024-05-08T05:53:05.505267+0000","last_became_active":"2024-05-08T05:52:05.027338+0000","last_became_peered":"2024-05-08T05:52:05.027338+0000","last_unstale":"2024-05-08T05:53:05.505267+0000","last_undegraded":"2024-05-08T05:53:05.505267+0000","last_fullsized":"2024-05-08T05:53:05.505267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:03.324348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032152+0000","last_change":"2024-05-08T05:52:05.574645+0000","last_active":"2024-05-08T05:53:07.032152+0000","last_peered":"2024-05-08T05:53:07.032152+0000","last_clean":"2024-05-08T05:53:07.032152+0000","last_became_active":"2024-05-08T05:52:05.573751+0000","last_became_peered":"2024-05-08T05:52:05.573751+0000","last_unstale":"2024-05-08T05:53:07.032152+0000","last_undegraded":"2024-05-08T05:53:07.032152+0000","last_fullsized":"2024-05-08T05:53:07.032152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:57:40.656001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820990+0000","last_change":"2024-05-08T05:51:04.315769+0000","last_active":"2024-05-08T05:53:05.820990+0000","last_peered":"2024-05-08T05:53:05.820990+0000","last_clean":"2024-05-08T05:53:05.820990+0000","last_became_active":"2024-05-08T05:51:04.315322+0000","last_became_peered":"2024-05-08T05:51:04.315322+0000","last_unstale":"2024-05-08T05:53:05.820990+0000","last_undegraded":"2024-05-08T05:53:05.820990+0000","last_fullsized":"2024-05-08T05:53:05.820990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:52.096556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579878+0000","last_change":"2024-05-08T05:51:04.298128+0000","last_active":"2024-05-08T05:53:07.579878+0000","last_peered":"2024-05-08T05:53:07.579878+0000","last_clean":"2024-05-08T05:53:07.579878+0000","last_became_active":"2024-05-08T05:51:04.297743+0000","last_became_peered":"2024-05-08T05:51:04.297743+0000","last_unstale":"2024-05-08T05:53:07.579878+0000","last_undegraded":"2024-05-08T05:53:07.579878+0000","last_fullsized":"2024-05-08T05:53:07.579878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:18.334026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505656+0000","last_change":"2024-05-08T05:51:04.301056+0000","last_active":"2024-05-08T05:53:05.505656+0000","last_peered":"2024-05-08T05:53:05.505656+0000","last_clean":"2024-05-08T05:53:05.505656+0000","last_became_active":"2024-05-08T05:51:04.300733+0000","last_became_peered":"2024-05-08T05:51:04.300733+0000","last_unstale":"2024-05-08T05:53:05.505656+0000","last_undegraded":"2024-05-08T05:53:05.505656+0000","last_fullsized":"2024-05-08T05:53:05.505656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:21.616867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572880+0000","last_change":"2024-05-08T05:51:04.299079+0000","last_active":"2024-05-08T05:53:05.572880+0000","last_peered":"2024-05-08T05:53:05.572880+0000","last_clean":"2024-05-08T05:53:05.572880+0000","last_became_active":"2024-05-08T05:51:04.298628+0000","last_became_peered":"2024-05-08T05:51:04.298628+0000","last_unstale":"2024-05-08T05:53:05.572880+0000","last_undegraded":"2024-05-08T05:53:05.572880+0000","last_fullsized":"2024-05-08T05:53:05.572880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:45.031695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270220+0000","last_change":"2024-05-08T05:51:04.296140+0000","last_active":"2024-05-08T05:53:05.270220+0000","last_peered":"2024-05-08T05:53:05.270220+0000","last_clean":"2024-05-08T05:53:05.270220+0000","last_became_active":"2024-05-08T05:51:04.294940+0000","last_became_peered":"2024-05-08T05:51:04.294940+0000","last_unstale":"2024-05-08T05:53:05.270220+0000","last_undegraded":"2024-05-08T05:53:05.270220+0000","last_fullsized":"2024-05-08T05:53:05.270220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:37.192639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571649+0000","last_change":"2024-05-08T05:51:04.305803+0000","last_active":"2024-05-08T05:53:05.571649+0000","last_peered":"2024-05-08T05:53:05.571649+0000","last_clean":"2024-05-08T05:53:05.571649+0000","last_became_active":"2024-05-08T05:51:04.305482+0000","last_became_peered":"2024-05-08T05:51:04.305482+0000","last_unstale":"2024-05-08T05:53:05.571649+0000","last_undegraded":"2024-05-08T05:53:05.571649+0000","last_fullsized":"2024-05-08T05:53:05.571649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:02.050153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269746+0000","last_change":"2024-05-08T05:51:04.293787+0000","last_active":"2024-05-08T05:53:05.269746+0000","last_peered":"2024-05-08T05:53:05.269746+0000","last_clean":"2024-05-08T05:53:05.269746+0000","last_became_active":"2024-05-08T05:51:04.293455+0000","last_became_peered":"2024-05-08T05:51:04.293455+0000","last_unstale":"2024-05-08T05:53:05.269746+0000","last_undegraded":"2024-05-08T05:53:05.269746+0000","last_fullsized":"2024-05-08T05:53:05.269746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:09:53.211411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820895+0000","last_change":"2024-05-08T05:51:04.321442+0000","last_active":"2024-05-08T05:53:05.820895+0000","last_peered":"2024-05-08T05:53:05.820895+0000","last_clean":"2024-05-08T05:53:05.820895+0000","last_became_active":"2024-05-08T05:51:04.320982+0000","last_became_peered":"2024-05-08T05:51:04.320982+0000","last_unstale":"2024-05-08T05:53:05.820895+0000","last_undegraded":"2024-05-08T05:53:05.820895+0000","last_fullsized":"2024-05-08T05:53:05.820895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:45.089609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579671+0000","last_change":"2024-05-08T05:51:04.294866+0000","last_active":"2024-05-08T05:53:07.579671+0000","last_peered":"2024-05-08T05:53:07.579671+0000","last_clean":"2024-05-08T05:53:07.579671+0000","last_became_active":"2024-05-08T05:51:04.294504+0000","last_became_peered":"2024-05-08T05:51:04.294504+0000","last_unstale":"2024-05-08T05:53:07.579671+0000","last_undegraded":"2024-05-08T05:53:07.579671+0000","last_fullsized":"2024-05-08T05:53:07.579671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:33.139864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66141,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":727,"num_read_kb":432,"num_write":172,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":134,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645357,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":452538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841585,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1106528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005111,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":452752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201340,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":452650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430274,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1540096,"data_stored":1044743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593799,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":452141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":514045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":453412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411422,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":515171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182488,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":452452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986259,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":453110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790032,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36856,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708296,"statfs":{"total":95995035648,"available":95957295104,"internally_reserved":0,"allocated":1552384,"data_stored":1043895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:53:27.240 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:53:27.350 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:27 smithi112 ceph-mon[30003]: pgmap v385: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:27.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:27 smithi150 ceph-mon[34315]: pgmap v385: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:27 smithi162 ceph-mon[34800]: pgmap v385: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:27.717 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:28.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:28 smithi112 ceph-mon[30003]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:53:28.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:28 smithi150 ceph-mon[34315]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:53:28.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:28 smithi162 ceph-mon[34800]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:53:29.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:29 smithi150 ceph-mon[34315]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:29.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:29 smithi112 ceph-mon[30003]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:29.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:29 smithi162 ceph-mon[34800]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.097+0000 7fe96e400700 1 -- 172.21.15.112:0/1694573120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968075ff0 msgr2=0x7fe96810f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.097+0000 7fe96e400700 1 --2- 172.21.15.112:0/1694573120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968075ff0 0x7fe96810f960 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe954009990 tx=0x7fe954009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.097+0000 7fe96e400700 1 -- 172.21.15.112:0/1694573120 shutdown_connections 2024-05-08T05:53:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.097+0000 7fe96e400700 1 --2- 172.21.15.112:0/1694573120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe96810fea0 0x7fe968112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.097+0000 7fe96e400700 1 --2- 172.21.15.112:0/1694573120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9680756d0 0x7fe968075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.097+0000 7fe96e400700 1 --2- 172.21.15.112:0/1694573120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968075ff0 0x7fe96810f960 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.097+0000 7fe96e400700 1 -- 172.21.15.112:0/1694573120 >> 172.21.15.112:0/1694573120 conn(0x7fe96807a600 msgr2=0x7fe96807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.097+0000 7fe96e400700 1 -- 172.21.15.112:0/1694573120 shutdown_connections 2024-05-08T05:53:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.097+0000 7fe96e400700 1 -- 172.21.15.112:0/1694573120 wait complete. 2024-05-08T05:53:30.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.098+0000 7fe96e400700 1 Processor -- start 2024-05-08T05:53:30.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.099+0000 7fe96e400700 1 -- start start 2024-05-08T05:53:30.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.099+0000 7fe96e400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9680756d0 0x7fe96819be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.099+0000 7fe96e400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968075ff0 0x7fe96819c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.099+0000 7fe96e400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe96810fea0 0x7fe9681a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.099+0000 7fe96e400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe968115880 con 0x7fe968075ff0 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.099+0000 7fe96e400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe968115700 con 0x7fe96810fea0 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.099+0000 7fe96e400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe968115a00 con 0x7fe9680756d0 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.099+0000 7fe966a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968075ff0 0x7fe96819c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.099+0000 7fe967400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9680756d0 0x7fe96819be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.099+0000 7fe967400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9680756d0 0x7fe96819be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:43176/0 (socket says 172.21.15.112:43176) 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.099+0000 7fe967400700 1 -- 172.21.15.112:0/3885725761 learned_addr learned my addr 172.21.15.112:0/3885725761 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.099+0000 7fe967e00700 1 --2- 172.21.15.112:0/3885725761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe96810fea0 0x7fe9681a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.100+0000 7fe967400700 1 -- 172.21.15.112:0/3885725761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe96810fea0 msgr2=0x7fe9681a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.100+0000 7fe967400700 1 --2- 172.21.15.112:0/3885725761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe96810fea0 0x7fe9681a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.100+0000 7fe967400700 1 -- 172.21.15.112:0/3885725761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968075ff0 msgr2=0x7fe96819c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.100+0000 7fe967400700 1 --2- 172.21.15.112:0/3885725761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968075ff0 0x7fe96819c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.100+0000 7fe967400700 1 -- 172.21.15.112:0/3885725761 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9681a0e30 con 0x7fe9680756d0 2024-05-08T05:53:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.100+0000 7fe966a00700 1 --2- 172.21.15.112:0/3885725761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968075ff0 0x7fe96819c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.100+0000 7fe967400700 1 --2- 172.21.15.112:0/3885725761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9680756d0 0x7fe96819be20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe95800e9a0 tx=0x7fe95800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.100+0000 7fe95fe00700 1 -- 172.21.15.112:0/3885725761 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95800ca10 con 0x7fe9680756d0 2024-05-08T05:53:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.100+0000 7fe95fe00700 1 -- 172.21.15.112:0/3885725761 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95800cbb0 con 0x7fe9680756d0 2024-05-08T05:53:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.100+0000 7fe96e400700 1 -- 172.21.15.112:0/3885725761 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9681a1120 con 0x7fe9680756d0 2024-05-08T05:53:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.101+0000 7fe95fe00700 1 -- 172.21.15.112:0/3885725761 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958018680 con 0x7fe9680756d0 2024-05-08T05:53:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.101+0000 7fe96e400700 1 -- 172.21.15.112:0/3885725761 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9681a89a0 con 0x7fe9680756d0 2024-05-08T05:53:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.102+0000 7fe96e400700 1 -- 172.21.15.112:0/3885725761 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe96806a190 con 0x7fe9680756d0 2024-05-08T05:53:30.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.104+0000 7fe95fe00700 1 -- 172.21.15.112:0/3885725761 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe958010430 con 0x7fe9680756d0 2024-05-08T05:53:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.106+0000 7fe95fe00700 1 --2- 172.21.15.112:0/3885725761 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe94c0b2880 0x7fe94c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.106+0000 7fe95fe00700 1 -- 172.21.15.112:0/3885725761 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fe9580e2750 con 0x7fe9680756d0 2024-05-08T05:53:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.106+0000 7fe966a00700 1 --2- 172.21.15.112:0/3885725761 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe94c0b2880 0x7fe94c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.107+0000 7fe966a00700 1 --2- 172.21.15.112:0/3885725761 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe94c0b2880 0x7fe94c0b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe9540098e0 tx=0x7fe954005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.108+0000 7fe95fe00700 1 -- 172.21.15.112:0/3885725761 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe9580aa3e0 con 0x7fe9680756d0 2024-05-08T05:53:30.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.332+0000 7fe96e400700 1 -- 172.21.15.112:0/3885725761 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe96819d180 con 0x7fe94c0b2880 2024-05-08T05:53:30.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.347+0000 7fe95fe00700 1 -- 172.21.15.112:0/3885725761 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258828 (secure 0 0 0) 0x7fe954065320 con 0x7fe94c0b2880 2024-05-08T05:53:30.352 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:53:30.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.353+0000 7fe96e400700 1 -- 172.21.15.112:0/3885725761 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe94c0b2880 msgr2=0x7fe94c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:30.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.353+0000 7fe96e400700 1 --2- 172.21.15.112:0/3885725761 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe94c0b2880 0x7fe94c0b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe9540098e0 tx=0x7fe954005960 comp rx=0 tx=0).stop 2024-05-08T05:53:30.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.353+0000 7fe96e400700 1 -- 172.21.15.112:0/3885725761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9680756d0 msgr2=0x7fe96819be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:30.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.353+0000 7fe96e400700 1 --2- 172.21.15.112:0/3885725761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9680756d0 0x7fe96819be20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe95800e9a0 tx=0x7fe95800ecb0 comp rx=0 tx=0).stop 2024-05-08T05:53:30.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.353+0000 7fe96e400700 1 -- 172.21.15.112:0/3885725761 shutdown_connections 2024-05-08T05:53:30.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.353+0000 7fe96e400700 1 --2- 172.21.15.112:0/3885725761 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe94c0b2880 0x7fe94c0b4d40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:30.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.353+0000 7fe96e400700 1 --2- 172.21.15.112:0/3885725761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe96810fea0 0x7fe9681a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:30.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.353+0000 7fe96e400700 1 --2- 172.21.15.112:0/3885725761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9680756d0 0x7fe96819be20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:30.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.353+0000 7fe96e400700 1 --2- 172.21.15.112:0/3885725761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe968075ff0 0x7fe96819c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:30.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.353+0000 7fe96e400700 1 -- 172.21.15.112:0/3885725761 >> 172.21.15.112:0/3885725761 conn(0x7fe96807a600 msgr2=0x7fe968101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:30.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.354+0000 7fe96e400700 1 -- 172.21.15.112:0/3885725761 shutdown_connections 2024-05-08T05:53:30.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:30.354+0000 7fe96e400700 1 -- 172.21.15.112:0/3885725761 wait complete. 2024-05-08T05:53:30.356 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-05-08T05:53:31.308 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-05-08T05:53:29.526996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656743,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":833,"num_read_kb":511,"num_write":297,"num_write_kb":2269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951974,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":740,"ondisk_log_size":740,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366404,"kb_used_data":13020,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575420,"statfs":{"total":1151940427776,"available":1151565230080,"internally_reserved":0,"allocated":13332480,"data_stored":7393437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004754"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.580018+0000","last_change":"2024-05-08T05:51:04.320319+0000","last_active":"2024-05-08T05:53:07.580018+0000","last_peered":"2024-05-08T05:53:07.580018+0000","last_clean":"2024-05-08T05:53:07.580018+0000","last_became_active":"2024-05-08T05:51:04.319961+0000","last_became_peered":"2024-05-08T05:51:04.319961+0000","last_unstale":"2024-05-08T05:53:07.580018+0000","last_undegraded":"2024-05-08T05:53:07.580018+0000","last_fullsized":"2024-05-08T05:53:07.580018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:32.463261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032466+0000","last_change":"2024-05-08T05:51:04.315531+0000","last_active":"2024-05-08T05:53:07.032466+0000","last_peered":"2024-05-08T05:53:07.032466+0000","last_clean":"2024-05-08T05:53:07.032466+0000","last_became_active":"2024-05-08T05:51:04.314116+0000","last_became_peered":"2024-05-08T05:51:04.314116+0000","last_unstale":"2024-05-08T05:53:07.032466+0000","last_undegraded":"2024-05-08T05:53:07.032466+0000","last_fullsized":"2024-05-08T05:53:07.032466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:12:03.505888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032236+0000","last_change":"2024-05-08T05:51:04.315826+0000","last_active":"2024-05-08T05:53:07.032236+0000","last_peered":"2024-05-08T05:53:07.032236+0000","last_clean":"2024-05-08T05:53:07.032236+0000","last_became_active":"2024-05-08T05:51:04.315460+0000","last_became_peered":"2024-05-08T05:51:04.315460+0000","last_unstale":"2024-05-08T05:53:07.032236+0000","last_undegraded":"2024-05-08T05:53:07.032236+0000","last_fullsized":"2024-05-08T05:53:07.032236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:22:20.412368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715439+0000","last_change":"2024-05-08T05:51:04.297401+0000","last_active":"2024-05-08T05:53:05.715439+0000","last_peered":"2024-05-08T05:53:05.715439+0000","last_clean":"2024-05-08T05:53:05.715439+0000","last_became_active":"2024-05-08T05:51:04.297061+0000","last_became_peered":"2024-05-08T05:51:04.297061+0000","last_unstale":"2024-05-08T05:53:05.715439+0000","last_undegraded":"2024-05-08T05:53:05.715439+0000","last_fullsized":"2024-05-08T05:53:05.715439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:27.967517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268733+0000","last_change":"2024-05-08T05:51:04.315069+0000","last_active":"2024-05-08T05:53:05.268733+0000","last_peered":"2024-05-08T05:53:05.268733+0000","last_clean":"2024-05-08T05:53:05.268733+0000","last_became_active":"2024-05-08T05:51:04.314513+0000","last_became_peered":"2024-05-08T05:51:04.314513+0000","last_unstale":"2024-05-08T05:53:05.268733+0000","last_undegraded":"2024-05-08T05:53:05.268733+0000","last_fullsized":"2024-05-08T05:53:05.268733+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:02.520130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265149+0000","last_change":"2024-05-08T05:51:04.302851+0000","last_active":"2024-05-08T05:53:05.265149+0000","last_peered":"2024-05-08T05:53:05.265149+0000","last_clean":"2024-05-08T05:53:05.265149+0000","last_became_active":"2024-05-08T05:51:04.302256+0000","last_became_peered":"2024-05-08T05:51:04.302256+0000","last_unstale":"2024-05-08T05:53:05.265149+0000","last_undegraded":"2024-05-08T05:53:05.265149+0000","last_fullsized":"2024-05-08T05:53:05.265149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:45.587319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139237+0000","last_change":"2024-05-08T05:51:04.308674+0000","last_active":"2024-05-08T05:53:07.139237+0000","last_peered":"2024-05-08T05:53:07.139237+0000","last_clean":"2024-05-08T05:53:07.139237+0000","last_became_active":"2024-05-08T05:51:04.308295+0000","last_became_peered":"2024-05-08T05:51:04.308295+0000","last_unstale":"2024-05-08T05:53:07.139237+0000","last_undegraded":"2024-05-08T05:53:07.139237+0000","last_fullsized":"2024-05-08T05:53:07.139237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:59:17.561655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571478+0000","last_change":"2024-05-08T05:51:04.314511+0000","last_active":"2024-05-08T05:53:05.571478+0000","last_peered":"2024-05-08T05:53:05.571478+0000","last_clean":"2024-05-08T05:53:05.571478+0000","last_became_active":"2024-05-08T05:51:04.314082+0000","last_became_peered":"2024-05-08T05:51:04.314082+0000","last_unstale":"2024-05-08T05:53:05.571478+0000","last_undegraded":"2024-05-08T05:53:05.571478+0000","last_fullsized":"2024-05-08T05:53:05.571478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:24.182391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269793+0000","last_change":"2024-05-08T05:51:04.293745+0000","last_active":"2024-05-08T05:53:05.269793+0000","last_peered":"2024-05-08T05:53:05.269793+0000","last_clean":"2024-05-08T05:53:05.269793+0000","last_became_active":"2024-05-08T05:51:04.293334+0000","last_became_peered":"2024-05-08T05:51:04.293334+0000","last_unstale":"2024-05-08T05:53:05.269793+0000","last_undegraded":"2024-05-08T05:53:05.269793+0000","last_fullsized":"2024-05-08T05:53:05.269793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:16.576967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265481+0000","last_change":"2024-05-08T05:51:04.297800+0000","last_active":"2024-05-08T05:53:05.265481+0000","last_peered":"2024-05-08T05:53:05.265481+0000","last_clean":"2024-05-08T05:53:05.265481+0000","last_became_active":"2024-05-08T05:51:04.297349+0000","last_became_peered":"2024-05-08T05:51:04.297349+0000","last_unstale":"2024-05-08T05:53:05.265481+0000","last_undegraded":"2024-05-08T05:53:05.265481+0000","last_fullsized":"2024-05-08T05:53:05.265481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:55.784512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139575+0000","last_change":"2024-05-08T05:51:04.297997+0000","last_active":"2024-05-08T05:53:07.139575+0000","last_peered":"2024-05-08T05:53:07.139575+0000","last_clean":"2024-05-08T05:53:07.139575+0000","last_became_active":"2024-05-08T05:51:04.297516+0000","last_became_peered":"2024-05-08T05:51:04.297516+0000","last_unstale":"2024-05-08T05:53:07.139575+0000","last_undegraded":"2024-05-08T05:53:07.139575+0000","last_fullsized":"2024-05-08T05:53:07.139575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:25.991555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715483+0000","last_change":"2024-05-08T05:51:04.295610+0000","last_active":"2024-05-08T05:53:05.715483+0000","last_peered":"2024-05-08T05:53:05.715483+0000","last_clean":"2024-05-08T05:53:05.715483+0000","last_became_active":"2024-05-08T05:51:04.294553+0000","last_became_peered":"2024-05-08T05:51:04.294553+0000","last_unstale":"2024-05-08T05:53:05.715483+0000","last_undegraded":"2024-05-08T05:53:05.715483+0000","last_fullsized":"2024-05-08T05:53:05.715483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:40:14.492057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820576+0000","last_change":"2024-05-08T05:51:04.303879+0000","last_active":"2024-05-08T05:53:05.820576+0000","last_peered":"2024-05-08T05:53:05.820576+0000","last_clean":"2024-05-08T05:53:05.820576+0000","last_became_active":"2024-05-08T05:51:04.303423+0000","last_became_peered":"2024-05-08T05:51:04.303423+0000","last_unstale":"2024-05-08T05:53:05.820576+0000","last_undegraded":"2024-05-08T05:53:05.820576+0000","last_fullsized":"2024-05-08T05:53:05.820576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:50:55.905858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.031822+0000","last_change":"2024-05-08T05:51:04.309818+0000","last_active":"2024-05-08T05:53:07.031822+0000","last_peered":"2024-05-08T05:53:07.031822+0000","last_clean":"2024-05-08T05:53:07.031822+0000","last_became_active":"2024-05-08T05:51:04.309434+0000","last_became_peered":"2024-05-08T05:51:04.309434+0000","last_unstale":"2024-05-08T05:53:07.031822+0000","last_undegraded":"2024-05-08T05:53:07.031822+0000","last_fullsized":"2024-05-08T05:53:07.031822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:55.832239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.031573+0000","last_change":"2024-05-08T05:51:04.316708+0000","last_active":"2024-05-08T05:53:07.031573+0000","last_peered":"2024-05-08T05:53:07.031573+0000","last_clean":"2024-05-08T05:53:07.031573+0000","last_became_active":"2024-05-08T05:51:04.316525+0000","last_became_peered":"2024-05-08T05:51:04.316525+0000","last_unstale":"2024-05-08T05:53:07.031573+0000","last_undegraded":"2024-05-08T05:53:07.031573+0000","last_fullsized":"2024-05-08T05:53:07.031573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:33:38.246611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269307+0000","last_change":"2024-05-08T05:51:04.314780+0000","last_active":"2024-05-08T05:53:05.269307+0000","last_peered":"2024-05-08T05:53:05.269307+0000","last_clean":"2024-05-08T05:53:05.269307+0000","last_became_active":"2024-05-08T05:51:04.314406+0000","last_became_peered":"2024-05-08T05:51:04.314406+0000","last_unstale":"2024-05-08T05:53:05.269307+0000","last_undegraded":"2024-05-08T05:53:05.269307+0000","last_fullsized":"2024-05-08T05:53:05.269307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:39:26.926238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579947+0000","last_change":"2024-05-08T05:51:04.319425+0000","last_active":"2024-05-08T05:53:07.579947+0000","last_peered":"2024-05-08T05:53:07.579947+0000","last_clean":"2024-05-08T05:53:07.579947+0000","last_became_active":"2024-05-08T05:51:04.319106+0000","last_became_peered":"2024-05-08T05:51:04.319106+0000","last_unstale":"2024-05-08T05:53:07.579947+0000","last_undegraded":"2024-05-08T05:53:07.579947+0000","last_fullsized":"2024-05-08T05:53:07.579947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:59:54.023162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267374+0000","last_change":"2024-05-08T05:51:04.303547+0000","last_active":"2024-05-08T05:53:05.267374+0000","last_peered":"2024-05-08T05:53:05.267374+0000","last_clean":"2024-05-08T05:53:05.267374+0000","last_became_active":"2024-05-08T05:51:04.303182+0000","last_became_peered":"2024-05-08T05:51:04.303182+0000","last_unstale":"2024-05-08T05:53:05.267374+0000","last_undegraded":"2024-05-08T05:53:05.267374+0000","last_fullsized":"2024-05-08T05:53:05.267374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:11:59.624329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505589+0000","last_change":"2024-05-08T05:51:04.315674+0000","last_active":"2024-05-08T05:53:05.505589+0000","last_peered":"2024-05-08T05:53:05.505589+0000","last_clean":"2024-05-08T05:53:05.505589+0000","last_became_active":"2024-05-08T05:51:04.314911+0000","last_became_peered":"2024-05-08T05:51:04.314911+0000","last_unstale":"2024-05-08T05:53:05.505589+0000","last_undegraded":"2024-05-08T05:53:05.505589+0000","last_fullsized":"2024-05-08T05:53:05.505589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:49.501508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267963+0000","last_change":"2024-05-08T05:51:00.273554+0000","last_active":"2024-05-08T05:53:05.267963+0000","last_peered":"2024-05-08T05:53:05.267963+0000","last_clean":"2024-05-08T05:53:05.267963+0000","last_became_active":"2024-05-08T05:51:00.272858+0000","last_became_peered":"2024-05-08T05:51:00.272858+0000","last_unstale":"2024-05-08T05:53:05.267963+0000","last_undegraded":"2024-05-08T05:53:05.267963+0000","last_fullsized":"2024-05-08T05:53:05.267963+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:47:37.418887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572441+0000","last_change":"2024-05-08T05:51:04.306392+0000","last_active":"2024-05-08T05:53:05.572441+0000","last_peered":"2024-05-08T05:53:05.572441+0000","last_clean":"2024-05-08T05:53:05.572441+0000","last_became_active":"2024-05-08T05:51:04.304248+0000","last_became_peered":"2024-05-08T05:51:04.304248+0000","last_unstale":"2024-05-08T05:53:05.572441+0000","last_undegraded":"2024-05-08T05:53:05.572441+0000","last_fullsized":"2024-05-08T05:53:05.572441+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:49:26.034883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572774+0000","last_change":"2024-05-08T05:51:01.187292+0000","last_active":"2024-05-08T05:53:05.572774+0000","last_peered":"2024-05-08T05:53:05.572774+0000","last_clean":"2024-05-08T05:53:05.572774+0000","last_became_active":"2024-05-08T05:51:01.185948+0000","last_became_peered":"2024-05-08T05:51:01.185948+0000","last_unstale":"2024-05-08T05:53:05.572774+0000","last_undegraded":"2024-05-08T05:53:05.572774+0000","last_fullsized":"2024-05-08T05:53:05.572774+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:02.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032756+0000","last_change":"2024-05-08T05:51:04.318439+0000","last_active":"2024-05-08T05:53:07.032756+0000","last_peered":"2024-05-08T05:53:07.032756+0000","last_clean":"2024-05-08T05:53:07.032756+0000","last_became_active":"2024-05-08T05:51:04.317722+0000","last_became_peered":"2024-05-08T05:51:04.317722+0000","last_unstale":"2024-05-08T05:53:07.032756+0000","last_undegraded":"2024-05-08T05:53:07.032756+0000","last_fullsized":"2024-05-08T05:53:07.032756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:36:43.648643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269243+0000","last_change":"2024-05-08T05:51:00.269684+0000","last_active":"2024-05-08T05:53:05.269243+0000","last_peered":"2024-05-08T05:53:05.269243+0000","last_clean":"2024-05-08T05:53:05.269243+0000","last_became_active":"2024-05-08T05:51:00.269329+0000","last_became_peered":"2024-05-08T05:51:00.269329+0000","last_unstale":"2024-05-08T05:53:05.269243+0000","last_undegraded":"2024-05-08T05:53:05.269243+0000","last_fullsized":"2024-05-08T05:53:05.269243+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:27:10.422994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269350+0000","last_change":"2024-05-08T05:51:04.308048+0000","last_active":"2024-05-08T05:53:05.269350+0000","last_peered":"2024-05-08T05:53:05.269350+0000","last_clean":"2024-05-08T05:53:05.269350+0000","last_became_active":"2024-05-08T05:51:04.307550+0000","last_became_peered":"2024-05-08T05:51:04.307550+0000","last_unstale":"2024-05-08T05:53:05.269350+0000","last_undegraded":"2024-05-08T05:53:05.269350+0000","last_fullsized":"2024-05-08T05:53:05.269350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:13.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268974+0000","last_change":"2024-05-08T05:51:00.268142+0000","last_active":"2024-05-08T05:53:05.268974+0000","last_peered":"2024-05-08T05:53:05.268974+0000","last_clean":"2024-05-08T05:53:05.268974+0000","last_became_active":"2024-05-08T05:51:00.267756+0000","last_became_peered":"2024-05-08T05:51:00.267756+0000","last_unstale":"2024-05-08T05:53:05.268974+0000","last_undegraded":"2024-05-08T05:53:05.268974+0000","last_fullsized":"2024-05-08T05:53:05.268974+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:01.439680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270137+0000","last_change":"2024-05-08T05:51:04.292341+0000","last_active":"2024-05-08T05:53:05.270137+0000","last_peered":"2024-05-08T05:53:05.270137+0000","last_clean":"2024-05-08T05:53:05.270137+0000","last_became_active":"2024-05-08T05:51:04.291086+0000","last_became_peered":"2024-05-08T05:51:04.291086+0000","last_unstale":"2024-05-08T05:53:05.270137+0000","last_undegraded":"2024-05-08T05:53:05.270137+0000","last_fullsized":"2024-05-08T05:53:05.270137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:47.108100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":160,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:28.830153+0000","last_change":"2024-05-08T05:51:00.268100+0000","last_active":"2024-05-08T05:53:28.830153+0000","last_peered":"2024-05-08T05:53:28.830153+0000","last_clean":"2024-05-08T05:53:28.830153+0000","last_became_active":"2024-05-08T05:51:00.267678+0000","last_became_peered":"2024-05-08T05:51:00.267678+0000","last_unstale":"2024-05-08T05:53:28.830153+0000","last_undegraded":"2024-05-08T05:53:28.830153+0000","last_fullsized":"2024-05-08T05:53:28.830153+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:11:31.731548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268695+0000","last_change":"2024-05-08T05:51:04.303872+0000","last_active":"2024-05-08T05:53:05.268695+0000","last_peered":"2024-05-08T05:53:05.268695+0000","last_clean":"2024-05-08T05:53:05.268695+0000","last_became_active":"2024-05-08T05:51:04.303456+0000","last_became_peered":"2024-05-08T05:51:04.303456+0000","last_unstale":"2024-05-08T05:53:05.268695+0000","last_undegraded":"2024-05-08T05:53:05.268695+0000","last_fullsized":"2024-05-08T05:53:05.268695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:42:05.098488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265391+0000","last_change":"2024-05-08T05:51:00.275657+0000","last_active":"2024-05-08T05:53:05.265391+0000","last_peered":"2024-05-08T05:53:05.265391+0000","last_clean":"2024-05-08T05:53:05.265391+0000","last_became_active":"2024-05-08T05:51:00.275254+0000","last_became_peered":"2024-05-08T05:51:00.275254+0000","last_unstale":"2024-05-08T05:53:05.265391+0000","last_undegraded":"2024-05-08T05:53:05.265391+0000","last_fullsized":"2024-05-08T05:53:05.265391+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:28.555698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270045+0000","last_change":"2024-05-08T05:51:04.293791+0000","last_active":"2024-05-08T05:53:05.270045+0000","last_peered":"2024-05-08T05:53:05.270045+0000","last_clean":"2024-05-08T05:53:05.270045+0000","last_became_active":"2024-05-08T05:51:04.293405+0000","last_became_peered":"2024-05-08T05:51:04.293405+0000","last_unstale":"2024-05-08T05:53:05.270045+0000","last_undegraded":"2024-05-08T05:53:05.270045+0000","last_fullsized":"2024-05-08T05:53:05.270045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:28.064452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820147+0000","last_change":"2024-05-08T05:51:00.280295+0000","last_active":"2024-05-08T05:53:05.820147+0000","last_peered":"2024-05-08T05:53:05.820147+0000","last_clean":"2024-05-08T05:53:05.820147+0000","last_became_active":"2024-05-08T05:51:00.279721+0000","last_became_peered":"2024-05-08T05:51:00.279721+0000","last_unstale":"2024-05-08T05:53:05.820147+0000","last_undegraded":"2024-05-08T05:53:05.820147+0000","last_fullsized":"2024-05-08T05:53:05.820147+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:06:38.064872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571814+0000","last_change":"2024-05-08T05:51:04.307546+0000","last_active":"2024-05-08T05:53:05.571814+0000","last_peered":"2024-05-08T05:53:05.571814+0000","last_clean":"2024-05-08T05:53:05.571814+0000","last_became_active":"2024-05-08T05:51:04.307164+0000","last_became_peered":"2024-05-08T05:51:04.307164+0000","last_unstale":"2024-05-08T05:53:05.571814+0000","last_undegraded":"2024-05-08T05:53:05.571814+0000","last_fullsized":"2024-05-08T05:53:05.571814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:36:02.269040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269777+0000","last_change":"2024-05-08T05:51:00.267987+0000","last_active":"2024-05-08T05:53:05.269777+0000","last_peered":"2024-05-08T05:53:05.269777+0000","last_clean":"2024-05-08T05:53:05.269777+0000","last_became_active":"2024-05-08T05:51:00.267633+0000","last_became_peered":"2024-05-08T05:51:00.267633+0000","last_unstale":"2024-05-08T05:53:05.269777+0000","last_undegraded":"2024-05-08T05:53:05.269777+0000","last_fullsized":"2024-05-08T05:53:05.269777+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:13.034300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.266338+0000","last_change":"2024-05-08T05:51:04.311422+0000","last_active":"2024-05-08T05:53:05.266338+0000","last_peered":"2024-05-08T05:53:05.266338+0000","last_clean":"2024-05-08T05:53:05.266338+0000","last_became_active":"2024-05-08T05:51:04.311052+0000","last_became_peered":"2024-05-08T05:51:04.311052+0000","last_unstale":"2024-05-08T05:53:05.266338+0000","last_undegraded":"2024-05-08T05:53:05.266338+0000","last_fullsized":"2024-05-08T05:53:05.266338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:47.790583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":155,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:27.827487+0000","last_change":"2024-05-08T05:51:00.274252+0000","last_active":"2024-05-08T05:53:27.827487+0000","last_peered":"2024-05-08T05:53:27.827487+0000","last_clean":"2024-05-08T05:53:27.827487+0000","last_became_active":"2024-05-08T05:51:00.273942+0000","last_became_peered":"2024-05-08T05:51:00.273942+0000","last_unstale":"2024-05-08T05:53:27.827487+0000","last_undegraded":"2024-05-08T05:53:27.827487+0000","last_fullsized":"2024-05-08T05:53:27.827487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:36.778032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269700+0000","last_change":"2024-05-08T05:51:04.296414+0000","last_active":"2024-05-08T05:53:05.269700+0000","last_peered":"2024-05-08T05:53:05.269700+0000","last_clean":"2024-05-08T05:53:05.269700+0000","last_became_active":"2024-05-08T05:51:04.296015+0000","last_became_peered":"2024-05-08T05:51:04.296015+0000","last_unstale":"2024-05-08T05:53:05.269700+0000","last_undegraded":"2024-05-08T05:53:05.269700+0000","last_fullsized":"2024-05-08T05:53:05.269700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:48:15.609385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139516+0000","last_change":"2024-05-08T05:51:00.268234+0000","last_active":"2024-05-08T05:53:07.139516+0000","last_peered":"2024-05-08T05:53:07.139516+0000","last_clean":"2024-05-08T05:53:07.139516+0000","last_became_active":"2024-05-08T05:51:00.267846+0000","last_became_peered":"2024-05-08T05:51:00.267846+0000","last_unstale":"2024-05-08T05:53:07.139516+0000","last_undegraded":"2024-05-08T05:53:07.139516+0000","last_fullsized":"2024-05-08T05:53:07.139516+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:26.919004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032562+0000","last_change":"2024-05-08T05:51:04.318215+0000","last_active":"2024-05-08T05:53:07.032562+0000","last_peered":"2024-05-08T05:53:07.032562+0000","last_clean":"2024-05-08T05:53:07.032562+0000","last_became_active":"2024-05-08T05:51:04.317309+0000","last_became_peered":"2024-05-08T05:51:04.317309+0000","last_unstale":"2024-05-08T05:53:07.032562+0000","last_undegraded":"2024-05-08T05:53:07.032562+0000","last_fullsized":"2024-05-08T05:53:07.032562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:11.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270484+0000","last_change":"2024-05-08T05:51:01.186438+0000","last_active":"2024-05-08T05:53:05.270484+0000","last_peered":"2024-05-08T05:53:05.270484+0000","last_clean":"2024-05-08T05:53:05.270484+0000","last_became_active":"2024-05-08T05:51:01.186039+0000","last_became_peered":"2024-05-08T05:51:01.186039+0000","last_unstale":"2024-05-08T05:53:05.270484+0000","last_undegraded":"2024-05-08T05:53:05.270484+0000","last_fullsized":"2024-05-08T05:53:05.270484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:34.652184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269557+0000","last_change":"2024-05-08T05:52:05.575531+0000","last_active":"2024-05-08T05:53:05.269557+0000","last_peered":"2024-05-08T05:53:05.269557+0000","last_clean":"2024-05-08T05:53:05.269557+0000","last_became_active":"2024-05-08T05:52:05.575042+0000","last_became_peered":"2024-05-08T05:52:05.575042+0000","last_unstale":"2024-05-08T05:53:05.269557+0000","last_undegraded":"2024-05-08T05:53:05.269557+0000","last_fullsized":"2024-05-08T05:53:05.269557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:56.096772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'33","reported_seq":317,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:28.829169+0000","last_change":"2024-05-08T05:51:00.263385+0000","last_active":"2024-05-08T05:53:28.829169+0000","last_peered":"2024-05-08T05:53:28.829169+0000","last_clean":"2024-05-08T05:53:28.829169+0000","last_became_active":"2024-05-08T05:51:00.262997+0000","last_became_peered":"2024-05-08T05:51:00.262997+0000","last_unstale":"2024-05-08T05:53:28.829169+0000","last_undegraded":"2024-05-08T05:53:28.829169+0000","last_fullsized":"2024-05-08T05:53:28.829169+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:22:09.727481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61999,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":254,"num_read_kb":187,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269032+0000","last_change":"2024-05-08T05:51:04.294790+0000","last_active":"2024-05-08T05:53:05.269032+0000","last_peered":"2024-05-08T05:53:05.269032+0000","last_clean":"2024-05-08T05:53:05.269032+0000","last_became_active":"2024-05-08T05:51:04.294418+0000","last_became_peered":"2024-05-08T05:51:04.294418+0000","last_unstale":"2024-05-08T05:53:05.269032+0000","last_undegraded":"2024-05-08T05:53:05.269032+0000","last_fullsized":"2024-05-08T05:53:05.269032+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:11:10.624434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505362+0000","last_change":"2024-05-08T05:51:00.258507+0000","last_active":"2024-05-08T05:53:05.505362+0000","last_peered":"2024-05-08T05:53:05.505362+0000","last_clean":"2024-05-08T05:53:05.505362+0000","last_became_active":"2024-05-08T05:51:00.258179+0000","last_became_peered":"2024-05-08T05:51:00.258179+0000","last_unstale":"2024-05-08T05:53:05.505362+0000","last_undegraded":"2024-05-08T05:53:05.505362+0000","last_fullsized":"2024-05-08T05:53:05.505362+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:22.376499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820379+0000","last_change":"2024-05-08T05:51:04.310947+0000","last_active":"2024-05-08T05:53:05.820379+0000","last_peered":"2024-05-08T05:53:05.820379+0000","last_clean":"2024-05-08T05:53:05.820379+0000","last_became_active":"2024-05-08T05:51:04.310378+0000","last_became_peered":"2024-05-08T05:51:04.310378+0000","last_unstale":"2024-05-08T05:53:05.820379+0000","last_undegraded":"2024-05-08T05:53:05.820379+0000","last_fullsized":"2024-05-08T05:53:05.820379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:19.367660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270609+0000","last_change":"2024-05-08T05:51:00.271104+0000","last_active":"2024-05-08T05:53:05.270609+0000","last_peered":"2024-05-08T05:53:05.270609+0000","last_clean":"2024-05-08T05:53:05.270609+0000","last_became_active":"2024-05-08T05:51:00.270497+0000","last_became_peered":"2024-05-08T05:51:00.270497+0000","last_unstale":"2024-05-08T05:53:05.270609+0000","last_undegraded":"2024-05-08T05:53:05.270609+0000","last_fullsized":"2024-05-08T05:53:05.270609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:40.959423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139512+0000","last_change":"2024-05-08T05:51:04.310243+0000","last_active":"2024-05-08T05:53:07.139512+0000","last_peered":"2024-05-08T05:53:07.139512+0000","last_clean":"2024-05-08T05:53:07.139512+0000","last_became_active":"2024-05-08T05:51:04.309871+0000","last_became_peered":"2024-05-08T05:51:04.309871+0000","last_unstale":"2024-05-08T05:53:07.139512+0000","last_undegraded":"2024-05-08T05:53:07.139512+0000","last_fullsized":"2024-05-08T05:53:07.139512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:18:21.045621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":161,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:26.873237+0000","last_change":"2024-05-08T05:51:00.266096+0000","last_active":"2024-05-08T05:53:26.873237+0000","last_peered":"2024-05-08T05:53:26.873237+0000","last_clean":"2024-05-08T05:53:26.873237+0000","last_became_active":"2024-05-08T05:51:00.265706+0000","last_became_peered":"2024-05-08T05:51:00.265706+0000","last_unstale":"2024-05-08T05:53:26.873237+0000","last_undegraded":"2024-05-08T05:53:26.873237+0000","last_fullsized":"2024-05-08T05:53:26.873237+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:47.388732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139267+0000","last_change":"2024-05-08T05:51:04.293141+0000","last_active":"2024-05-08T05:53:07.139267+0000","last_peered":"2024-05-08T05:53:07.139267+0000","last_clean":"2024-05-08T05:53:07.139267+0000","last_became_active":"2024-05-08T05:51:04.292772+0000","last_became_peered":"2024-05-08T05:51:04.292772+0000","last_unstale":"2024-05-08T05:53:07.139267+0000","last_undegraded":"2024-05-08T05:53:07.139267+0000","last_fullsized":"2024-05-08T05:53:07.139267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:25:32.801209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268837+0000","last_change":"2024-05-08T05:52:05.042102+0000","last_active":"2024-05-08T05:53:05.268837+0000","last_peered":"2024-05-08T05:53:05.268837+0000","last_clean":"2024-05-08T05:53:05.268837+0000","last_became_active":"2024-05-08T05:52:05.027787+0000","last_became_peered":"2024-05-08T05:52:05.027787+0000","last_unstale":"2024-05-08T05:53:05.268837+0000","last_undegraded":"2024-05-08T05:53:05.268837+0000","last_fullsized":"2024-05-08T05:53:05.268837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:05.580309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269940+0000","last_change":"2024-05-08T05:51:04.290096+0000","last_active":"2024-05-08T05:53:05.269940+0000","last_peered":"2024-05-08T05:53:05.269940+0000","last_clean":"2024-05-08T05:53:05.269940+0000","last_became_active":"2024-05-08T05:51:04.289684+0000","last_became_peered":"2024-05-08T05:51:04.289684+0000","last_unstale":"2024-05-08T05:53:05.269940+0000","last_undegraded":"2024-05-08T05:53:05.269940+0000","last_fullsized":"2024-05-08T05:53:05.269940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:02.400348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579740+0000","last_change":"2024-05-08T05:52:05.034004+0000","last_active":"2024-05-08T05:53:07.579740+0000","last_peered":"2024-05-08T05:53:07.579740+0000","last_clean":"2024-05-08T05:53:07.579740+0000","last_became_active":"2024-05-08T05:52:05.023497+0000","last_became_peered":"2024-05-08T05:52:05.023497+0000","last_unstale":"2024-05-08T05:53:07.579740+0000","last_undegraded":"2024-05-08T05:53:07.579740+0000","last_fullsized":"2024-05-08T05:53:07.579740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:06.462940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579850+0000","last_change":"2024-05-08T05:52:05.027648+0000","last_active":"2024-05-08T05:53:07.579850+0000","last_peered":"2024-05-08T05:53:07.579850+0000","last_clean":"2024-05-08T05:53:07.579850+0000","last_became_active":"2024-05-08T05:52:05.027262+0000","last_became_peered":"2024-05-08T05:52:05.027262+0000","last_unstale":"2024-05-08T05:53:07.579850+0000","last_undegraded":"2024-05-08T05:53:07.579850+0000","last_fullsized":"2024-05-08T05:53:07.579850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:36:18.588967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267116+0000","last_change":"2024-05-08T05:51:01.189324+0000","last_active":"2024-05-08T05:53:05.267116+0000","last_peered":"2024-05-08T05:53:05.267116+0000","last_clean":"2024-05-08T05:53:05.267116+0000","last_became_active":"2024-05-08T05:51:01.188651+0000","last_became_peered":"2024-05-08T05:51:01.188651+0000","last_unstale":"2024-05-08T05:53:05.267116+0000","last_undegraded":"2024-05-08T05:53:05.267116+0000","last_fullsized":"2024-05-08T05:53:05.267116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:45.149213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265294+0000","last_change":"2024-05-08T05:51:04.306811+0000","last_active":"2024-05-08T05:53:05.265294+0000","last_peered":"2024-05-08T05:53:05.265294+0000","last_clean":"2024-05-08T05:53:05.265294+0000","last_became_active":"2024-05-08T05:51:04.306423+0000","last_became_peered":"2024-05-08T05:51:04.306423+0000","last_unstale":"2024-05-08T05:53:05.265294+0000","last_undegraded":"2024-05-08T05:53:05.265294+0000","last_fullsized":"2024-05-08T05:53:05.265294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:08.830682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139525+0000","last_change":"2024-05-08T05:51:00.276543+0000","last_active":"2024-05-08T05:53:07.139525+0000","last_peered":"2024-05-08T05:53:07.139525+0000","last_clean":"2024-05-08T05:53:07.139525+0000","last_became_active":"2024-05-08T05:51:00.276233+0000","last_became_peered":"2024-05-08T05:51:00.276233+0000","last_unstale":"2024-05-08T05:53:07.139525+0000","last_undegraded":"2024-05-08T05:53:07.139525+0000","last_fullsized":"2024-05-08T05:53:07.139525+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:11.229942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265173+0000","last_change":"2024-05-08T05:51:04.322033+0000","last_active":"2024-05-08T05:53:05.265173+0000","last_peered":"2024-05-08T05:53:05.265173+0000","last_clean":"2024-05-08T05:53:05.265173+0000","last_became_active":"2024-05-08T05:51:04.321555+0000","last_became_peered":"2024-05-08T05:51:04.321555+0000","last_unstale":"2024-05-08T05:53:05.265173+0000","last_undegraded":"2024-05-08T05:53:05.265173+0000","last_fullsized":"2024-05-08T05:53:05.265173+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:21:50.820709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.266264+0000","last_change":"2024-05-08T05:52:05.027723+0000","last_active":"2024-05-08T05:53:05.266264+0000","last_peered":"2024-05-08T05:53:05.266264+0000","last_clean":"2024-05-08T05:53:05.266264+0000","last_became_active":"2024-05-08T05:52:05.026508+0000","last_became_peered":"2024-05-08T05:52:05.026508+0000","last_unstale":"2024-05-08T05:53:05.266264+0000","last_undegraded":"2024-05-08T05:53:05.266264+0000","last_fullsized":"2024-05-08T05:53:05.266264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:14:32.399433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":155,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:24.867868+0000","last_change":"2024-05-08T05:51:01.187556+0000","last_active":"2024-05-08T05:53:24.867868+0000","last_peered":"2024-05-08T05:53:24.867868+0000","last_clean":"2024-05-08T05:53:24.867868+0000","last_became_active":"2024-05-08T05:51:01.186584+0000","last_became_peered":"2024-05-08T05:51:01.186584+0000","last_unstale":"2024-05-08T05:53:24.867868+0000","last_undegraded":"2024-05-08T05:53:24.867868+0000","last_fullsized":"2024-05-08T05:53:24.867868+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:29.036633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.274091+0000","last_change":"2024-05-08T05:53:05.272875+0000","last_active":"2024-05-08T05:53:05.274091+0000","last_peered":"2024-05-08T05:53:05.274091+0000","last_clean":"2024-05-08T05:53:05.274091+0000","last_became_active":"2024-05-08T05:53:05.271672+0000","last_became_peered":"2024-05-08T05:53:05.271672+0000","last_unstale":"2024-05-08T05:53:05.274091+0000","last_undegraded":"2024-05-08T05:53:05.274091+0000","last_fullsized":"2024-05-08T05:53:05.274091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:08:28.776982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715396+0000","last_change":"2024-05-08T05:51:00.272501+0000","last_active":"2024-05-08T05:53:05.715396+0000","last_peered":"2024-05-08T05:53:05.715396+0000","last_clean":"2024-05-08T05:53:05.715396+0000","last_became_active":"2024-05-08T05:51:00.272166+0000","last_became_peered":"2024-05-08T05:51:00.272166+0000","last_unstale":"2024-05-08T05:53:05.715396+0000","last_undegraded":"2024-05-08T05:53:05.715396+0000","last_fullsized":"2024-05-08T05:53:05.715396+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:30.980047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":103,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139306+0000","last_change":"2024-05-08T05:49:31.820799+0000","last_active":"2024-05-08T05:53:07.139306+0000","last_peered":"2024-05-08T05:53:07.139306+0000","last_clean":"2024-05-08T05:53:07.139306+0000","last_became_active":"2024-05-08T05:49:31.815568+0000","last_became_peered":"2024-05-08T05:49:31.815568+0000","last_unstale":"2024-05-08T05:53:07.139306+0000","last_undegraded":"2024-05-08T05:53:07.139306+0000","last_fullsized":"2024-05-08T05:53:07.139306+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:48.170242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:48.170242+0000","last_clean_scrub_stamp":"2024-05-08T05:45:48.170242+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:39.527036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820012+0000","last_change":"2024-05-08T05:51:04.301375+0000","last_active":"2024-05-08T05:53:05.820012+0000","last_peered":"2024-05-08T05:53:05.820012+0000","last_clean":"2024-05-08T05:53:05.820012+0000","last_became_active":"2024-05-08T05:51:04.301027+0000","last_became_peered":"2024-05-08T05:51:04.301027+0000","last_unstale":"2024-05-08T05:53:05.820012+0000","last_undegraded":"2024-05-08T05:53:05.820012+0000","last_fullsized":"2024-05-08T05:53:05.820012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:21.393242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579548+0000","last_change":"2024-05-08T05:52:05.037427+0000","last_active":"2024-05-08T05:53:07.579548+0000","last_peered":"2024-05-08T05:53:07.579548+0000","last_clean":"2024-05-08T05:53:07.579548+0000","last_became_active":"2024-05-08T05:52:05.023644+0000","last_became_peered":"2024-05-08T05:52:05.023644+0000","last_unstale":"2024-05-08T05:53:07.579548+0000","last_undegraded":"2024-05-08T05:53:07.579548+0000","last_fullsized":"2024-05-08T05:53:07.579548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:49.193375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269166+0000","last_change":"2024-05-08T05:51:01.187171+0000","last_active":"2024-05-08T05:53:05.269166+0000","last_peered":"2024-05-08T05:53:05.269166+0000","last_clean":"2024-05-08T05:53:05.269166+0000","last_became_active":"2024-05-08T05:51:01.186142+0000","last_became_peered":"2024-05-08T05:51:01.186142+0000","last_unstale":"2024-05-08T05:53:05.269166+0000","last_undegraded":"2024-05-08T05:53:05.269166+0000","last_fullsized":"2024-05-08T05:53:05.269166+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:37:17.021905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820912+0000","last_change":"2024-05-08T05:51:04.311538+0000","last_active":"2024-05-08T05:53:05.820912+0000","last_peered":"2024-05-08T05:53:05.820912+0000","last_clean":"2024-05-08T05:53:05.820912+0000","last_became_active":"2024-05-08T05:51:04.310535+0000","last_became_peered":"2024-05-08T05:51:04.310535+0000","last_unstale":"2024-05-08T05:53:05.820912+0000","last_undegraded":"2024-05-08T05:53:05.820912+0000","last_fullsized":"2024-05-08T05:53:05.820912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:01:04.497986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'12","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267458+0000","last_change":"2024-05-08T05:51:00.277923+0000","last_active":"2024-05-08T05:53:05.267458+0000","last_peered":"2024-05-08T05:53:05.267458+0000","last_clean":"2024-05-08T05:53:05.267458+0000","last_became_active":"2024-05-08T05:51:00.274896+0000","last_became_peered":"2024-05-08T05:51:00.274896+0000","last_unstale":"2024-05-08T05:53:05.267458+0000","last_undegraded":"2024-05-08T05:53:05.267458+0000","last_fullsized":"2024-05-08T05:53:05.267458+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:04:32.543451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.573064+0000","last_change":"2024-05-08T05:51:04.306185+0000","last_active":"2024-05-08T05:53:05.573064+0000","last_peered":"2024-05-08T05:53:05.573064+0000","last_clean":"2024-05-08T05:53:05.573064+0000","last_became_active":"2024-05-08T05:51:04.305229+0000","last_became_peered":"2024-05-08T05:51:04.305229+0000","last_unstale":"2024-05-08T05:53:05.573064+0000","last_undegraded":"2024-05-08T05:53:05.573064+0000","last_fullsized":"2024-05-08T05:53:05.573064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:48.859978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715588+0000","last_change":"2024-05-08T05:51:01.140178+0000","last_active":"2024-05-08T05:53:05.715588+0000","last_peered":"2024-05-08T05:53:05.715588+0000","last_clean":"2024-05-08T05:53:05.715588+0000","last_became_active":"2024-05-08T05:51:01.139772+0000","last_became_peered":"2024-05-08T05:51:01.139772+0000","last_unstale":"2024-05-08T05:53:05.715588+0000","last_undegraded":"2024-05-08T05:53:05.715588+0000","last_fullsized":"2024-05-08T05:53:05.715588+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:54:40.132683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032654+0000","last_change":"2024-05-08T05:51:04.319015+0000","last_active":"2024-05-08T05:53:07.032654+0000","last_peered":"2024-05-08T05:53:07.032654+0000","last_clean":"2024-05-08T05:53:07.032654+0000","last_became_active":"2024-05-08T05:51:04.318465+0000","last_became_peered":"2024-05-08T05:51:04.318465+0000","last_unstale":"2024-05-08T05:53:07.032654+0000","last_undegraded":"2024-05-08T05:53:07.032654+0000","last_fullsized":"2024-05-08T05:53:07.032654+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:07:19.336656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269322+0000","last_change":"2024-05-08T05:51:01.185619+0000","last_active":"2024-05-08T05:53:05.269322+0000","last_peered":"2024-05-08T05:53:05.269322+0000","last_clean":"2024-05-08T05:53:05.269322+0000","last_became_active":"2024-05-08T05:51:01.185325+0000","last_became_peered":"2024-05-08T05:51:01.185325+0000","last_unstale":"2024-05-08T05:53:05.269322+0000","last_undegraded":"2024-05-08T05:53:05.269322+0000","last_fullsized":"2024-05-08T05:53:05.269322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:41.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268756+0000","last_change":"2024-05-08T05:51:04.295018+0000","last_active":"2024-05-08T05:53:05.268756+0000","last_peered":"2024-05-08T05:53:05.268756+0000","last_clean":"2024-05-08T05:53:05.268756+0000","last_became_active":"2024-05-08T05:51:04.294432+0000","last_became_peered":"2024-05-08T05:51:04.294432+0000","last_unstale":"2024-05-08T05:53:05.268756+0000","last_undegraded":"2024-05-08T05:53:05.268756+0000","last_fullsized":"2024-05-08T05:53:05.268756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:01.925456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571832+0000","last_change":"2024-05-08T05:51:00.280527+0000","last_active":"2024-05-08T05:53:05.571832+0000","last_peered":"2024-05-08T05:53:05.571832+0000","last_clean":"2024-05-08T05:53:05.571832+0000","last_became_active":"2024-05-08T05:51:00.278718+0000","last_became_peered":"2024-05-08T05:51:00.278718+0000","last_unstale":"2024-05-08T05:53:05.571832+0000","last_undegraded":"2024-05-08T05:53:05.571832+0000","last_fullsized":"2024-05-08T05:53:05.571832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:48:27.149713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579777+0000","last_change":"2024-05-08T05:51:04.314543+0000","last_active":"2024-05-08T05:53:07.579777+0000","last_peered":"2024-05-08T05:53:07.579777+0000","last_clean":"2024-05-08T05:53:07.579777+0000","last_became_active":"2024-05-08T05:51:04.314168+0000","last_became_peered":"2024-05-08T05:51:04.314168+0000","last_unstale":"2024-05-08T05:53:07.579777+0000","last_undegraded":"2024-05-08T05:53:07.579777+0000","last_fullsized":"2024-05-08T05:53:07.579777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:10:23.215322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505255+0000","last_change":"2024-05-08T05:51:00.268081+0000","last_active":"2024-05-08T05:53:05.505255+0000","last_peered":"2024-05-08T05:53:05.505255+0000","last_clean":"2024-05-08T05:53:05.505255+0000","last_became_active":"2024-05-08T05:51:00.267470+0000","last_became_peered":"2024-05-08T05:51:00.267470+0000","last_unstale":"2024-05-08T05:53:05.505255+0000","last_undegraded":"2024-05-08T05:53:05.505255+0000","last_fullsized":"2024-05-08T05:53:05.505255+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:45.796019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572395+0000","last_change":"2024-05-08T05:51:04.322010+0000","last_active":"2024-05-08T05:53:05.572395+0000","last_peered":"2024-05-08T05:53:05.572395+0000","last_clean":"2024-05-08T05:53:05.572395+0000","last_became_active":"2024-05-08T05:51:04.320867+0000","last_became_peered":"2024-05-08T05:51:04.320867+0000","last_unstale":"2024-05-08T05:53:05.572395+0000","last_undegraded":"2024-05-08T05:53:05.572395+0000","last_fullsized":"2024-05-08T05:53:05.572395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:49.853518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268990+0000","last_change":"2024-05-08T05:52:05.045290+0000","last_active":"2024-05-08T05:53:05.268990+0000","last_peered":"2024-05-08T05:53:05.268990+0000","last_clean":"2024-05-08T05:53:05.268990+0000","last_became_active":"2024-05-08T05:52:05.029284+0000","last_became_peered":"2024-05-08T05:52:05.029284+0000","last_unstale":"2024-05-08T05:53:05.268990+0000","last_undegraded":"2024-05-08T05:53:05.268990+0000","last_fullsized":"2024-05-08T05:53:05.268990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:20.491862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139647+0000","last_change":"2024-05-08T05:51:04.302813+0000","last_active":"2024-05-08T05:53:07.139647+0000","last_peered":"2024-05-08T05:53:07.139647+0000","last_clean":"2024-05-08T05:53:07.139647+0000","last_became_active":"2024-05-08T05:51:04.302424+0000","last_became_peered":"2024-05-08T05:51:04.302424+0000","last_unstale":"2024-05-08T05:53:07.139647+0000","last_undegraded":"2024-05-08T05:53:07.139647+0000","last_fullsized":"2024-05-08T05:53:07.139647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:52:17.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505505+0000","last_change":"2024-05-08T05:51:01.187513+0000","last_active":"2024-05-08T05:53:05.505505+0000","last_peered":"2024-05-08T05:53:05.505505+0000","last_clean":"2024-05-08T05:53:05.505505+0000","last_became_active":"2024-05-08T05:51:01.186780+0000","last_became_peered":"2024-05-08T05:51:01.186780+0000","last_unstale":"2024-05-08T05:53:05.505505+0000","last_undegraded":"2024-05-08T05:53:05.505505+0000","last_fullsized":"2024-05-08T05:53:05.505505+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:42:26.206351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139833+0000","last_change":"2024-05-08T05:51:04.295338+0000","last_active":"2024-05-08T05:53:07.139833+0000","last_peered":"2024-05-08T05:53:07.139833+0000","last_clean":"2024-05-08T05:53:07.139833+0000","last_became_active":"2024-05-08T05:51:04.294990+0000","last_became_peered":"2024-05-08T05:51:04.294990+0000","last_unstale":"2024-05-08T05:53:07.139833+0000","last_undegraded":"2024-05-08T05:53:07.139833+0000","last_fullsized":"2024-05-08T05:53:07.139833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:29.411662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270051+0000","last_change":"2024-05-08T05:51:00.273052+0000","last_active":"2024-05-08T05:53:05.270051+0000","last_peered":"2024-05-08T05:53:05.270051+0000","last_clean":"2024-05-08T05:53:05.270051+0000","last_became_active":"2024-05-08T05:51:00.272718+0000","last_became_peered":"2024-05-08T05:51:00.272718+0000","last_unstale":"2024-05-08T05:53:05.270051+0000","last_undegraded":"2024-05-08T05:53:05.270051+0000","last_fullsized":"2024-05-08T05:53:05.270051+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:44.494378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139455+0000","last_change":"2024-05-08T05:51:04.306209+0000","last_active":"2024-05-08T05:53:07.139455+0000","last_peered":"2024-05-08T05:53:07.139455+0000","last_clean":"2024-05-08T05:53:07.139455+0000","last_became_active":"2024-05-08T05:51:04.305752+0000","last_became_peered":"2024-05-08T05:51:04.305752+0000","last_unstale":"2024-05-08T05:53:07.139455+0000","last_undegraded":"2024-05-08T05:53:07.139455+0000","last_fullsized":"2024-05-08T05:53:07.139455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:24.586111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820689+0000","last_change":"2024-05-08T05:51:00.278426+0000","last_active":"2024-05-08T05:53:05.820689+0000","last_peered":"2024-05-08T05:53:05.820689+0000","last_clean":"2024-05-08T05:53:05.820689+0000","last_became_active":"2024-05-08T05:51:00.276764+0000","last_became_peered":"2024-05-08T05:51:00.276764+0000","last_unstale":"2024-05-08T05:53:05.820689+0000","last_undegraded":"2024-05-08T05:53:05.820689+0000","last_fullsized":"2024-05-08T05:53:05.820689+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:34.158628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139683+0000","last_change":"2024-05-08T05:51:04.294511+0000","last_active":"2024-05-08T05:53:07.139683+0000","last_peered":"2024-05-08T05:53:07.139683+0000","last_clean":"2024-05-08T05:53:07.139683+0000","last_became_active":"2024-05-08T05:51:04.294145+0000","last_became_peered":"2024-05-08T05:51:04.294145+0000","last_unstale":"2024-05-08T05:53:07.139683+0000","last_undegraded":"2024-05-08T05:53:07.139683+0000","last_fullsized":"2024-05-08T05:53:07.139683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:10.034060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579833+0000","last_change":"2024-05-08T05:51:04.298905+0000","last_active":"2024-05-08T05:53:07.579833+0000","last_peered":"2024-05-08T05:53:07.579833+0000","last_clean":"2024-05-08T05:53:07.579833+0000","last_became_active":"2024-05-08T05:51:04.298586+0000","last_became_peered":"2024-05-08T05:51:04.298586+0000","last_unstale":"2024-05-08T05:53:07.579833+0000","last_undegraded":"2024-05-08T05:53:07.579833+0000","last_fullsized":"2024-05-08T05:53:07.579833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:43:24.899015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505323+0000","last_change":"2024-05-08T05:51:04.300503+0000","last_active":"2024-05-08T05:53:05.505323+0000","last_peered":"2024-05-08T05:53:05.505323+0000","last_clean":"2024-05-08T05:53:05.505323+0000","last_became_active":"2024-05-08T05:51:04.299771+0000","last_became_peered":"2024-05-08T05:51:04.299771+0000","last_unstale":"2024-05-08T05:53:05.505323+0000","last_undegraded":"2024-05-08T05:53:05.505323+0000","last_fullsized":"2024-05-08T05:53:05.505323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:19.790501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505267+0000","last_change":"2024-05-08T05:52:05.027724+0000","last_active":"2024-05-08T05:53:05.505267+0000","last_peered":"2024-05-08T05:53:05.505267+0000","last_clean":"2024-05-08T05:53:05.505267+0000","last_became_active":"2024-05-08T05:52:05.027338+0000","last_became_peered":"2024-05-08T05:52:05.027338+0000","last_unstale":"2024-05-08T05:53:05.505267+0000","last_undegraded":"2024-05-08T05:53:05.505267+0000","last_fullsized":"2024-05-08T05:53:05.505267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:03.324348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032152+0000","last_change":"2024-05-08T05:52:05.574645+0000","last_active":"2024-05-08T05:53:07.032152+0000","last_peered":"2024-05-08T05:53:07.032152+0000","last_clean":"2024-05-08T05:53:07.032152+0000","last_became_active":"2024-05-08T05:52:05.573751+0000","last_became_peered":"2024-05-08T05:52:05.573751+0000","last_unstale":"2024-05-08T05:53:07.032152+0000","last_undegraded":"2024-05-08T05:53:07.032152+0000","last_fullsized":"2024-05-08T05:53:07.032152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:57:40.656001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820990+0000","last_change":"2024-05-08T05:51:04.315769+0000","last_active":"2024-05-08T05:53:05.820990+0000","last_peered":"2024-05-08T05:53:05.820990+0000","last_clean":"2024-05-08T05:53:05.820990+0000","last_became_active":"2024-05-08T05:51:04.315322+0000","last_became_peered":"2024-05-08T05:51:04.315322+0000","last_unstale":"2024-05-08T05:53:05.820990+0000","last_undegraded":"2024-05-08T05:53:05.820990+0000","last_fullsized":"2024-05-08T05:53:05.820990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:52.096556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579878+0000","last_change":"2024-05-08T05:51:04.298128+0000","last_active":"2024-05-08T05:53:07.579878+0000","last_peered":"2024-05-08T05:53:07.579878+0000","last_clean":"2024-05-08T05:53:07.579878+0000","last_became_active":"2024-05-08T05:51:04.297743+0000","last_became_peered":"2024-05-08T05:51:04.297743+0000","last_unstale":"2024-05-08T05:53:07.579878+0000","last_undegraded":"2024-05-08T05:53:07.579878+0000","last_fullsized":"2024-05-08T05:53:07.579878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:18.334026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505656+0000","last_change":"2024-05-08T05:51:04.301056+0000","last_active":"2024-05-08T05:53:05.505656+0000","last_peered":"2024-05-08T05:53:05.505656+0000","last_clean":"2024-05-08T05:53:05.505656+0000","last_became_active":"2024-05-08T05:51:04.300733+0000","last_became_peered":"2024-05-08T05:51:04.300733+0000","last_unstale":"2024-05-08T05:53:05.505656+0000","last_undegraded":"2024-05-08T05:53:05.505656+0000","last_fullsized":"2024-05-08T05:53:05.505656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:21.616867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572880+0000","last_change":"2024-05-08T05:51:04.299079+0000","last_active":"2024-05-08T05:53:05.572880+0000","last_peered":"2024-05-08T05:53:05.572880+0000","last_clean":"2024-05-08T05:53:05.572880+0000","last_became_active":"2024-05-08T05:51:04.298628+0000","last_became_peered":"2024-05-08T05:51:04.298628+0000","last_unstale":"2024-05-08T05:53:05.572880+0000","last_undegraded":"2024-05-08T05:53:05.572880+0000","last_fullsized":"2024-05-08T05:53:05.572880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:45.031695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270220+0000","last_change":"2024-05-08T05:51:04.296140+0000","last_active":"2024-05-08T05:53:05.270220+0000","last_peered":"2024-05-08T05:53:05.270220+0000","last_clean":"2024-05-08T05:53:05.270220+0000","last_became_active":"2024-05-08T05:51:04.294940+0000","last_became_peered":"2024-05-08T05:51:04.294940+0000","last_unstale":"2024-05-08T05:53:05.270220+0000","last_undegraded":"2024-05-08T05:53:05.270220+0000","last_fullsized":"2024-05-08T05:53:05.270220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:37.192639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571649+0000","last_change":"2024-05-08T05:51:04.305803+0000","last_active":"2024-05-08T05:53:05.571649+0000","last_peered":"2024-05-08T05:53:05.571649+0000","last_clean":"2024-05-08T05:53:05.571649+0000","last_became_active":"2024-05-08T05:51:04.305482+0000","last_became_peered":"2024-05-08T05:51:04.305482+0000","last_unstale":"2024-05-08T05:53:05.571649+0000","last_undegraded":"2024-05-08T05:53:05.571649+0000","last_fullsized":"2024-05-08T05:53:05.571649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:02.050153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269746+0000","last_change":"2024-05-08T05:51:04.293787+0000","last_active":"2024-05-08T05:53:05.269746+0000","last_peered":"2024-05-08T05:53:05.269746+0000","last_clean":"2024-05-08T05:53:05.269746+0000","last_became_active":"2024-05-08T05:51:04.293455+0000","last_became_peered":"2024-05-08T05:51:04.293455+0000","last_unstale":"2024-05-08T05:53:05.269746+0000","last_undegraded":"2024-05-08T05:53:05.269746+0000","last_fullsized":"2024-05-08T05:53:05.269746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:09:53.211411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820895+0000","last_change":"2024-05-08T05:51:04.321442+0000","last_active":"2024-05-08T05:53:05.820895+0000","last_peered":"2024-05-08T05:53:05.820895+0000","last_clean":"2024-05-08T05:53:05.820895+0000","last_became_active":"2024-05-08T05:51:04.320982+0000","last_became_peered":"2024-05-08T05:51:04.320982+0000","last_unstale":"2024-05-08T05:53:05.820895+0000","last_undegraded":"2024-05-08T05:53:05.820895+0000","last_fullsized":"2024-05-08T05:53:05.820895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:45.089609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579671+0000","last_change":"2024-05-08T05:51:04.294866+0000","last_active":"2024-05-08T05:53:07.579671+0000","last_peered":"2024-05-08T05:53:07.579671+0000","last_clean":"2024-05-08T05:53:07.579671+0000","last_became_active":"2024-05-08T05:51:04.294504+0000","last_became_peered":"2024-05-08T05:51:04.294504+0000","last_unstale":"2024-05-08T05:53:07.579671+0000","last_undegraded":"2024-05-08T05:53:07.579671+0000","last_fullsized":"2024-05-08T05:53:07.579671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:33.139864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66141,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":757,"num_read_kb":447,"num_write":172,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":134,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645357,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":452538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841586,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1106528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005112,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":452752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201341,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":452650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430274,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1540096,"data_stored":1044743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593800,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":452141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":514045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":453412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":515171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":452452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986260,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":453110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790033,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36856,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708296,"statfs":{"total":95995035648,"available":95957295104,"internally_reserved":0,"allocated":1552384,"data_stored":1043895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:53:31.312 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T05:53:31.312 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T05:53:31.312 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-08T05:53:31.312 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-08T05:53:31.416 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:31 smithi112 ceph-mon[30003]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:31.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:31 smithi150 ceph-mon[34315]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:31.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:31 smithi162 ceph-mon[34800]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:31.791 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:32.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:32 smithi112 ceph-mon[30003]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:53:32.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:32 smithi150 ceph-mon[34315]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:53:32.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:32 smithi162 ceph-mon[34800]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:53:33.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:33 smithi150 ceph-mon[34315]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:33.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:33 smithi112 ceph-mon[30003]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:33.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:33 smithi162 ceph-mon[34800]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:34.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.176+0000 7f19ea200700 1 -- 172.21.15.112:0/3282483246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e4104100 msgr2=0x7f19e4108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:34.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.176+0000 7f19ea200700 1 --2- 172.21.15.112:0/3282483246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e4104100 0x7f19e4108120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f19d00099c0 tx=0x7f19d0009cd0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.176+0000 7f19ea200700 1 -- 172.21.15.112:0/3282483246 shutdown_connections 2024-05-08T05:53:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.176+0000 7f19ea200700 1 --2- 172.21.15.112:0/3282483246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e4108850 0x7f19e41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.176+0000 7f19ea200700 1 --2- 172.21.15.112:0/3282483246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19e4103750 0x7f19e4103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.176+0000 7f19ea200700 1 --2- 172.21.15.112:0/3282483246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e4104100 0x7f19e4108120 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.176+0000 7f19ea200700 1 -- 172.21.15.112:0/3282483246 >> 172.21.15.112:0/3282483246 conn(0x7f19e407a600 msgr2=0x7f19e407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.176+0000 7f19ea200700 1 -- 172.21.15.112:0/3282483246 shutdown_connections 2024-05-08T05:53:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.176+0000 7f19ea200700 1 -- 172.21.15.112:0/3282483246 wait complete. 2024-05-08T05:53:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.177+0000 7f19ea200700 1 Processor -- start 2024-05-08T05:53:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.177+0000 7f19ea200700 1 -- start start 2024-05-08T05:53:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.178+0000 7f19ea200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e4103750 0x7f19e419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.178+0000 7f19ea200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19e4104100 0x7f19e419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.178+0000 7f19ea200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e4108850 0x7f19e41a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.178+0000 7f19ea200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19e4113680 con 0x7f19e4103750 2024-05-08T05:53:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.178+0000 7f19ea200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f19e4113500 con 0x7f19e4108850 2024-05-08T05:53:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.178+0000 7f19ea200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f19e4113800 con 0x7f19e4104100 2024-05-08T05:53:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.178+0000 7f19e3400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e4103750 0x7f19e419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.178+0000 7f19e3400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e4103750 0x7f19e419bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60076/0 (socket says 172.21.15.112:60076) 2024-05-08T05:53:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.178+0000 7f19e3400700 1 -- 172.21.15.112:0/772124911 learned_addr learned my addr 172.21.15.112:0/772124911 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.178+0000 7f19e2a00700 1 --2- 172.21.15.112:0/772124911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19e4104100 0x7f19e419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.178+0000 7f19e3e00700 1 --2- 172.21.15.112:0/772124911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e4108850 0x7f19e41a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.178+0000 7f19e3400700 1 -- 172.21.15.112:0/772124911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19e4104100 msgr2=0x7f19e419c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.178+0000 7f19e3400700 1 --2- 172.21.15.112:0/772124911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19e4104100 0x7f19e419c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.179+0000 7f19e3400700 1 -- 172.21.15.112:0/772124911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e4108850 msgr2=0x7f19e41a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.179+0000 7f19e3400700 1 --2- 172.21.15.112:0/772124911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e4108850 0x7f19e41a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.179+0000 7f19e3400700 1 -- 172.21.15.112:0/772124911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e41a0d80 con 0x7f19e4103750 2024-05-08T05:53:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.179+0000 7f19e3400700 1 --2- 172.21.15.112:0/772124911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e4103750 0x7f19e419bd70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f19d400e9c0 tx=0x7f19d400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.179+0000 7f19dbe00700 1 -- 172.21.15.112:0/772124911 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d400cbd0 con 0x7f19e4103750 2024-05-08T05:53:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.179+0000 7f19dbe00700 1 -- 172.21.15.112:0/772124911 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19d400ee60 con 0x7f19e4103750 2024-05-08T05:53:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.179+0000 7f19ea200700 1 -- 172.21.15.112:0/772124911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e41a1070 con 0x7f19e4103750 2024-05-08T05:53:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.179+0000 7f19dbe00700 1 -- 172.21.15.112:0/772124911 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d40055c0 con 0x7f19e4103750 2024-05-08T05:53:34.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.179+0000 7f19ea200700 1 -- 172.21.15.112:0/772124911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e41a88f0 con 0x7f19e4103750 2024-05-08T05:53:34.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.180+0000 7f19ea200700 1 -- 172.21.15.112:0/772124911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e406a190 con 0x7f19e4103750 2024-05-08T05:53:34.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.183+0000 7f19dbe00700 1 -- 172.21.15.112:0/772124911 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f19d4004750 con 0x7f19e4103750 2024-05-08T05:53:34.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.184+0000 7f19dbe00700 1 --2- 172.21.15.112:0/772124911 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f19c80b2880 0x7f19c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.184+0000 7f19dbe00700 1 -- 172.21.15.112:0/772124911 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f19d40e2790 con 0x7f19e4103750 2024-05-08T05:53:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.184+0000 7f19e2a00700 1 --2- 172.21.15.112:0/772124911 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f19c80b2880 0x7f19c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:34.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.185+0000 7f19e2a00700 1 --2- 172.21.15.112:0/772124911 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f19c80b2880 0x7f19c80b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19e4077730 tx=0x7f19d0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:34.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.186+0000 7f19dbe00700 1 -- 172.21.15.112:0/772124911 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f19d4054df0 con 0x7f19e4103750 2024-05-08T05:53:34.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.470+0000 7f19ea200700 1 -- 172.21.15.112:0/772124911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f19e419d0d0 con 0x7f19e4103750 2024-05-08T05:53:34.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.471+0000 7f19dbe00700 1 -- 172.21.15.112:0/772124911 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f19e419d0d0 con 0x7f19e4103750 2024-05-08T05:53:34.472 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:53:34.472 INFO:teuthology.orchestra.run.smithi112.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T05:53:34.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.474+0000 7f19ea200700 1 -- 172.21.15.112:0/772124911 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f19c80b2880 msgr2=0x7f19c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:34.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.474+0000 7f19ea200700 1 --2- 172.21.15.112:0/772124911 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f19c80b2880 0x7f19c80b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19e4077730 tx=0x7f19d0005fb0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.475+0000 7f19ea200700 1 -- 172.21.15.112:0/772124911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e4103750 msgr2=0x7f19e419bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:34.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.475+0000 7f19ea200700 1 --2- 172.21.15.112:0/772124911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e4103750 0x7f19e419bd70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f19d400e9c0 tx=0x7f19d400ecd0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.475+0000 7f19ea200700 1 -- 172.21.15.112:0/772124911 shutdown_connections 2024-05-08T05:53:34.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.475+0000 7f19ea200700 1 --2- 172.21.15.112:0/772124911 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f19c80b2880 0x7f19c80b4d40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.475+0000 7f19ea200700 1 --2- 172.21.15.112:0/772124911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e4108850 0x7f19e41a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.475+0000 7f19ea200700 1 --2- 172.21.15.112:0/772124911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e4103750 0x7f19e419bd70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.475+0000 7f19ea200700 1 --2- 172.21.15.112:0/772124911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19e4104100 0x7f19e419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.475+0000 7f19ea200700 1 -- 172.21.15.112:0/772124911 >> 172.21.15.112:0/772124911 conn(0x7f19e407a600 msgr2=0x7f19e4101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:34.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.475+0000 7f19ea200700 1 -- 172.21.15.112:0/772124911 shutdown_connections 2024-05-08T05:53:34.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:34.475+0000 7f19ea200700 1 -- 172.21.15.112:0/772124911 wait complete. 2024-05-08T05:53:35.298 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-08T05:53:35.299 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:53:35.378 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:35 smithi112 ceph-mon[30003]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:53:35.378 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:35 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/772124911' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T05:53:35.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:35 smithi150 ceph-mon[34315]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:53:35.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:35 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/772124911' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T05:53:35.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:35 smithi162 ceph-mon[34800]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:53:35.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:35 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/772124911' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T05:53:35.753 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:37.413 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:37 smithi112 ceph-mon[30003]: pgmap v390: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:37.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:37 smithi150 ceph-mon[34315]: pgmap v390: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:37.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:37 smithi162 ceph-mon[34800]: pgmap v390: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:37.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.885+0000 7f00d8800700 1 -- 172.21.15.112:0/1058020129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00d01056c0 msgr2=0x7f00d0105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.885+0000 7f00d8800700 1 --2- 172.21.15.112:0/1058020129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00d01056c0 0x7f00d0105b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009990 tx=0x7f00c0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.885+0000 7f00d8800700 1 -- 172.21.15.112:0/1058020129 shutdown_connections 2024-05-08T05:53:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.885+0000 7f00d8800700 1 --2- 172.21.15.112:0/1058020129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00d0106080 0x7f00d0112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.885+0000 7f00d8800700 1 --2- 172.21.15.112:0/1058020129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d010b6c0 0x7f00d010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.885+0000 7f00d8800700 1 --2- 172.21.15.112:0/1058020129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00d01056c0 0x7f00d0105b40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.885+0000 7f00d8800700 1 -- 172.21.15.112:0/1058020129 >> 172.21.15.112:0/1058020129 conn(0x7f00d01011b0 msgr2=0x7f00d01035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.885+0000 7f00d8800700 1 -- 172.21.15.112:0/1058020129 shutdown_connections 2024-05-08T05:53:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.885+0000 7f00d8800700 1 -- 172.21.15.112:0/1058020129 wait complete. 2024-05-08T05:53:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.886+0000 7f00d8800700 1 Processor -- start 2024-05-08T05:53:37.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.886+0000 7f00d8800700 1 -- start start 2024-05-08T05:53:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.886+0000 7f00d8800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d01056c0 0x7f00d019bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.886+0000 7f00d8800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00d0106080 0x7f00d019c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.886+0000 7f00d8800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00d010b6c0 0x7f00d01a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.886+0000 7f00d8800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00d0115950 con 0x7f00d010b6c0 2024-05-08T05:53:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.886+0000 7f00d8800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01157d0 con 0x7f00d0106080 2024-05-08T05:53:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.886+0000 7f00d8800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f00d0115ad0 con 0x7f00d01056c0 2024-05-08T05:53:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.887+0000 7f00d5400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00d0106080 0x7f00d019c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.887+0000 7f00d6800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00d010b6c0 0x7f00d01a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.887+0000 7f00d5e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d01056c0 0x7f00d019bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.887+0000 7f00d5400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00d0106080 0x7f00d019c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:34128/0 (socket says 172.21.15.112:34128) 2024-05-08T05:53:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.887+0000 7f00d5400700 1 -- 172.21.15.112:0/1897089443 learned_addr learned my addr 172.21.15.112:0/1897089443 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.887+0000 7f00d5400700 1 -- 172.21.15.112:0/1897089443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d01056c0 msgr2=0x7f00d019bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.887+0000 7f00d5400700 1 --2- 172.21.15.112:0/1897089443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d01056c0 0x7f00d019bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.887+0000 7f00d5400700 1 -- 172.21.15.112:0/1897089443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00d010b6c0 msgr2=0x7f00d01a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:37.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.887+0000 7f00d5400700 1 --2- 172.21.15.112:0/1897089443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00d010b6c0 0x7f00d01a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.887+0000 7f00d5400700 1 -- 172.21.15.112:0/1897089443 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d01a0dd0 con 0x7f00d0106080 2024-05-08T05:53:37.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.887+0000 7f00d5e00700 1 --2- 172.21.15.112:0/1897089443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d01056c0 0x7f00d019bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:37.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.887+0000 7f00d6800700 1 --2- 172.21.15.112:0/1897089443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00d010b6c0 0x7f00d01a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:37.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.888+0000 7f00d5400700 1 --2- 172.21.15.112:0/1897089443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00d0106080 0x7f00d019c300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f00c0000c00 tx=0x7f00c00057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:37.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.888+0000 7f00caa00700 1 -- 172.21.15.112:0/1897089443 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0027070 con 0x7f00d0106080 2024-05-08T05:53:37.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.888+0000 7f00caa00700 1 -- 172.21.15.112:0/1897089443 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c0022470 con 0x7f00d0106080 2024-05-08T05:53:37.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.888+0000 7f00caa00700 1 -- 172.21.15.112:0/1897089443 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0017650 con 0x7f00d0106080 2024-05-08T05:53:37.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.888+0000 7f00d8800700 1 -- 172.21.15.112:0/1897089443 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d01a1060 con 0x7f00d0106080 2024-05-08T05:53:37.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.888+0000 7f00d8800700 1 -- 172.21.15.112:0/1897089443 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d01a8940 con 0x7f00d0106080 2024-05-08T05:53:37.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.889+0000 7f00d8800700 1 -- 172.21.15.112:0/1897089443 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d010dc50 con 0x7f00d0106080 2024-05-08T05:53:37.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.892+0000 7f00caa00700 1 -- 172.21.15.112:0/1897089443 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f00c002a070 con 0x7f00d0106080 2024-05-08T05:53:37.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.893+0000 7f00caa00700 1 --2- 172.21.15.112:0/1897089443 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f00b80b2900 0x7f00b80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.893+0000 7f00caa00700 1 -- 172.21.15.112:0/1897089443 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f00c00e2f80 con 0x7f00d0106080 2024-05-08T05:53:37.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.893+0000 7f00d5e00700 1 --2- 172.21.15.112:0/1897089443 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f00b80b2900 0x7f00b80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.894+0000 7f00d5e00700 1 --2- 172.21.15.112:0/1897089443 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f00b80b2900 0x7f00b80b4dc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009fd0 tx=0x7f00c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:37.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:37.895+0000 7f00caa00700 1 -- 172.21.15.112:0/1897089443 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00c0013070 con 0x7f00d0106080 2024-05-08T05:53:38.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:38.182+0000 7f00d8800700 1 -- 172.21.15.112:0/1897089443 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00d006a190 con 0x7f00d0106080 2024-05-08T05:53:38.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:38.183+0000 7f00caa00700 1 -- 172.21.15.112:0/1897089443 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11891 (secure 0 0 0) 0x7f00c00aaca0 con 0x7f00d0106080 2024-05-08T05:53:38.184 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:53:38.184 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T05:51:31.368725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24511,24518]},"id":1}]} 2024-05-08T05:53:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:38.187+0000 7f00d8800700 1 -- 172.21.15.112:0/1897089443 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f00b80b2900 msgr2=0x7f00b80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:38.187+0000 7f00d8800700 1 --2- 172.21.15.112:0/1897089443 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f00b80b2900 0x7f00b80b4dc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009fd0 tx=0x7f00c4009260 comp rx=0 tx=0).stop 2024-05-08T05:53:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:38.187+0000 7f00d8800700 1 -- 172.21.15.112:0/1897089443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00d0106080 msgr2=0x7f00d019c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:38.187+0000 7f00d8800700 1 --2- 172.21.15.112:0/1897089443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00d0106080 0x7f00d019c300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f00c0000c00 tx=0x7f00c00057c0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:38.187+0000 7f00d8800700 1 -- 172.21.15.112:0/1897089443 shutdown_connections 2024-05-08T05:53:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:38.187+0000 7f00d8800700 1 --2- 172.21.15.112:0/1897089443 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f00b80b2900 0x7f00b80b4dc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:38.187+0000 7f00d8800700 1 --2- 172.21.15.112:0/1897089443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00d010b6c0 0x7f00d01a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:38.187+0000 7f00d8800700 1 --2- 172.21.15.112:0/1897089443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d01056c0 0x7f00d019bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:38.187+0000 7f00d8800700 1 --2- 172.21.15.112:0/1897089443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00d0106080 0x7f00d019c300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:38.187+0000 7f00d8800700 1 -- 172.21.15.112:0/1897089443 >> 172.21.15.112:0/1897089443 conn(0x7f00d01011b0 msgr2=0x7f00d01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:38.187+0000 7f00d8800700 1 -- 172.21.15.112:0/1897089443 shutdown_connections 2024-05-08T05:53:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:38.187+0000 7f00d8800700 1 -- 172.21.15.112:0/1897089443 wait complete. 2024-05-08T05:53:38.189 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 24 2024-05-08T05:53:38.981 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:53:39.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:39 smithi112 ceph-mon[30003]: pgmap v391: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:39.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:39 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1897089443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:39.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:39 smithi150 ceph-mon[34315]: pgmap v391: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:39.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:39 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1897089443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:39.450 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:39.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:39 smithi162 ceph-mon[34800]: pgmap v391: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:39.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:39 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1897089443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:41.271 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:41 smithi112 ceph-mon[30003]: pgmap v392: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:41.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:41 smithi150 ceph-mon[34315]: pgmap v392: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:41.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.519+0000 7f4002000700 1 -- 172.21.15.112:0/3389979823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ffc10fb60 msgr2=0x7f3ffc10ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:41.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.519+0000 7f4002000700 1 --2- 172.21.15.112:0/3389979823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ffc10fb60 0x7f3ffc10ff40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009990 tx=0x7f3fec009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.519+0000 7f4002000700 1 -- 172.21.15.112:0/3389979823 shutdown_connections 2024-05-08T05:53:41.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.519+0000 7f4002000700 1 --2- 172.21.15.112:0/3389979823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc10a490 0x7f3ffc1169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.519+0000 7f4002000700 1 --2- 172.21.15.112:0/3389979823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ffc10fb60 0x7f3ffc10ff40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.519+0000 7f4002000700 1 --2- 172.21.15.112:0/3389979823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ffc109ad0 0x7f3ffc109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.519+0000 7f4002000700 1 -- 172.21.15.112:0/3389979823 >> 172.21.15.112:0/3389979823 conn(0x7f3ffc076c70 msgr2=0x7f3ffc1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:41.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.519+0000 7f4002000700 1 -- 172.21.15.112:0/3389979823 shutdown_connections 2024-05-08T05:53:41.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.519+0000 7f4002000700 1 -- 172.21.15.112:0/3389979823 wait complete. 2024-05-08T05:53:41.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.520+0000 7f4002000700 1 Processor -- start 2024-05-08T05:53:41.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.520+0000 7f4002000700 1 -- start start 2024-05-08T05:53:41.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.521+0000 7f4002000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ffc109ad0 0x7f3ffc1a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:41.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.521+0000 7f4002000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc10a490 0x7f3ffc1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:41.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.521+0000 7f4002000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ffc10fb60 0x7f3ffc1a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.521+0000 7f4002000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc119d20 con 0x7f3ffc10fb60 2024-05-08T05:53:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.521+0000 7f4002000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc119ba0 con 0x7f3ffc10a490 2024-05-08T05:53:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.521+0000 7f4002000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc119ea0 con 0x7f3ffc109ad0 2024-05-08T05:53:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.521+0000 7f3ffbe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ffc10fb60 0x7f3ffc1a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.521+0000 7f3ffaa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc10a490 0x7f3ffc1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.521+0000 7f3ffbe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ffc10fb60 0x7f3ffc1a4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60098/0 (socket says 172.21.15.112:60098) 2024-05-08T05:53:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.521+0000 7f3ffb400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ffc109ad0 0x7f3ffc1a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.521+0000 7f3ffbe00700 1 -- 172.21.15.112:0/2894237015 learned_addr learned my addr 172.21.15.112:0/2894237015 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.521+0000 7f3ffbe00700 1 -- 172.21.15.112:0/2894237015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ffc109ad0 msgr2=0x7f3ffc1a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.521+0000 7f3ffbe00700 1 --2- 172.21.15.112:0/2894237015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ffc109ad0 0x7f3ffc1a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.521+0000 7f3ffbe00700 1 -- 172.21.15.112:0/2894237015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc10a490 msgr2=0x7f3ffc1a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:41.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.521+0000 7f3ffbe00700 1 --2- 172.21.15.112:0/2894237015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc10a490 0x7f3ffc1a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.522+0000 7f3ffbe00700 1 -- 172.21.15.112:0/2894237015 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc1a51b0 con 0x7f3ffc10fb60 2024-05-08T05:53:41.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.522+0000 7f3ffaa00700 1 --2- 172.21.15.112:0/2894237015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc10a490 0x7f3ffc1a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:41.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.522+0000 7f3ffbe00700 1 --2- 172.21.15.112:0/2894237015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ffc10fb60 0x7f3ffc1a4a70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400ef30 tx=0x7f3ff400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:41.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.522+0000 7f3ff3e00700 1 -- 172.21.15.112:0/2894237015 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff40092e0 con 0x7f3ffc10fb60 2024-05-08T05:53:41.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.522+0000 7f4002000700 1 -- 172.21.15.112:0/2894237015 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ffc1a5440 con 0x7f3ffc10fb60 2024-05-08T05:53:41.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.522+0000 7f3ff3e00700 1 -- 172.21.15.112:0/2894237015 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff4004bb0 con 0x7f3ffc10fb60 2024-05-08T05:53:41.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.522+0000 7f3ff3e00700 1 -- 172.21.15.112:0/2894237015 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4004020 con 0x7f3ffc10fb60 2024-05-08T05:53:41.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.522+0000 7f4002000700 1 -- 172.21.15.112:0/2894237015 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ffc1acd20 con 0x7f3ffc10fb60 2024-05-08T05:53:41.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.524+0000 7f4002000700 1 -- 172.21.15.112:0/2894237015 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ffc112060 con 0x7f3ffc10fb60 2024-05-08T05:53:41.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.528+0000 7f3ff3e00700 1 -- 172.21.15.112:0/2894237015 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3ff401b030 con 0x7f3ffc10fb60 2024-05-08T05:53:41.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.529+0000 7f3ff3e00700 1 --2- 172.21.15.112:0/2894237015 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3fe00b2880 0x7f3fe00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:41.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.529+0000 7f3ff3e00700 1 -- 172.21.15.112:0/2894237015 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f3ff40e2700 con 0x7f3ffc10fb60 2024-05-08T05:53:41.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.529+0000 7f3ffb400700 1 --2- 172.21.15.112:0/2894237015 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3fe00b2880 0x7f3fe00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:41.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.530+0000 7f3ffb400700 1 --2- 172.21.15.112:0/2894237015 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3fe00b2880 0x7f3fe00b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009750 tx=0x7f3fec00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:41.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.532+0000 7f3ff3e00700 1 -- 172.21.15.112:0/2894237015 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ff4022020 con 0x7f3ffc10fb60 2024-05-08T05:53:41.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:41 smithi162 ceph-mon[34800]: pgmap v392: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:41.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.809+0000 7f4002000700 1 -- 172.21.15.112:0/2894237015 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ffc1a1500 con 0x7f3ffc10fb60 2024-05-08T05:53:41.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.811+0000 7f3ff3e00700 1 -- 172.21.15.112:0/2894237015 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11891 (secure 0 0 0) 0x7f3ff4010040 con 0x7f3ffc10fb60 2024-05-08T05:53:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:53:41.812 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T05:51:31.368725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24511,24518]},"id":1}]} 2024-05-08T05:53:41.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.814+0000 7f4002000700 1 -- 172.21.15.112:0/2894237015 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3fe00b2880 msgr2=0x7f3fe00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:41.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.814+0000 7f4002000700 1 --2- 172.21.15.112:0/2894237015 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3fe00b2880 0x7f3fe00b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009750 tx=0x7f3fec00b260 comp rx=0 tx=0).stop 2024-05-08T05:53:41.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.814+0000 7f4002000700 1 -- 172.21.15.112:0/2894237015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ffc10fb60 msgr2=0x7f3ffc1a4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:41.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.814+0000 7f4002000700 1 --2- 172.21.15.112:0/2894237015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ffc10fb60 0x7f3ffc1a4a70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400ef30 tx=0x7f3ff400c3a0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.814+0000 7f4002000700 1 -- 172.21.15.112:0/2894237015 shutdown_connections 2024-05-08T05:53:41.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.814+0000 7f4002000700 1 --2- 172.21.15.112:0/2894237015 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3fe00b2880 0x7f3fe00b4d40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.814+0000 7f4002000700 1 --2- 172.21.15.112:0/2894237015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ffc10fb60 0x7f3ffc1a4a70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.814+0000 7f4002000700 1 --2- 172.21.15.112:0/2894237015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ffc109ad0 0x7f3ffc1a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.814+0000 7f4002000700 1 --2- 172.21.15.112:0/2894237015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc10a490 0x7f3ffc1a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.814+0000 7f4002000700 1 -- 172.21.15.112:0/2894237015 >> 172.21.15.112:0/2894237015 conn(0x7f3ffc076c70 msgr2=0x7f3ffc0772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:41.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.815+0000 7f4002000700 1 -- 172.21.15.112:0/2894237015 shutdown_connections 2024-05-08T05:53:41.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:41.815+0000 7f4002000700 1 -- 172.21.15.112:0/2894237015 wait complete. 2024-05-08T05:53:41.816 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 24 2024-05-08T05:53:42.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:42 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/2894237015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:42.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:42 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2894237015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:42.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:42 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/2894237015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:42.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:53:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:43.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:43 smithi150 ceph-mon[34315]: pgmap v393: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:43.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:43 smithi112 ceph-mon[30003]: pgmap v393: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:43.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:43 smithi162 ceph-mon[34800]: pgmap v393: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:45.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.278+0000 7f63b6200700 1 -- 172.21.15.112:0/775940765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b00761f0 msgr2=0x7f63b010f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:45.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.278+0000 7f63b6200700 1 --2- 172.21.15.112:0/775940765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b00761f0 0x7f63b010f6c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f639c009990 tx=0x7f639c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:45.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.278+0000 7f63b6200700 1 -- 172.21.15.112:0/775940765 shutdown_connections 2024-05-08T05:53:45.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.278+0000 7f63b6200700 1 --2- 172.21.15.112:0/775940765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63b010fc00 0x7f63b0112070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:45.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.278+0000 7f63b6200700 1 --2- 172.21.15.112:0/775940765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63b00758d0 0x7f63b0075cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:45.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.278+0000 7f63b6200700 1 --2- 172.21.15.112:0/775940765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b00761f0 0x7f63b010f6c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:45.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.278+0000 7f63b6200700 1 -- 172.21.15.112:0/775940765 >> 172.21.15.112:0/775940765 conn(0x7f63b00feeb0 msgr2=0x7f63b01012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:45.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.278+0000 7f63b6200700 1 -- 172.21.15.112:0/775940765 shutdown_connections 2024-05-08T05:53:45.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.278+0000 7f63b6200700 1 -- 172.21.15.112:0/775940765 wait complete. 2024-05-08T05:53:45.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.279+0000 7f63b6200700 1 Processor -- start 2024-05-08T05:53:45.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.279+0000 7f63b6200700 1 -- start start 2024-05-08T05:53:45.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.280+0000 7f63b6200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63b00758d0 0x7f63b019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:45.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.280+0000 7f63b6200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63b00761f0 0x7f63b019c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:45.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.280+0000 7f63b6200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b010fc00 0x7f63b01a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:45.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.280+0000 7f63b6200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01157f0 con 0x7f63b010fc00 2024-05-08T05:53:45.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.280+0000 7f63b6200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0115670 con 0x7f63b00761f0 2024-05-08T05:53:45.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.280+0000 7f63b6200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0115970 con 0x7f63b00758d0 2024-05-08T05:53:45.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.280+0000 7f63b5800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b010fc00 0x7f63b01a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:45.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.280+0000 7f63b5800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b010fc00 0x7f63b01a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48124/0 (socket says 172.21.15.112:48124) 2024-05-08T05:53:45.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.280+0000 7f63afe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63b00761f0 0x7f63b019c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:45.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.280+0000 7f63b5800700 1 -- 172.21.15.112:0/4043754149 learned_addr learned my addr 172.21.15.112:0/4043754149 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:45.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.280+0000 7f63b4e00700 1 --2- 172.21.15.112:0/4043754149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63b00758d0 0x7f63b019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:45.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.281+0000 7f63b4e00700 1 -- 172.21.15.112:0/4043754149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63b00761f0 msgr2=0x7f63b019c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:45.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.281+0000 7f63b4e00700 1 --2- 172.21.15.112:0/4043754149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63b00761f0 0x7f63b019c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:45.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.281+0000 7f63b4e00700 1 -- 172.21.15.112:0/4043754149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b010fc00 msgr2=0x7f63b01a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:45.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.281+0000 7f63b4e00700 1 --2- 172.21.15.112:0/4043754149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b010fc00 0x7f63b01a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:45.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.281+0000 7f63b4e00700 1 -- 172.21.15.112:0/4043754149 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b01a0d30 con 0x7f63b00758d0 2024-05-08T05:53:45.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.281+0000 7f63b5800700 1 --2- 172.21.15.112:0/4043754149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b010fc00 0x7f63b01a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:45.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.281+0000 7f63afe00700 1 --2- 172.21.15.112:0/4043754149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63b00761f0 0x7f63b019c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:45.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.281+0000 7f63b4e00700 1 --2- 172.21.15.112:0/4043754149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63b00758d0 0x7f63b019bd20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f63a400e9f0 tx=0x7f63a400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:45.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.281+0000 7f63ad600700 1 -- 172.21.15.112:0/4043754149 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4004220 con 0x7f63b00758d0 2024-05-08T05:53:45.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.281+0000 7f63b6200700 1 -- 172.21.15.112:0/4043754149 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b01a1020 con 0x7f63b00758d0 2024-05-08T05:53:45.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.281+0000 7f63ad600700 1 -- 172.21.15.112:0/4043754149 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a40043c0 con 0x7f63b00758d0 2024-05-08T05:53:45.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.281+0000 7f63ad600700 1 -- 172.21.15.112:0/4043754149 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4005020 con 0x7f63b00758d0 2024-05-08T05:53:45.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.281+0000 7f63b6200700 1 -- 172.21.15.112:0/4043754149 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b01a88a0 con 0x7f63b00758d0 2024-05-08T05:53:45.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.282+0000 7f63b6200700 1 -- 172.21.15.112:0/4043754149 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b006a190 con 0x7f63b00758d0 2024-05-08T05:53:45.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.285+0000 7f63ad600700 1 -- 172.21.15.112:0/4043754149 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f63a4004750 con 0x7f63b00758d0 2024-05-08T05:53:45.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.286+0000 7f63ad600700 1 --2- 172.21.15.112:0/4043754149 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f63980b2900 0x7f63980b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.286+0000 7f63ad600700 1 -- 172.21.15.112:0/4043754149 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f63a40e2160 con 0x7f63b00758d0 2024-05-08T05:53:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.286+0000 7f63afe00700 1 --2- 172.21.15.112:0/4043754149 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f63980b2900 0x7f63980b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:45.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.287+0000 7f63afe00700 1 --2- 172.21.15.112:0/4043754149 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f63980b2900 0x7f63980b4dc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63b01110f0 tx=0x7f639c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:45.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.291+0000 7f63ad600700 1 -- 172.21.15.112:0/4043754149 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f63a40a9df0 con 0x7f63b00758d0 2024-05-08T05:53:45.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:45 smithi150 ceph-mon[34315]: pgmap v394: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:53:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.535+0000 7f63b6200700 1 -- 172.21.15.112:0/4043754149 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63b019d080 con 0x7f63b00758d0 2024-05-08T05:53:45.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:45 smithi112 ceph-mon[30003]: pgmap v394: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:53:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.537+0000 7f63ad600700 1 -- 172.21.15.112:0/4043754149 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22829 (secure 0 0 0) 0x7f63a40b0e20 con 0x7f63b00758d0 2024-05-08T05:53:45.538 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:53:45.538 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":84,"fsid":"7a07ff60-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:48.498871+0000","modified":"2024-05-08T05:53:05.251932+0000","last_up_change":"2024-05-08T05:49:54.039380+0000","last_in_change":"2024-05-08T05:49:34.443850+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:47.485105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:50:58.918909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.175022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b378542-1bbe-4614-8805-8a6b875ada55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6803","nonce":3419091820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6805","nonce":3419091820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6809","nonce":3419091820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6807","nonce":3419091820}]},"public_addr":"172.21.15.112:6803/3419091820","cluster_addr":"172.21.15.112:6805/3419091820","heartbeat_back_addr":"172.21.15.112:6809/3419091820","heartbeat_front_addr":"172.21.15.112:6807/3419091820","state":["exists","up"]},{"osd":1,"uuid":"826ef036-2db7-470e-9956-d9a58b9181e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6801","nonce":2728419797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6803","nonce":2728419797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6807","nonce":2728419797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6805","nonce":2728419797}]},"public_addr":"172.21.15.150:6801/2728419797","cluster_addr":"172.21.15.150:6803/2728419797","heartbeat_back_addr":"172.21.15.150:6807/2728419797","heartbeat_front_addr":"172.21.15.150:6805/2728419797","state":["exists","up"]},{"osd":2,"uuid":"32e710e7-2a87-4863-bc7a-906d31aa4ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6801","nonce":1556203215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6803","nonce":1556203215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6807","nonce":1556203215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6805","nonce":1556203215}]},"public_addr":"172.21.15.162:6801/1556203215","cluster_addr":"172.21.15.162:6803/1556203215","heartbeat_back_addr":"172.21.15.162:6807/1556203215","heartbeat_front_addr":"172.21.15.162:6805/1556203215","state":["exists","up"]},{"osd":3,"uuid":"ba1d4a5f-6a95-4b9e-8a48-8524a899b004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6811","nonce":3325718755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6813","nonce":3325718755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6817","nonce":3325718755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6815","nonce":3325718755}]},"public_addr":"172.21.15.112:6811/3325718755","cluster_addr":"172.21.15.112:6813/3325718755","heartbeat_back_addr":"172.21.15.112:6817/3325718755","heartbeat_front_addr":"172.21.15.112:6815/3325718755","state":["exists","up"]},{"osd":4,"uuid":"cb4f1a42-8de8-4b50-b3bc-602de9f4803a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6809","nonce":1548914683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6811","nonce":1548914683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6815","nonce":1548914683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6813","nonce":1548914683}]},"public_addr":"172.21.15.150:6809/1548914683","cluster_addr":"172.21.15.150:6811/1548914683","heartbeat_back_addr":"172.21.15.150:6815/1548914683","heartbeat_front_addr":"172.21.15.150:6813/1548914683","state":["exists","up"]},{"osd":5,"uuid":"a26c03b8-050c-4a2c-8f23-5801fffb7c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6809","nonce":409836750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6811","nonce":409836750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6815","nonce":409836750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6813","nonce":409836750}]},"public_addr":"172.21.15.162:6809/409836750","cluster_addr":"172.21.15.162:6811/409836750","heartbeat_back_addr":"172.21.15.162:6815/409836750","heartbeat_front_addr":"172.21.15.162:6813/409836750","state":["exists","up"]},{"osd":6,"uuid":"8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6819","nonce":1098984186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6821","nonce":1098984186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6825","nonce":1098984186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6823","nonce":1098984186}]},"public_addr":"172.21.15.112:6819/1098984186","cluster_addr":"172.21.15.112:6821/1098984186","heartbeat_back_addr":"172.21.15.112:6825/1098984186","heartbeat_front_addr":"172.21.15.112:6823/1098984186","state":["exists","up"]},{"osd":7,"uuid":"af3629b4-9bed-4669-97a6-48deba503a96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6817","nonce":4009836564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6819","nonce":4009836564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6823","nonce":4009836564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6821","nonce":4009836564}]},"public_addr":"172.21.15.150:6817/4009836564","cluster_addr":"172.21.15.150:6819/4009836564","heartbeat_back_addr":"172.21.15.150:6823/4009836564","heartbeat_front_addr":"172.21.15.150:6821/4009836564","state":["exists","up"]},{"osd":8,"uuid":"9333e87d-dd78-4467-bbde-c2b44f4ea07f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6817","nonce":852791606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6819","nonce":852791606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6823","nonce":852791606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6821","nonce":852791606}]},"public_addr":"172.21.15.162:6817/852791606","cluster_addr":"172.21.15.162:6819/852791606","heartbeat_back_addr":"172.21.15.162:6823/852791606","heartbeat_front_addr":"172.21.15.162:6821/852791606","state":["exists","up"]},{"osd":9,"uuid":"09723581-1995-42e0-8db7-776ea98a918c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6827","nonce":2186764674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6829","nonce":2186764674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6833","nonce":2186764674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6831","nonce":2186764674}]},"public_addr":"172.21.15.112:6827/2186764674","cluster_addr":"172.21.15.112:6829/2186764674","heartbeat_back_addr":"172.21.15.112:6833/2186764674","heartbeat_front_addr":"172.21.15.112:6831/2186764674","state":["exists","up"]},{"osd":10,"uuid":"34e08e3e-1231-4899-a5b3-76d449105bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6825","nonce":2838995474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6827","nonce":2838995474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6831","nonce":2838995474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6829","nonce":2838995474}]},"public_addr":"172.21.15.150:6825/2838995474","cluster_addr":"172.21.15.150:6827/2838995474","heartbeat_back_addr":"172.21.15.150:6831/2838995474","heartbeat_front_addr":"172.21.15.150:6829/2838995474","state":["exists","up"]},{"osd":11,"uuid":"6fc34ec0-78ca-403c-acb7-b29d83ff64e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6825","nonce":3702614331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6827","nonce":3702614331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6831","nonce":3702614331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6829","nonce":3702614331}]},"public_addr":"172.21.15.162:6825/3702614331","cluster_addr":"172.21.15.162:6827/3702614331","heartbeat_back_addr":"172.21.15.162:6831/3702614331","heartbeat_front_addr":"172.21.15.162:6829/3702614331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:54.518702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:19.939543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:44.425586+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:16.126471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:41.696071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:05.517405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:38.549318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:03.799649+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:28.007282+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:01.395459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:27.963637+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:52.181212+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3667148497":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6800/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3425327158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1404611205":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3765334043":"2024-05-09T05:42:29.819307+0000","172.21.15.112:6800/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/2814303790":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1091751583":"2024-05-09T05:42:29.819307+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:53:45.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.541+0000 7f63b6200700 1 -- 172.21.15.112:0/4043754149 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f63980b2900 msgr2=0x7f63980b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:45.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.541+0000 7f63b6200700 1 --2- 172.21.15.112:0/4043754149 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f63980b2900 0x7f63980b4dc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63b01110f0 tx=0x7f639c00b260 comp rx=0 tx=0).stop 2024-05-08T05:53:45.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.541+0000 7f63b6200700 1 -- 172.21.15.112:0/4043754149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63b00758d0 msgr2=0x7f63b019bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.541+0000 7f63b6200700 1 --2- 172.21.15.112:0/4043754149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63b00758d0 0x7f63b019bd20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f63a400e9f0 tx=0x7f63a400ed00 comp rx=0 tx=0).stop 2024-05-08T05:53:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.541+0000 7f63b6200700 1 -- 172.21.15.112:0/4043754149 shutdown_connections 2024-05-08T05:53:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.541+0000 7f63b6200700 1 --2- 172.21.15.112:0/4043754149 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f63980b2900 0x7f63980b4dc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.541+0000 7f63b6200700 1 --2- 172.21.15.112:0/4043754149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63b010fc00 0x7f63b01a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.541+0000 7f63b6200700 1 --2- 172.21.15.112:0/4043754149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63b00758d0 0x7f63b019bd20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.541+0000 7f63b6200700 1 --2- 172.21.15.112:0/4043754149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63b00761f0 0x7f63b019c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.541+0000 7f63b6200700 1 -- 172.21.15.112:0/4043754149 >> 172.21.15.112:0/4043754149 conn(0x7f63b00feeb0 msgr2=0x7f63b00ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.541+0000 7f63b6200700 1 -- 172.21.15.112:0/4043754149 shutdown_connections 2024-05-08T05:53:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:45.541+0000 7f63b6200700 1 -- 172.21.15.112:0/4043754149 wait complete. 2024-05-08T05:53:45.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:45 smithi162 ceph-mon[34800]: pgmap v394: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:53:46.232 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:46 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/4043754149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:53:46.426 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:53:46.432 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:46 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/4043754149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:53:46.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:46 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/4043754149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:53:46.880 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:47.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:47 smithi112 ceph-mon[30003]: pgmap v395: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:47 smithi150 ceph-mon[34315]: pgmap v395: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:47.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:47 smithi162 ceph-mon[34800]: pgmap v395: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:49.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.299+0000 7fddc1200700 1 -- 172.21.15.112:0/1276006933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddbc06b180 msgr2=0x7fddbc06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.299+0000 7fddc1200700 1 --2- 172.21.15.112:0/1276006933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddbc06b180 0x7fddbc06b600 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009990 tx=0x7fdda8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.299+0000 7fddc1200700 1 -- 172.21.15.112:0/1276006933 shutdown_connections 2024-05-08T05:53:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.299+0000 7fddc1200700 1 --2- 172.21.15.112:0/1276006933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddbc06bb40 0x7fddbc110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.299+0000 7fddc1200700 1 --2- 172.21.15.112:0/1276006933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddbc109510 0x7fddbc1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.299+0000 7fddc1200700 1 --2- 172.21.15.112:0/1276006933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddbc06b180 0x7fddbc06b600 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.299+0000 7fddc1200700 1 -- 172.21.15.112:0/1276006933 >> 172.21.15.112:0/1276006933 conn(0x7fddbc079350 msgr2=0x7fddbc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.299+0000 7fddc1200700 1 -- 172.21.15.112:0/1276006933 shutdown_connections 2024-05-08T05:53:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.299+0000 7fddc1200700 1 -- 172.21.15.112:0/1276006933 wait complete. 2024-05-08T05:53:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.300+0000 7fddc1200700 1 Processor -- start 2024-05-08T05:53:49.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.300+0000 7fddc1200700 1 -- start start 2024-05-08T05:53:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.301+0000 7fddc1200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddbc06b180 0x7fddbc19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.301+0000 7fddc1200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddbc06bb40 0x7fddbc19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.301+0000 7fddc1200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddbc109510 0x7fddbc1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.301+0000 7fddc1200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc113690 con 0x7fddbc06b180 2024-05-08T05:53:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.301+0000 7fddc1200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc113510 con 0x7fddbc06bb40 2024-05-08T05:53:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.301+0000 7fddc1200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc113810 con 0x7fddbc109510 2024-05-08T05:53:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.301+0000 7fddb9e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddbc06b180 0x7fddbc19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.301+0000 7fddb9e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddbc06b180 0x7fddbc19f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48140/0 (socket says 172.21.15.112:48140) 2024-05-08T05:53:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.301+0000 7fddb9e00700 1 -- 172.21.15.112:0/952030881 learned_addr learned my addr 172.21.15.112:0/952030881 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.301+0000 7fddb9400700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddbc06bb40 0x7fddbc19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.301+0000 7fddba800700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddbc109510 0x7fddbc1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.302+0000 7fddb9400700 1 -- 172.21.15.112:0/952030881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddbc109510 msgr2=0x7fddbc1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:49.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.302+0000 7fddb9400700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddbc109510 0x7fddbc1999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:49.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.302+0000 7fddb9400700 1 -- 172.21.15.112:0/952030881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddbc06b180 msgr2=0x7fddbc19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:49.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.302+0000 7fddb9400700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddbc06b180 0x7fddbc19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:49.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.302+0000 7fddb9400700 1 -- 172.21.15.112:0/952030881 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc19a150 con 0x7fddbc06bb40 2024-05-08T05:53:49.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.302+0000 7fddb9e00700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddbc06b180 0x7fddbc19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:49.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.302+0000 7fddba800700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddbc109510 0x7fddbc1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:49.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.302+0000 7fddb9400700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddbc06bb40 0x7fddbc19fd20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdda80098e0 tx=0x7fdda80057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:49.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.302+0000 7fddb2a00700 1 -- 172.21.15.112:0/952030881 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8027070 con 0x7fddbc06bb40 2024-05-08T05:53:49.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.302+0000 7fddc1200700 1 -- 172.21.15.112:0/952030881 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddbc19a3e0 con 0x7fddbc06bb40 2024-05-08T05:53:49.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.302+0000 7fddb2a00700 1 -- 172.21.15.112:0/952030881 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdda8022470 con 0x7fddbc06bb40 2024-05-08T05:53:49.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.302+0000 7fddb2a00700 1 -- 172.21.15.112:0/952030881 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8017650 con 0x7fddbc06bb40 2024-05-08T05:53:49.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.302+0000 7fddc1200700 1 -- 172.21.15.112:0/952030881 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddbc1a66b0 con 0x7fddbc06bb40 2024-05-08T05:53:49.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.303+0000 7fddc1200700 1 -- 172.21.15.112:0/952030881 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddbc06a190 con 0x7fddbc06bb40 2024-05-08T05:53:49.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.306+0000 7fddb2a00700 1 -- 172.21.15.112:0/952030881 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fdda802a070 con 0x7fddbc06bb40 2024-05-08T05:53:49.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.307+0000 7fddb2a00700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fdda00b2900 0x7fdda00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:49.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.307+0000 7fddb2a00700 1 -- 172.21.15.112:0/952030881 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fdda80e3080 con 0x7fddbc06bb40 2024-05-08T05:53:49.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.308+0000 7fddb9e00700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fdda00b2900 0x7fdda00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:49.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.308+0000 7fddb9e00700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fdda00b2900 0x7fdda00b4dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fddbc100a70 tx=0x7fddac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.309+0000 7fddb2a00700 1 -- 172.21.15.112:0/952030881 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdda8031080 con 0x7fddbc06bb40 2024-05-08T05:53:49.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:49 smithi150 ceph-mon[34315]: pgmap v396: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:49.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:49 smithi112 ceph-mon[30003]: pgmap v396: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:49.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:49 smithi162 ceph-mon[34800]: pgmap v396: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.590+0000 7fddc1200700 1 -- 172.21.15.112:0/952030881 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddbc02e510 con 0x7fddbc06bb40 2024-05-08T05:53:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.592+0000 7fddb2a00700 1 -- 172.21.15.112:0/952030881 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11891 (secure 0 0 0) 0x7fdda80a4e20 con 0x7fddbc06bb40 2024-05-08T05:53:49.593 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:53:49.593 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T05:51:31.368725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24511,24518]},"id":1}]} 2024-05-08T05:53:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.595+0000 7fddc1200700 1 -- 172.21.15.112:0/952030881 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fdda00b2900 msgr2=0x7fdda00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.595+0000 7fddc1200700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fdda00b2900 0x7fdda00b4dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fddbc100a70 tx=0x7fddac009260 comp rx=0 tx=0).stop 2024-05-08T05:53:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.595+0000 7fddc1200700 1 -- 172.21.15.112:0/952030881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddbc06bb40 msgr2=0x7fddbc19fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.595+0000 7fddc1200700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddbc06bb40 0x7fddbc19fd20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdda80098e0 tx=0x7fdda80057c0 comp rx=0 tx=0).stop 2024-05-08T05:53:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.595+0000 7fddc1200700 1 -- 172.21.15.112:0/952030881 shutdown_connections 2024-05-08T05:53:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.595+0000 7fddc1200700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fdda00b2900 0x7fdda00b4dc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.595+0000 7fddc1200700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddbc109510 0x7fddbc1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.595+0000 7fddc1200700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddbc06b180 0x7fddbc19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.595+0000 7fddc1200700 1 --2- 172.21.15.112:0/952030881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddbc06bb40 0x7fddbc19fd20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.595+0000 7fddc1200700 1 -- 172.21.15.112:0/952030881 >> 172.21.15.112:0/952030881 conn(0x7fddbc079350 msgr2=0x7fddbc1010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.596+0000 7fddc1200700 1 -- 172.21.15.112:0/952030881 shutdown_connections 2024-05-08T05:53:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:49.596+0000 7fddc1200700 1 -- 172.21.15.112:0/952030881 wait complete. 2024-05-08T05:53:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 24 2024-05-08T05:53:50.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:50 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/952030881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:50.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T05:51:07.312542+0000', 'modified': '2024-05-08T05:51:31.368725+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24529, 'mds_1': 24518, 'mds_2': 24511}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6837/4103977076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 4103977076}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 4103977076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.162:6835/3060051414', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6834', 'nonce': 3060051414}, {'type': 'v1', 'addr': '172.21.15.162:6835', 'nonce': 3060051414}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.162:6837/3092620994', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 3092620994}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 3092620994}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.162:6839/539986796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6838', 'nonce': 539986796}, {'type': 'v1', 'addr': '172.21.15.162:6839', 'nonce': 539986796}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'b', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.150:6839/1284437777', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 1284437777}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 1284437777}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24529': {'gid': 24529, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.112:6841/2816461111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6840', 'nonce': 2816461111}, {'type': 'v1', 'addr': '172.21.15.112:6841', 'nonce': 2816461111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24529, 'qdb_cluster': [24529, 24511, 24518]} 2024-05-08T05:53:50.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T05:53:50.503 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-08T05:53:50.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:50 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/952030881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:50.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:50 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/952030881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:50.969 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:51.377 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:51 smithi112 ceph-mon[30003]: pgmap v397: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:51.468 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:51 smithi162 ceph-mon[34800]: pgmap v397: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:51.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:51 smithi150 ceph-mon[34315]: pgmap v397: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:53.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.301+0000 7f61e4600700 1 -- 172.21.15.112:0/208482727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61dc1056c0 msgr2=0x7f61dc105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.301+0000 7f61e4600700 1 --2- 172.21.15.112:0/208482727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61dc1056c0 0x7f61dc105b40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009990 tx=0x7f61c8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.302+0000 7f61e4600700 1 -- 172.21.15.112:0/208482727 shutdown_connections 2024-05-08T05:53:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.302+0000 7f61e4600700 1 --2- 172.21.15.112:0/208482727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61dc106080 0x7f61dc112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.302+0000 7f61e4600700 1 --2- 172.21.15.112:0/208482727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61dc10b6c0 0x7f61dc10baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.302+0000 7f61e4600700 1 --2- 172.21.15.112:0/208482727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61dc1056c0 0x7f61dc105b40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.302+0000 7f61e4600700 1 -- 172.21.15.112:0/208482727 >> 172.21.15.112:0/208482727 conn(0x7f61dc1011b0 msgr2=0x7f61dc1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.302+0000 7f61e4600700 1 -- 172.21.15.112:0/208482727 shutdown_connections 2024-05-08T05:53:53.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.302+0000 7f61e4600700 1 -- 172.21.15.112:0/208482727 wait complete. 2024-05-08T05:53:53.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.303+0000 7f61e4600700 1 Processor -- start 2024-05-08T05:53:53.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.303+0000 7f61e4600700 1 -- start start 2024-05-08T05:53:53.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.303+0000 7f61e4600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61dc1056c0 0x7f61dc19be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.303+0000 7f61e4600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61dc106080 0x7f61dc19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.303+0000 7f61e4600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61dc10b6c0 0x7f61dc1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.303+0000 7f61e4600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc115950 con 0x7f61dc1056c0 2024-05-08T05:53:53.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.303+0000 7f61e4600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1157d0 con 0x7f61dc10b6c0 2024-05-08T05:53:53.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.303+0000 7f61e4600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc115ad0 con 0x7f61dc106080 2024-05-08T05:53:53.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.304+0000 7f61e1200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61dc106080 0x7f61dc19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.304+0000 7f61e1200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61dc106080 0x7f61dc19c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:33080/0 (socket says 172.21.15.112:33080) 2024-05-08T05:53:53.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.304+0000 7f61e1200700 1 -- 172.21.15.112:0/706323577 learned_addr learned my addr 172.21.15.112:0/706323577 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:53.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.304+0000 7f61e2600700 1 --2- 172.21.15.112:0/706323577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61dc10b6c0 0x7f61dc1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.304+0000 7f61e1c00700 1 --2- 172.21.15.112:0/706323577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61dc1056c0 0x7f61dc19be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.304+0000 7f61e1200700 1 -- 172.21.15.112:0/706323577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61dc10b6c0 msgr2=0x7f61dc1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:53.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.304+0000 7f61e1200700 1 --2- 172.21.15.112:0/706323577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61dc10b6c0 0x7f61dc1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.304+0000 7f61e1200700 1 -- 172.21.15.112:0/706323577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61dc1056c0 msgr2=0x7f61dc19be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:53.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.304+0000 7f61e1200700 1 --2- 172.21.15.112:0/706323577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61dc1056c0 0x7f61dc19be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.304+0000 7f61e1200700 1 -- 172.21.15.112:0/706323577 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc1a0dc0 con 0x7f61dc106080 2024-05-08T05:53:53.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.304+0000 7f61e2600700 1 --2- 172.21.15.112:0/706323577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61dc10b6c0 0x7f61dc1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:53.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.304+0000 7f61e1200700 1 --2- 172.21.15.112:0/706323577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61dc106080 0x7f61dc19c340 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f61c8000c00 tx=0x7f61c80057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:53.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.305+0000 7f61da000700 1 -- 172.21.15.112:0/706323577 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8027070 con 0x7f61dc106080 2024-05-08T05:53:53.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.305+0000 7f61e4600700 1 -- 172.21.15.112:0/706323577 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc1a1050 con 0x7f61dc106080 2024-05-08T05:53:53.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.305+0000 7f61da000700 1 -- 172.21.15.112:0/706323577 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c8022470 con 0x7f61dc106080 2024-05-08T05:53:53.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.305+0000 7f61da000700 1 -- 172.21.15.112:0/706323577 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8017650 con 0x7f61dc106080 2024-05-08T05:53:53.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.305+0000 7f61e4600700 1 -- 172.21.15.112:0/706323577 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc1a8930 con 0x7f61dc106080 2024-05-08T05:53:53.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.310+0000 7f61da000700 1 -- 172.21.15.112:0/706323577 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f61c802a070 con 0x7f61dc106080 2024-05-08T05:53:53.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.311+0000 7f61eef21b80 1 Processor -- start 2024-05-08T05:53:53.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.311+0000 7f61eef21b80 1 -- start start 2024-05-08T05:53:53.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.311+0000 7f61eef21b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b135010 0x556d2b1353f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.311+0000 7f61eef21b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b139440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.311+0000 7f61eef21b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b139980 0x556d2b139de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.311+0000 7f61eef21b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556d2b0879d0 con 0x556d2b135010 2024-05-08T05:53:53.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.311+0000 7f61eef21b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556d2b025710 con 0x556d2b139980 2024-05-08T05:53:53.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.311+0000 7f61eef21b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x556d2b083990 con 0x556d2b135930 2024-05-08T05:53:53.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.312+0000 7f61e1c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b139440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.312+0000 7f61e1c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b139440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:33086/0 (socket says 172.21.15.112:33086) 2024-05-08T05:53:53.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.312+0000 7f61e1c00700 1 -- 172.21.15.112:0/3943619374 learned_addr learned my addr 172.21.15.112:0/3943619374 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:53.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.312+0000 7f61e2600700 1 --2- 172.21.15.112:0/3943619374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b135010 0x556d2b1353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.312+0000 7f61e1200700 1 --2- 172.21.15.112:0/3943619374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b139980 0x556d2b139de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.312+0000 7f61da000700 1 --2- 172.21.15.112:0/706323577 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f61c40b2880 0x7f61c40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.312+0000 7f61e1c00700 1 -- 172.21.15.112:0/3943619374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b139980 msgr2=0x556d2b139de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:53.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.312+0000 7f61e1c00700 1 --2- 172.21.15.112:0/3943619374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b139980 0x556d2b139de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.312+0000 7f61e1c00700 1 -- 172.21.15.112:0/3943619374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b135010 msgr2=0x556d2b1353f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:53.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.312+0000 7f61e1c00700 1 --2- 172.21.15.112:0/3943619374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b135010 0x556d2b1353f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.312+0000 7f61e1c00700 1 -- 172.21.15.112:0/3943619374 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2afd4880 con 0x556d2b135930 2024-05-08T05:53:53.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.312+0000 7f61e2600700 1 --2- 172.21.15.112:0/3943619374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b135010 0x556d2b1353f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:53:53.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.312+0000 7f61da000700 1 -- 172.21.15.112:0/706323577 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f61c80e2f30 con 0x7f61dc106080 2024-05-08T05:53:53.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.312+0000 7f61e2600700 1 --2- 172.21.15.112:0/706323577 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f61c40b2880 0x7f61c40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.313+0000 7f61e1c00700 1 --2- 172.21.15.112:0/3943619374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b139440 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009d00 tx=0x7f61d0011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dedcafbbd9bb756f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:53.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.313+0000 7f61cea00700 1 -- 172.21.15.112:0/3943619374 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0004230 con 0x556d2b135930 2024-05-08T05:53:53.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.313+0000 7f61cea00700 1 -- 172.21.15.112:0/3943619374 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d00043d0 con 0x556d2b135930 2024-05-08T05:53:53.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.313+0000 7f61cea00700 1 -- 172.21.15.112:0/3943619374 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d00198c0 con 0x556d2b135930 2024-05-08T05:53:53.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.314+0000 7f61eef21b80 1 -- 172.21.15.112:0/3943619374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 msgr2=0x556d2b139440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:53.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.314+0000 7f61e2600700 1 --2- 172.21.15.112:0/706323577 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f61c40b2880 0x7f61c40b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009fd0 tx=0x7f61d4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:53.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.314+0000 7f61eef21b80 1 --2- 172.21.15.112:0/3943619374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b139440 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009d00 tx=0x7f61d0011720 comp rx=0 tx=0).stop 2024-05-08T05:53:53.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.314+0000 7f61eef21b80 1 -- 172.21.15.112:0/3943619374 shutdown_connections 2024-05-08T05:53:53.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.314+0000 7f61eef21b80 1 --2- 172.21.15.112:0/3943619374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b139980 0x556d2b139de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.314+0000 7f61eef21b80 1 --2- 172.21.15.112:0/3943619374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b135010 0x556d2b1353f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.314+0000 7f61eef21b80 1 --2- 172.21.15.112:0/3943619374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b139440 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.314+0000 7f61eef21b80 1 -- 172.21.15.112:0/3943619374 >> 172.21.15.112:0/3943619374 conn(0x556d2b063b10 msgr2=0x556d2b12f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:53.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.314+0000 7f61eef21b80 1 -- 172.21.15.112:0/3943619374 shutdown_connections 2024-05-08T05:53:53.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.314+0000 7f61eef21b80 1 -- 172.21.15.112:0/3943619374 wait complete. 2024-05-08T05:53:53.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.314+0000 7f61eef21b80 1 -- start start 2024-05-08T05:53:53.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.315+0000 7f61eef21b80 1 Processor -- start 2024-05-08T05:53:53.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.315+0000 7f61eef21b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135010 0x556d2b140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.315+0000 7f61eef21b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b13e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.315+0000 7f61eef21b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b139980 0x556d2b13e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.315+0000 7f61eef21b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556d2b141b90 con 0x556d2b139980 2024-05-08T05:53:53.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.315+0000 7f61eef21b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556d2afedad0 con 0x556d2b135010 2024-05-08T05:53:53.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.315+0000 7f61eef21b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x556d2b141d10 con 0x556d2b135930 2024-05-08T05:53:53.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.315+0000 7f61eef21b80 10 client.0 _ll_register_callbacks cb 0x556d2b02ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:53:53.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.316+0000 7f61e1200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b13e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.316+0000 7f61e1c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135010 0x556d2b140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.316+0000 7f61e1200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b13e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:33094/0 (socket says 172.21.15.112:33094) 2024-05-08T05:53:53.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.316+0000 7f61e1200700 1 -- 172.21.15.112:0/1323193715 learned_addr learned my addr 172.21.15.112:0/1323193715 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:53.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.316+0000 7f61e2600700 1 --2- 172.21.15.112:0/1323193715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b139980 0x556d2b13e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.316+0000 7f61e1200700 1 -- 172.21.15.112:0/1323193715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135010 msgr2=0x556d2b140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:53.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.316+0000 7f61e1200700 1 --2- 172.21.15.112:0/1323193715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135010 0x556d2b140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.316+0000 7f61e1200700 1 -- 172.21.15.112:0/1323193715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b139980 msgr2=0x556d2b13e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:53.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.316+0000 7f61e1200700 1 --2- 172.21.15.112:0/1323193715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b139980 0x556d2b13e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.316+0000 7f61e1200700 1 -- 172.21.15.112:0/1323193715 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2b04d3d0 con 0x556d2b135930 2024-05-08T05:53:53.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.316+0000 7f61e1200700 1 --2- 172.21.15.112:0/1323193715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b13e160 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f61c80aa250 tx=0x7f61c80a6540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55d8e831cf86131c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:53.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.316+0000 7f61c3400700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:53:53.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.316+0000 7f61c3400700 1 -- 172.21.15.112:0/1323193715 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c80e0e60 con 0x556d2b135930 2024-05-08T05:53:53.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.317+0000 7f61eef21b80 1 -- 172.21.15.112:0/1323193715 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d00043d0 con 0x556d2b135930 2024-05-08T05:53:53.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.317+0000 7f61c3400700 1 -- 172.21.15.112:0/1323193715 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f61c80e0ab0 con 0x556d2b135930 2024-05-08T05:53:53.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.317+0000 7f61c3400700 1 -- 172.21.15.112:0/1323193715 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f61c80b3320 con 0x556d2b135930 2024-05-08T05:53:53.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.317+0000 7f61eef21b80 10 client.24883 fetch_fsmap learned FSMap version 24 2024-05-08T05:53:53.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.317+0000 7f61eef21b80 1 -- 172.21.15.112:0/1323193715 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556d2b141c40 con 0x556d2b135930 2024-05-08T05:53:53.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.317+0000 7f61c3400700 1 -- 172.21.15.112:0/1323193715 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d00043d0 con 0x556d2b135930 2024-05-08T05:53:53.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.317+0000 7f61c3400700 1 -- 172.21.15.112:0/1323193715 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f61c80b2e90 con 0x556d2b135930 2024-05-08T05:53:53.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.317+0000 7f61eef21b80 10 client.24883 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T05:53:53.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.317+0000 7f61eef21b80 10 client.24883 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:53:53.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.318+0000 7f61eef21b80 20 client.24883 populate_metadata read hostname 'smithi112' 2024-05-08T05:53:53.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.318+0000 7f61eef21b80 1 --2- 172.21.15.112:0/1323193715 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x556d2b14e9f0 0x556d2b14fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.318+0000 7f61eef21b80 4 client.24883 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:53:53.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.318+0000 7f61eef21b80 1 -- 172.21.15.112:0/1323193715 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d2b102050 con 0x556d2b14e9f0 2024-05-08T05:53:53.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.318+0000 7f61e1c00700 1 --2- 172.21.15.112:0/1323193715 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x556d2b14e9f0 0x556d2b14fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.319+0000 7f61c3400700 10 client.24883 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:53:53.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.318+0000 7f61e1c00700 1 --2- 172.21.15.112:0/1323193715 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x556d2b14e9f0 0x556d2b14fe60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3565de7d904d42d9 server_cookie=626805d2ad0e1388 in_seq=0 out_seq=0 2024-05-08T05:53:53.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.320+0000 7f61c3400700 1 -- 172.21.15.112:0/1323193715 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556d2b102050 con 0x556d2b14e9f0 2024-05-08T05:53:53.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.320+0000 7f61c3400700 10 client.24883 handle_command_reply: tid=0 2024-05-08T05:53:53.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.320+0000 7f61eef21b80 1 client.24883 shutdown 2024-05-08T05:53:53.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.320+0000 7f61eef21b80 1 -- 172.21.15.112:0/1323193715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 msgr2=0x556d2b13e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:53.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.320+0000 7f61eef21b80 1 --2- 172.21.15.112:0/1323193715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b13e160 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f61c80aa250 tx=0x7f61c80a6540 comp rx=0 tx=0).stop 2024-05-08T05:53:53.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.320+0000 7f61eef21b80 1 -- 172.21.15.112:0/1323193715 shutdown_connections 2024-05-08T05:53:53.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.320+0000 7f61eef21b80 1 --2- 172.21.15.112:0/1323193715 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x556d2b14e9f0 0x556d2b14fe60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.320+0000 7f61eef21b80 1 --2- 172.21.15.112:0/1323193715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b139980 0x556d2b13e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.320+0000 7f61eef21b80 1 --2- 172.21.15.112:0/1323193715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135010 0x556d2b140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.320+0000 7f61eef21b80 1 --2- 172.21.15.112:0/1323193715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b13e160 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.320+0000 7f61eef21b80 1 -- 172.21.15.112:0/1323193715 >> 172.21.15.112:0/1323193715 conn(0x556d2b063b10 msgr2=0x556d2b12a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:53.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.320+0000 7f61c3400700 0 client.24883 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:53:53.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.320+0000 7f61eef21b80 1 -- 172.21.15.112:0/1323193715 shutdown_connections 2024-05-08T05:53:53.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.320+0000 7f61eef21b80 1 -- 172.21.15.112:0/1323193715 wait complete. 2024-05-08T05:53:53.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.320+0000 7f61eef21b80 20 client.24883 trim_cache size 0 max 16384 2024-05-08T05:53:53.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.343+0000 7f61eef21b80 1 Processor -- start 2024-05-08T05:53:53.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.343+0000 7f61eef21b80 1 -- start start 2024-05-08T05:53:53.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.343+0000 7f61eef21b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b135010 0x556d2b13bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.343+0000 7f61eef21b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135930 0x556d2b1299d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.343+0000 7f61eef21b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b139980 0x556d2b129f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.343+0000 7f61eef21b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556d2b14e1a0 con 0x556d2b135010 2024-05-08T05:53:53.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.343+0000 7f61eef21b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556d2b14e870 con 0x556d2b135930 2024-05-08T05:53:53.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.343+0000 7f61eef21b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x556d2b14ef40 con 0x556d2b139980 2024-05-08T05:53:53.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.344+0000 7f61e1c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b135010 0x556d2b13bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.344+0000 7f61e2600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b139980 0x556d2b129f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.344+0000 7f61e1c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b135010 0x556d2b13bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48176/0 (socket says 172.21.15.112:48176) 2024-05-08T05:53:53.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.344+0000 7f61e1c00700 1 -- 172.21.15.112:0/4091153401 learned_addr learned my addr 172.21.15.112:0/4091153401 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:53.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.344+0000 7f61e1200700 1 --2- 172.21.15.112:0/4091153401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135930 0x556d2b1299d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.344+0000 7f61e1200700 1 -- 172.21.15.112:0/4091153401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b139980 msgr2=0x556d2b129f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:53.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.344+0000 7f61e1200700 1 --2- 172.21.15.112:0/4091153401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b139980 0x556d2b129f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.344+0000 7f61e1200700 1 -- 172.21.15.112:0/4091153401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b135010 msgr2=0x556d2b13bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:53.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.344+0000 7f61e1200700 1 --2- 172.21.15.112:0/4091153401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b135010 0x556d2b13bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.344+0000 7f61e1200700 1 -- 172.21.15.112:0/4091153401 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2b14e690 con 0x556d2b135930 2024-05-08T05:53:53.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.344+0000 7f61e1c00700 1 --2- 172.21.15.112:0/4091153401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b135010 0x556d2b13bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:53.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.344+0000 7f61e1200700 1 --2- 172.21.15.112:0/4091153401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135930 0x556d2b1299d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f61c80a65d0 tx=0x7f61c8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b35bc4e1ce660d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:53.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.345+0000 7f61ce000700 1 -- 172.21.15.112:0/4091153401 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c80e0e60 con 0x556d2b135930 2024-05-08T05:53:53.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.345+0000 7f61ce000700 1 -- 172.21.15.112:0/4091153401 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c80aac50 con 0x556d2b135930 2024-05-08T05:53:53.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.345+0000 7f61ce000700 1 -- 172.21.15.112:0/4091153401 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c80050c0 con 0x556d2b135930 2024-05-08T05:53:53.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.345+0000 7f61eef21b80 1 -- 172.21.15.112:0/4091153401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135930 msgr2=0x556d2b1299d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:53.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.345+0000 7f61eef21b80 1 --2- 172.21.15.112:0/4091153401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135930 0x556d2b1299d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f61c80a65d0 tx=0x7f61c8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.345+0000 7f61eef21b80 1 -- 172.21.15.112:0/4091153401 shutdown_connections 2024-05-08T05:53:53.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.345+0000 7f61eef21b80 1 --2- 172.21.15.112:0/4091153401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b139980 0x556d2b129f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.345+0000 7f61eef21b80 1 --2- 172.21.15.112:0/4091153401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b135010 0x556d2b13bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.345+0000 7f61eef21b80 1 --2- 172.21.15.112:0/4091153401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135930 0x556d2b1299d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.345+0000 7f61eef21b80 1 -- 172.21.15.112:0/4091153401 >> 172.21.15.112:0/4091153401 conn(0x556d2b063b10 msgr2=0x556d2b13ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:53.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.345+0000 7f61eef21b80 1 -- 172.21.15.112:0/4091153401 shutdown_connections 2024-05-08T05:53:53.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.345+0000 7f61eef21b80 1 -- 172.21.15.112:0/4091153401 wait complete. 2024-05-08T05:53:53.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.346+0000 7f61eef21b80 1 -- start start 2024-05-08T05:53:53.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.346+0000 7f61eef21b80 1 Processor -- start 2024-05-08T05:53:53.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.346+0000 7f61eef21b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135010 0x556d2b1582d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.346+0000 7f61eef21b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b158810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.346+0000 7f61eef21b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b139980 0x556d2b158d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.347+0000 7f61eef21b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556d2b14fce0 con 0x556d2b139980 2024-05-08T05:53:53.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.347+0000 7f61eef21b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556d2b14f610 con 0x556d2b135010 2024-05-08T05:53:53.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.347+0000 7f61eef21b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x556d2b1503b0 con 0x556d2b135930 2024-05-08T05:53:53.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.347+0000 7f61eef21b80 10 client.0 _ll_register_callbacks cb 0x556d2b02ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:53:53.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.347+0000 7f61e1c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135010 0x556d2b1582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.347+0000 7f61e1c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135010 0x556d2b1582d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:44878/0 (socket says 172.21.15.112:44878) 2024-05-08T05:53:53.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.347+0000 7f61e1c00700 1 -- 172.21.15.112:0/1336219770 learned_addr learned my addr 172.21.15.112:0/1336219770 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:53.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.347+0000 7f61e2600700 1 --2- 172.21.15.112:0/1336219770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b139980 0x556d2b158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.347+0000 7f61e1200700 1 --2- 172.21.15.112:0/1336219770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.347+0000 7f61e1c00700 1 -- 172.21.15.112:0/1336219770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 msgr2=0x556d2b158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:53.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.347+0000 7f61e1c00700 1 --2- 172.21.15.112:0/1336219770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.347+0000 7f61e1c00700 1 -- 172.21.15.112:0/1336219770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b139980 msgr2=0x556d2b158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:53.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.347+0000 7f61e1c00700 1 --2- 172.21.15.112:0/1336219770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b139980 0x556d2b158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.347+0000 7f61e1c00700 1 -- 172.21.15.112:0/1336219770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2b159550 con 0x556d2b135010 2024-05-08T05:53:53.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.347+0000 7f61e2600700 1 --2- 172.21.15.112:0/1336219770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b139980 0x556d2b158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:53:53.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.348+0000 7f61e1c00700 1 --2- 172.21.15.112:0/1336219770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135010 0x556d2b1582d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f61d001b040 tx=0x7f61d0009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da70a36c4833336b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:53.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.348+0000 7f61c3400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:53:53.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.348+0000 7f61c3400700 1 -- 172.21.15.112:0/1336219770 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d002f070 con 0x556d2b135010 2024-05-08T05:53:53.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.348+0000 7f61eef21b80 1 -- 172.21.15.112:0/1336219770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c80aac50 con 0x556d2b135010 2024-05-08T05:53:53.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.348+0000 7f61c3400700 1 -- 172.21.15.112:0/1336219770 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f61d0005040 con 0x556d2b135010 2024-05-08T05:53:53.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.348+0000 7f61c3400700 1 -- 172.21.15.112:0/1336219770 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f61d003a030 con 0x556d2b135010 2024-05-08T05:53:53.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.348+0000 7f61eef21b80 10 client.24866 fetch_fsmap learned FSMap version 24 2024-05-08T05:53:53.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.348+0000 7f61eef21b80 1 -- 172.21.15.112:0/1336219770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556d2b136b40 con 0x556d2b135010 2024-05-08T05:53:53.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.349+0000 7f61c3400700 1 -- 172.21.15.112:0/1336219770 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c80aac50 con 0x556d2b135010 2024-05-08T05:53:53.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.349+0000 7f61c3400700 1 -- 172.21.15.112:0/1336219770 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f61d002bce0 con 0x556d2b135010 2024-05-08T05:53:53.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.349+0000 7f61eef21b80 10 client.24866 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T05:53:53.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.349+0000 7f61eef21b80 10 client.24866 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:53:53.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.349+0000 7f61eef21b80 20 client.24866 populate_metadata read hostname 'smithi112' 2024-05-08T05:53:53.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.349+0000 7f61eef21b80 1 --2- 172.21.15.112:0/1336219770 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x556d2b1c79c0 0x556d2b1c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:53.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.349+0000 7f61eef21b80 4 client.24866 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:53:53.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.349+0000 7f61eef21b80 1 -- 172.21.15.112:0/1336219770 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "status"}) -- 0x556d2b0953b0 con 0x556d2b1c79c0 2024-05-08T05:53:53.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.349+0000 7f61e1200700 1 --2- 172.21.15.112:0/1336219770 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x556d2b1c79c0 0x556d2b1c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:53.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61e1200700 1 --2- 172.21.15.112:0/1336219770 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x556d2b1c79c0 0x556d2b1c66d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e428be674d397af1 server_cookie=e3b64781194b6a00 in_seq=0 out_seq=0 2024-05-08T05:53:53.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61c3400700 10 client.24866 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:53:53.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61c3400700 1 -- 172.21.15.112:0/1336219770 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556d2b0953b0 con 0x556d2b1c79c0 2024-05-08T05:53:53.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61c3400700 10 client.24866 handle_command_reply: tid=0 2024-05-08T05:53:53.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61eef21b80 1 client.24866 shutdown 2024-05-08T05:53:53.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61eef21b80 1 -- 172.21.15.112:0/1336219770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135010 msgr2=0x556d2b1582d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:53.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61eef21b80 1 --2- 172.21.15.112:0/1336219770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135010 0x556d2b1582d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f61d001b040 tx=0x7f61d0009f70 comp rx=0 tx=0).stop 2024-05-08T05:53:53.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61eef21b80 1 -- 172.21.15.112:0/1336219770 shutdown_connections 2024-05-08T05:53:53.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61eef21b80 1 --2- 172.21.15.112:0/1336219770 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x556d2b1c79c0 0x556d2b1c66d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61eef21b80 1 --2- 172.21.15.112:0/1336219770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d2b139980 0x556d2b158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61eef21b80 1 --2- 172.21.15.112:0/1336219770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d2b135010 0x556d2b1582d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61eef21b80 1 --2- 172.21.15.112:0/1336219770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d2b135930 0x556d2b158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61c3400700 0 client.24866 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:53:53.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61eef21b80 1 -- 172.21.15.112:0/1336219770 >> 172.21.15.112:0/1336219770 conn(0x556d2b063b10 msgr2=0x556d2b12a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:53.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61eef21b80 1 -- 172.21.15.112:0/1336219770 shutdown_connections 2024-05-08T05:53:53.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.350+0000 7f61eef21b80 1 -- 172.21.15.112:0/1336219770 wait complete. 2024-05-08T05:53:53.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.351+0000 7f61eef21b80 20 client.24866 trim_cache size 0 max 16384 2024-05-08T05:53:53.356 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:53:53.356 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "7a07ff60-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:53:53.356 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 2, 2024-05-08T05:53:53.356 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24511, 2024-05-08T05:53:53.356 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-08T05:53:53.356 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-08T05:53:53.356 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-08T05:53:53.356 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 148.746920703, 2024-05-08T05:53:53.356 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 24, 2024-05-08T05:53:53.357 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 84, 2024-05-08T05:53:53.357 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 80, 2024-05-08T05:53:53.357 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 198.95884073799999 2024-05-08T05:53:53.357 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:53:53.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.351+0000 7f61e4600700 1 -- 172.21.15.112:0/706323577 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f61c40b2880 msgr2=0x7f61c40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:53.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.351+0000 7f61e4600700 1 --2- 172.21.15.112:0/706323577 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f61c40b2880 0x7f61c40b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009fd0 tx=0x7f61d4009040 comp rx=0 tx=0).stop 2024-05-08T05:53:53.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.351+0000 7f61e4600700 1 -- 172.21.15.112:0/706323577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61dc106080 msgr2=0x7f61dc19c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:53.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.351+0000 7f61e4600700 1 --2- 172.21.15.112:0/706323577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61dc106080 0x7f61dc19c340 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f61c8000c00 tx=0x7f61c80057c0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.351+0000 7f61e4600700 1 -- 172.21.15.112:0/706323577 shutdown_connections 2024-05-08T05:53:53.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.351+0000 7f61e4600700 1 --2- 172.21.15.112:0/706323577 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f61c40b2880 0x7f61c40b4d40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.351+0000 7f61e4600700 1 --2- 172.21.15.112:0/706323577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61dc10b6c0 0x7f61dc1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.351+0000 7f61e4600700 1 --2- 172.21.15.112:0/706323577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61dc1056c0 0x7f61dc19be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.351+0000 7f61e4600700 1 --2- 172.21.15.112:0/706323577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61dc106080 0x7f61dc19c340 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:53.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.351+0000 7f61e4600700 1 -- 172.21.15.112:0/706323577 >> 172.21.15.112:0/706323577 conn(0x7f61dc1011b0 msgr2=0x7f61dc1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:53.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.352+0000 7f61e4600700 1 -- 172.21.15.112:0/706323577 shutdown_connections 2024-05-08T05:53:53.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:53.352+0000 7f61e4600700 1 -- 172.21.15.112:0/706323577 wait complete. 2024-05-08T05:53:53.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:53 smithi112 ceph-mon[30003]: pgmap v398: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:53.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:53 smithi162 ceph-mon[34800]: pgmap v398: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:53.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:53 smithi150 ceph-mon[34315]: pgmap v398: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:54.236 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-08T05:53:54.674 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:55.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:55 smithi150 ceph-mon[34315]: pgmap v399: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:53:55.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:55 smithi112 ceph-mon[30003]: pgmap v399: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:53:55.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:55 smithi162 ceph-mon[34800]: pgmap v399: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:53:56.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.907+0000 7f9324c00700 1 -- 172.21.15.112:0/530422595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932010a490 msgr2=0x7f93201169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:56.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.907+0000 7f9324c00700 1 --2- 172.21.15.112:0/530422595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932010a490 0x7f93201169a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f931800b0a0 tx=0x7f931800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.907+0000 7f9324c00700 1 -- 172.21.15.112:0/530422595 shutdown_connections 2024-05-08T05:53:56.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.907+0000 7f9324c00700 1 --2- 172.21.15.112:0/530422595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932010a490 0x7f93201169a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.907+0000 7f9324c00700 1 --2- 172.21.15.112:0/530422595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f932010fb60 0x7f932010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.907+0000 7f9324c00700 1 --2- 172.21.15.112:0/530422595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9320109ad0 0x7f9320109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.907+0000 7f9324c00700 1 -- 172.21.15.112:0/530422595 >> 172.21.15.112:0/530422595 conn(0x7f9320076c70 msgr2=0x7f93201082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:56.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.908+0000 7f9324c00700 1 -- 172.21.15.112:0/530422595 shutdown_connections 2024-05-08T05:53:56.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.908+0000 7f9324c00700 1 -- 172.21.15.112:0/530422595 wait complete. 2024-05-08T05:53:56.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.909+0000 7f9324c00700 1 Processor -- start 2024-05-08T05:53:56.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.909+0000 7f9324c00700 1 -- start start 2024-05-08T05:53:56.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f9324c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9320109ad0 0x7f93201a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f9324c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f932010a490 0x7f93201a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f9324c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932010fb60 0x7f93201a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f9324c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9320119d00 con 0x7f932010fb60 2024-05-08T05:53:56.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f9324c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9320119b80 con 0x7f932010a490 2024-05-08T05:53:56.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f9324c00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9320119e80 con 0x7f9320109ad0 2024-05-08T05:53:56.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f931de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9320109ad0 0x7f93201a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f931e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932010fb60 0x7f93201a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f931de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9320109ad0 0x7f93201a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:44804/0 (socket says 172.21.15.112:44804) 2024-05-08T05:53:56.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f931de00700 1 -- 172.21.15.112:0/2755844241 learned_addr learned my addr 172.21.15.112:0/2755844241 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:56.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f931d400700 1 --2- 172.21.15.112:0/2755844241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f932010a490 0x7f93201a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f931de00700 1 -- 172.21.15.112:0/2755844241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f932010a490 msgr2=0x7f93201a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:56.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f931de00700 1 --2- 172.21.15.112:0/2755844241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f932010a490 0x7f93201a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f931de00700 1 -- 172.21.15.112:0/2755844241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932010fb60 msgr2=0x7f93201a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:56.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f931de00700 1 --2- 172.21.15.112:0/2755844241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932010fb60 0x7f93201a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.910+0000 7f931de00700 1 -- 172.21.15.112:0/2755844241 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93201a5160 con 0x7f9320109ad0 2024-05-08T05:53:56.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.911+0000 7f931e800700 1 --2- 172.21.15.112:0/2755844241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932010fb60 0x7f93201a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:56.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.911+0000 7f931de00700 1 --2- 172.21.15.112:0/2755844241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9320109ad0 0x7f93201a0150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f931000e9a0 tx=0x7f931000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:56.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.911+0000 7f9316a00700 1 -- 172.21.15.112:0/2755844241 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931000ca10 con 0x7f9320109ad0 2024-05-08T05:53:56.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.911+0000 7f9324c00700 1 -- 172.21.15.112:0/2755844241 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93201a5450 con 0x7f9320109ad0 2024-05-08T05:53:56.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.911+0000 7f9316a00700 1 -- 172.21.15.112:0/2755844241 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f931000cbb0 con 0x7f9320109ad0 2024-05-08T05:53:56.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.911+0000 7f9316a00700 1 -- 172.21.15.112:0/2755844241 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9310018680 con 0x7f9320109ad0 2024-05-08T05:53:56.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.911+0000 7f9324c00700 1 -- 172.21.15.112:0/2755844241 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93201accd0 con 0x7f9320109ad0 2024-05-08T05:53:56.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.917+0000 7f9316a00700 1 -- 172.21.15.112:0/2755844241 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9310010430 con 0x7f9320109ad0 2024-05-08T05:53:56.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.917+0000 7f932f52fb80 1 Processor -- start 2024-05-08T05:53:56.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.917+0000 7f932f52fb80 1 -- start start 2024-05-08T05:53:56.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.918+0000 7f932f52fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b010 0x56021e98b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.918+0000 7f932f52fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98b930 0x56021e98f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.918+0000 7f932f52fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e98fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.918+0000 7f932f52fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56021e8dd9d0 con 0x56021e98f980 2024-05-08T05:53:56.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.918+0000 7f932f52fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56021e87b710 con 0x56021e98b010 2024-05-08T05:53:56.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.918+0000 7f932f52fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56021e8d9990 con 0x56021e98b930 2024-05-08T05:53:56.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.918+0000 7f931e800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b010 0x56021e98b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.918+0000 7f931d400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e98fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.918+0000 7f931e800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b010 0x56021e98b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:41464/0 (socket says 172.21.15.112:41464) 2024-05-08T05:53:56.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.918+0000 7f931e800700 1 -- 172.21.15.112:0/2033139317 learned_addr learned my addr 172.21.15.112:0/2033139317 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:56.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.918+0000 7f931de00700 1 --2- 172.21.15.112:0/2033139317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98b930 0x56021e98f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.919+0000 7f9316a00700 1 --2- 172.21.15.112:0/2755844241 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f93040b2880 0x7f93040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.919+0000 7f931e800700 1 -- 172.21.15.112:0/2033139317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98b930 msgr2=0x56021e98f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:56.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.919+0000 7f931e800700 1 --2- 172.21.15.112:0/2033139317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98b930 0x56021e98f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.919+0000 7f931e800700 1 -- 172.21.15.112:0/2033139317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 msgr2=0x56021e98fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:56.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.919+0000 7f931e800700 1 --2- 172.21.15.112:0/2033139317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e98fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.919+0000 7f931e800700 1 -- 172.21.15.112:0/2033139317 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56021e82a880 con 0x56021e98b010 2024-05-08T05:53:56.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.919+0000 7f9316a00700 1 -- 172.21.15.112:0/2755844241 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f93100e2350 con 0x7f9320109ad0 2024-05-08T05:53:56.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.919+0000 7f931e800700 1 --2- 172.21.15.112:0/2033139317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b010 0x56021e98b3f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f931800b0a0 tx=0x7f931800bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0e5c3d615a3d017 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:56.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.919+0000 7f931e800700 1 --2- 172.21.15.112:0/2755844241 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f93040b2880 0x7f93040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.920+0000 7f930b400700 1 -- 172.21.15.112:0/2033139317 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318021070 con 0x56021e98b010 2024-05-08T05:53:56.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.920+0000 7f930b400700 1 -- 172.21.15.112:0/2033139317 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9318003c00 con 0x56021e98b010 2024-05-08T05:53:56.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.920+0000 7f930b400700 1 -- 172.21.15.112:0/2033139317 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318005320 con 0x56021e98b010 2024-05-08T05:53:56.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.920+0000 7f932f52fb80 1 -- 172.21.15.112:0/2033139317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b010 msgr2=0x56021e98b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:56.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.920+0000 7f931e800700 1 --2- 172.21.15.112:0/2755844241 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f93040b2880 0x7f93040b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9318027040 tx=0x7f9318005580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:56.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.920+0000 7f932f52fb80 1 --2- 172.21.15.112:0/2033139317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b010 0x56021e98b3f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f931800b0a0 tx=0x7f931800bf60 comp rx=0 tx=0).stop 2024-05-08T05:53:56.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.920+0000 7f932f52fb80 1 -- 172.21.15.112:0/2033139317 shutdown_connections 2024-05-08T05:53:56.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.920+0000 7f932f52fb80 1 --2- 172.21.15.112:0/2033139317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e98fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.920+0000 7f932f52fb80 1 --2- 172.21.15.112:0/2033139317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b010 0x56021e98b3f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.920+0000 7f932f52fb80 1 --2- 172.21.15.112:0/2033139317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98b930 0x56021e98f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.920+0000 7f932f52fb80 1 -- 172.21.15.112:0/2033139317 >> 172.21.15.112:0/2033139317 conn(0x56021e8b9b10 msgr2=0x56021e985750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:56.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.921+0000 7f932f52fb80 1 -- 172.21.15.112:0/2033139317 shutdown_connections 2024-05-08T05:53:56.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.921+0000 7f932f52fb80 1 -- 172.21.15.112:0/2033139317 wait complete. 2024-05-08T05:53:56.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.921+0000 7f932f52fb80 1 -- start start 2024-05-08T05:53:56.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.921+0000 7f932f52fb80 1 Processor -- start 2024-05-08T05:53:56.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.922+0000 7f932f52fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b010 0x56021e996dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.922+0000 7f932f52fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98b930 0x56021e9941c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.922+0000 7f932f52fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98f980 0x56021e994700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.922+0000 7f932f52fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56021e997ad0 con 0x56021e98b930 2024-05-08T05:53:56.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.922+0000 7f932f52fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56021e843ad0 con 0x56021e98b010 2024-05-08T05:53:56.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.922+0000 7f932f52fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56021e997c50 con 0x56021e98f980 2024-05-08T05:53:56.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.922+0000 7f932f52fb80 10 client.0 _ll_register_callbacks cb 0x56021e884c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:53:56.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.922+0000 7f931d400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98b930 0x56021e9941c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.922+0000 7f931de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b010 0x56021e996dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.922+0000 7f931d400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98b930 0x56021e9941c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41322/0 (socket says 172.21.15.112:41322) 2024-05-08T05:53:56.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.922+0000 7f931de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b010 0x56021e996dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:41480/0 (socket says 172.21.15.112:41480) 2024-05-08T05:53:56.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.922+0000 7f931d400700 1 -- 172.21.15.112:0/334025756 learned_addr learned my addr 172.21.15.112:0/334025756 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:56.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.922+0000 7f931e800700 1 --2- 172.21.15.112:0/334025756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98f980 0x56021e994700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.923+0000 7f931de00700 1 -- 172.21.15.112:0/334025756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98f980 msgr2=0x56021e994700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:56.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.923+0000 7f931de00700 1 --2- 172.21.15.112:0/334025756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98f980 0x56021e994700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.923+0000 7f931de00700 1 -- 172.21.15.112:0/334025756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98b930 msgr2=0x56021e9941c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:56.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.923+0000 7f931de00700 1 --2- 172.21.15.112:0/334025756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98b930 0x56021e9941c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.923+0000 7f931de00700 1 -- 172.21.15.112:0/334025756 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56021e8a33d0 con 0x56021e98b010 2024-05-08T05:53:56.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.923+0000 7f931e800700 1 --2- 172.21.15.112:0/334025756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98f980 0x56021e994700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:53:56.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.923+0000 7f931de00700 1 --2- 172.21.15.112:0/334025756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b010 0x56021e996dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f93100dfe70 tx=0x7f93100a9e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b33f3f012da0995 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:56.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.923+0000 7f92ffe00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:53:56.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.923+0000 7f92ffe00700 1 -- 172.21.15.112:0/334025756 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93100a90b0 con 0x56021e98b010 2024-05-08T05:53:56.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.923+0000 7f92ffe00700 1 -- 172.21.15.112:0/334025756 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f93100a8da0 con 0x56021e98b010 2024-05-08T05:53:56.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.923+0000 7f932f52fb80 1 -- 172.21.15.112:0/334025756 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9318003c00 con 0x56021e98b010 2024-05-08T05:53:56.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.923+0000 7f92ffe00700 1 -- 172.21.15.112:0/334025756 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f93100b20e0 con 0x56021e98b010 2024-05-08T05:53:56.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.923+0000 7f932f52fb80 10 client.24878 fetch_fsmap learned FSMap version 24 2024-05-08T05:53:56.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.923+0000 7f932f52fb80 1 -- 172.21.15.112:0/334025756 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56021e9a23c0 con 0x56021e98b010 2024-05-08T05:53:56.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.924+0000 7f92ffe00700 1 -- 172.21.15.112:0/334025756 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9318003c00 con 0x56021e98b010 2024-05-08T05:53:56.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.924+0000 7f92ffe00700 1 -- 172.21.15.112:0/334025756 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f93100e5b20 con 0x56021e98b010 2024-05-08T05:53:56.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.924+0000 7f932f52fb80 10 client.24878 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T05:53:56.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.924+0000 7f932f52fb80 10 client.24878 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:53:56.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.924+0000 7f932f52fb80 20 client.24878 populate_metadata read hostname 'smithi112' 2024-05-08T05:53:56.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.924+0000 7f932f52fb80 1 --2- 172.21.15.112:0/334025756 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x56021e9a49c0 0x56021e9a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.924+0000 7f932f52fb80 4 client.24878 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:53:56.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.924+0000 7f932f52fb80 1 -- 172.21.15.112:0/334025756 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56021e958050 con 0x56021e9a49c0 2024-05-08T05:53:56.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.924+0000 7f931d400700 1 --2- 172.21.15.112:0/334025756 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x56021e9a49c0 0x56021e9a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.925+0000 7f931d400700 1 --2- 172.21.15.112:0/334025756 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x56021e9a49c0 0x56021e9a5e30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7140931447efbdd7 server_cookie=f845def3854c5a7 in_seq=0 out_seq=0 2024-05-08T05:53:56.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.925+0000 7f92ffe00700 10 client.24878 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T05:53:56.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.926+0000 7f92ffe00700 1 -- 172.21.15.112:0/334025756 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56021e958050 con 0x56021e9a49c0 2024-05-08T05:53:56.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.926+0000 7f92ffe00700 10 client.24878 handle_command_reply: tid=0 2024-05-08T05:53:56.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.926+0000 7f932f52fb80 1 client.24878 shutdown 2024-05-08T05:53:56.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.926+0000 7f932f52fb80 1 -- 172.21.15.112:0/334025756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b010 msgr2=0x56021e996dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:56.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.926+0000 7f932f52fb80 1 --2- 172.21.15.112:0/334025756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b010 0x56021e996dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f93100dfe70 tx=0x7f93100a9e80 comp rx=0 tx=0).stop 2024-05-08T05:53:56.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.926+0000 7f932f52fb80 1 -- 172.21.15.112:0/334025756 shutdown_connections 2024-05-08T05:53:56.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.926+0000 7f932f52fb80 1 --2- 172.21.15.112:0/334025756 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x56021e9a49c0 0x56021e9a5e30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.926+0000 7f932f52fb80 1 --2- 172.21.15.112:0/334025756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98f980 0x56021e994700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.926+0000 7f932f52fb80 1 --2- 172.21.15.112:0/334025756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b010 0x56021e996dd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.926+0000 7f932f52fb80 1 --2- 172.21.15.112:0/334025756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98b930 0x56021e9941c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.926+0000 7f92ffe00700 0 client.24878 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T05:53:56.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.926+0000 7f932f52fb80 1 -- 172.21.15.112:0/334025756 >> 172.21.15.112:0/334025756 conn(0x56021e8b9b10 msgr2=0x56021e985710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:56.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.926+0000 7f932f52fb80 1 -- 172.21.15.112:0/334025756 shutdown_connections 2024-05-08T05:53:56.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.926+0000 7f932f52fb80 1 -- 172.21.15.112:0/334025756 wait complete. 2024-05-08T05:53:56.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.927+0000 7f932f52fb80 20 client.24878 trim_cache size 0 max 16384 2024-05-08T05:53:56.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.950+0000 7f932f52fb80 1 Processor -- start 2024-05-08T05:53:56.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.950+0000 7f932f52fb80 1 -- start start 2024-05-08T05:53:56.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.950+0000 7f932f52fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98b010 0x56021e9800e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.950+0000 7f932f52fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b930 0x56021e98fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.950+0000 7f932f52fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e9902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.950+0000 7f932f52fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56021e9a4170 con 0x56021e98f980 2024-05-08T05:53:56.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.950+0000 7f932f52fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56021e9a4840 con 0x56021e98b930 2024-05-08T05:53:56.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.950+0000 7f932f52fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56021e9a4f10 con 0x56021e98b010 2024-05-08T05:53:56.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.950+0000 7f931e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e9902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.950+0000 7f931d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b930 0x56021e98fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.951+0000 7f931e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e9902a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41328/0 (socket says 172.21.15.112:41328) 2024-05-08T05:53:56.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.951+0000 7f931e800700 1 -- 172.21.15.112:0/490086339 learned_addr learned my addr 172.21.15.112:0/490086339 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:56.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.951+0000 7f931d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b930 0x56021e98fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:41488/0 (socket says 172.21.15.112:41488) 2024-05-08T05:53:56.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.951+0000 7f931de00700 1 --2- 172.21.15.112:0/490086339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98b010 0x56021e9800e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.951+0000 7f931d400700 1 -- 172.21.15.112:0/490086339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98b010 msgr2=0x56021e9800e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:56.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.951+0000 7f931d400700 1 --2- 172.21.15.112:0/490086339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98b010 0x56021e9800e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.951+0000 7f931d400700 1 -- 172.21.15.112:0/490086339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 msgr2=0x56021e9902a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:56.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.951+0000 7f931d400700 1 --2- 172.21.15.112:0/490086339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e9902a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.951+0000 7f931d400700 1 -- 172.21.15.112:0/490086339 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56021e9823a0 con 0x56021e98b930 2024-05-08T05:53:56.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.951+0000 7f931e800700 1 --2- 172.21.15.112:0/490086339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e9902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:56.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.951+0000 7f931d400700 1 --2- 172.21.15.112:0/490086339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b930 0x56021e98fd60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x56021e9a67f0 tx=0x7f930c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7db2c900d28b6678 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:56.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.951+0000 7f930aa00700 1 -- 172.21.15.112:0/490086339 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c011d40 con 0x56021e98b930 2024-05-08T05:53:56.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.952+0000 7f930aa00700 1 -- 172.21.15.112:0/490086339 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f930c0205c0 con 0x56021e98b930 2024-05-08T05:53:56.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.952+0000 7f930aa00700 1 -- 172.21.15.112:0/490086339 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c01e8c0 con 0x56021e98b930 2024-05-08T05:53:56.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.952+0000 7f932f52fb80 1 -- 172.21.15.112:0/490086339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b930 msgr2=0x56021e98fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:56.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.952+0000 7f932f52fb80 1 --2- 172.21.15.112:0/490086339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b930 0x56021e98fd60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x56021e9a67f0 tx=0x7f930c009260 comp rx=0 tx=0).stop 2024-05-08T05:53:56.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.952+0000 7f932f52fb80 1 -- 172.21.15.112:0/490086339 shutdown_connections 2024-05-08T05:53:56.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.952+0000 7f932f52fb80 1 --2- 172.21.15.112:0/490086339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e9902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.952+0000 7f932f52fb80 1 --2- 172.21.15.112:0/490086339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98b010 0x56021e9800e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.952+0000 7f932f52fb80 1 --2- 172.21.15.112:0/490086339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b930 0x56021e98fd60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.952+0000 7f932f52fb80 1 -- 172.21.15.112:0/490086339 >> 172.21.15.112:0/490086339 conn(0x56021e8b9b10 msgr2=0x56021e97f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:56.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.952+0000 7f932f52fb80 1 -- 172.21.15.112:0/490086339 shutdown_connections 2024-05-08T05:53:56.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.952+0000 7f932f52fb80 1 -- 172.21.15.112:0/490086339 wait complete. 2024-05-08T05:53:56.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.953+0000 7f932f52fb80 1 -- start start 2024-05-08T05:53:56.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.953+0000 7f932f52fb80 1 Processor -- start 2024-05-08T05:53:56.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f932f52fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98b010 0x56021e9af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f932f52fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b930 0x56021e9aa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f932f52fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e9aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f932f52fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56021e9a5cb0 con 0x56021e98f980 2024-05-08T05:53:56.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f932f52fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56021e9a55e0 con 0x56021e98b930 2024-05-08T05:53:56.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f932f52fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56021e9a6380 con 0x56021e98b010 2024-05-08T05:53:56.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f932f52fb80 10 client.0 _ll_register_callbacks cb 0x56021e884c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:53:56.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f931e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e9aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f931d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b930 0x56021e9aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f931de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98b010 0x56021e9af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f931e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e9aa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41334/0 (socket says 172.21.15.112:41334) 2024-05-08T05:53:56.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f931e800700 1 -- 172.21.15.112:0/1132638842 learned_addr learned my addr 172.21.15.112:0/1132638842 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:53:56.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f931d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b930 0x56021e9aa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:41496/0 (socket says 172.21.15.112:41496) 2024-05-08T05:53:56.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f931e800700 1 -- 172.21.15.112:0/1132638842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98b010 msgr2=0x56021e9af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:56.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f931e800700 1 --2- 172.21.15.112:0/1132638842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98b010 0x56021e9af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f931e800700 1 -- 172.21.15.112:0/1132638842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b930 msgr2=0x56021e9aa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:56.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f931e800700 1 --2- 172.21.15.112:0/1132638842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b930 0x56021e9aa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.954+0000 7f931e800700 1 -- 172.21.15.112:0/1132638842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56021e9aafc0 con 0x56021e98f980 2024-05-08T05:53:56.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.955+0000 7f931e800700 1 --2- 172.21.15.112:0/1132638842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e9aa7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9318009fd0 tx=0x7f9318031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812615b3521f2ec2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:56.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.955+0000 7f92ffe00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:53:56.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.955+0000 7f92ffe00700 1 -- 172.21.15.112:0/1132638842 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931800b700 con 0x56021e98f980 2024-05-08T05:53:56.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.955+0000 7f92ffe00700 1 -- 172.21.15.112:0/1132638842 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f9318005670 con 0x56021e98f980 2024-05-08T05:53:56.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.955+0000 7f932f52fb80 1 -- 172.21.15.112:0/1132638842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f930c0205c0 con 0x56021e98f980 2024-05-08T05:53:56.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.955+0000 7f92ffe00700 1 -- 172.21.15.112:0/1132638842 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f93180076b0 con 0x56021e98f980 2024-05-08T05:53:56.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.955+0000 7f932f52fb80 10 client.14778 fetch_fsmap learned FSMap version 24 2024-05-08T05:53:56.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.955+0000 7f932f52fb80 1 -- 172.21.15.112:0/1132638842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56021ea10e90 con 0x56021e98f980 2024-05-08T05:53:56.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.956+0000 7f92ffe00700 1 -- 172.21.15.112:0/1132638842 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f930c0205c0 con 0x56021e98f980 2024-05-08T05:53:56.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.956+0000 7f92ffe00700 1 -- 172.21.15.112:0/1132638842 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9318011070 con 0x56021e98f980 2024-05-08T05:53:56.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.956+0000 7f932f52fb80 10 client.14778 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T05:53:56.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.956+0000 7f932f52fb80 10 client.14778 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:53:56.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.956+0000 7f932f52fb80 20 client.14778 populate_metadata read hostname 'smithi112' 2024-05-08T05:53:56.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.956+0000 7f932f52fb80 1 --2- 172.21.15.112:0/1132638842 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x56021ea15590 0x56021ea142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:56.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.956+0000 7f932f52fb80 4 client.14778 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:53:56.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.956+0000 7f932f52fb80 1 -- 172.21.15.112:0/1132638842 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "status"}) -- 0x56021e8eb3b0 con 0x56021ea15590 2024-05-08T05:53:56.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.956+0000 7f931de00700 1 --2- 172.21.15.112:0/1132638842 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x56021ea15590 0x56021ea142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:56.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.957+0000 7f931de00700 1 --2- 172.21.15.112:0/1132638842 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x56021ea15590 0x56021ea142a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d37b31762bef1786 server_cookie=583e94a70829ce63 in_seq=0 out_seq=0 2024-05-08T05:53:56.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.957+0000 7f92ffe00700 10 client.14778 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T05:53:56.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.957+0000 7f92ffe00700 1 -- 172.21.15.112:0/1132638842 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56021e8eb3b0 con 0x56021ea15590 2024-05-08T05:53:56.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.957+0000 7f92ffe00700 10 client.14778 handle_command_reply: tid=0 2024-05-08T05:53:56.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.957+0000 7f932f52fb80 1 client.14778 shutdown 2024-05-08T05:53:56.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.957+0000 7f932f52fb80 1 -- 172.21.15.112:0/1132638842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 msgr2=0x56021e9aa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:56.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.957+0000 7f932f52fb80 1 --2- 172.21.15.112:0/1132638842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e9aa7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9318009fd0 tx=0x7f9318031000 comp rx=0 tx=0).stop 2024-05-08T05:53:56.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.957+0000 7f932f52fb80 1 -- 172.21.15.112:0/1132638842 shutdown_connections 2024-05-08T05:53:56.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.957+0000 7f932f52fb80 1 --2- 172.21.15.112:0/1132638842 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x56021ea15590 0x56021ea142a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.957+0000 7f932f52fb80 1 --2- 172.21.15.112:0/1132638842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56021e98f980 0x56021e9aa7a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.957+0000 7f932f52fb80 1 --2- 172.21.15.112:0/1132638842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021e98b010 0x56021e9af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.957+0000 7f932f52fb80 1 --2- 172.21.15.112:0/1132638842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56021e98b930 0x56021e9aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.957+0000 7f932f52fb80 1 -- 172.21.15.112:0/1132638842 >> 172.21.15.112:0/1132638842 conn(0x56021e8b9b10 msgr2=0x56021e980260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:56.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.957+0000 7f92ffe00700 0 client.14778 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T05:53:56.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.958+0000 7f932f52fb80 1 -- 172.21.15.112:0/1132638842 shutdown_connections 2024-05-08T05:53:56.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.958+0000 7f932f52fb80 1 -- 172.21.15.112:0/1132638842 wait complete. 2024-05-08T05:53:56.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.958+0000 7f932f52fb80 20 client.14778 trim_cache size 0 max 16384 2024-05-08T05:53:56.962 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:53:56.962 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "7a07ff60-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:53:56.962 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-05-08T05:53:56.962 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24518, 2024-05-08T05:53:56.962 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-08T05:53:56.962 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-08T05:53:56.962 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-08T05:53:56.962 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 153.36870969899999, 2024-05-08T05:53:56.962 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 24, 2024-05-08T05:53:56.962 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 84, 2024-05-08T05:53:56.962 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 80, 2024-05-08T05:53:56.962 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 190.99259708100001 2024-05-08T05:53:56.963 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:53:56.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.958+0000 7f9324c00700 1 -- 172.21.15.112:0/2755844241 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f93040b2880 msgr2=0x7f93040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:56.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.958+0000 7f9324c00700 1 --2- 172.21.15.112:0/2755844241 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f93040b2880 0x7f93040b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9318027040 tx=0x7f9318005580 comp rx=0 tx=0).stop 2024-05-08T05:53:56.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.958+0000 7f9324c00700 1 -- 172.21.15.112:0/2755844241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9320109ad0 msgr2=0x7f93201a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:56.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.958+0000 7f9324c00700 1 --2- 172.21.15.112:0/2755844241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9320109ad0 0x7f93201a0150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f931000e9a0 tx=0x7f931000ecb0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.959+0000 7f9324c00700 1 -- 172.21.15.112:0/2755844241 shutdown_connections 2024-05-08T05:53:56.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.959+0000 7f9324c00700 1 --2- 172.21.15.112:0/2755844241 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f93040b2880 0x7f93040b4d40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.959+0000 7f9324c00700 1 --2- 172.21.15.112:0/2755844241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f932010fb60 0x7f93201a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.959+0000 7f9324c00700 1 --2- 172.21.15.112:0/2755844241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9320109ad0 0x7f93201a0150 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.959+0000 7f9324c00700 1 --2- 172.21.15.112:0/2755844241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f932010a490 0x7f93201a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:56.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.959+0000 7f9324c00700 1 -- 172.21.15.112:0/2755844241 >> 172.21.15.112:0/2755844241 conn(0x7f9320076c70 msgr2=0x7f93200771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:56.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.959+0000 7f9324c00700 1 -- 172.21.15.112:0/2755844241 shutdown_connections 2024-05-08T05:53:56.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:53:56.959+0000 7f9324c00700 1 -- 172.21.15.112:0/2755844241 wait complete. 2024-05-08T05:53:57.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:57 smithi112 ceph-mon[30003]: pgmap v400: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:57.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:57 smithi162 ceph-mon[34800]: pgmap v400: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:57.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:57 smithi150 ceph-mon[34315]: pgmap v400: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:53:58.002 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.a status 2024-05-08T05:53:58.452 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:59.466 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:53:59 smithi112 ceph-mon[30003]: pgmap v401: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:59.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:53:59 smithi162 ceph-mon[34800]: pgmap v401: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:53:59.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:53:59 smithi150 ceph-mon[34315]: pgmap v401: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:00.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.679+0000 7fbe46c00700 1 -- 172.21.15.112:0/1519346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe400756d0 msgr2=0x7fbe40075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:00.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.679+0000 7fbe46c00700 1 --2- 172.21.15.112:0/1519346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe400756d0 0x7fbe40075ab0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe300098f0 tx=0x7fbe30009c00 comp rx=0 tx=0).stop 2024-05-08T05:54:00.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.679+0000 7fbe46c00700 1 -- 172.21.15.112:0/1519346 shutdown_connections 2024-05-08T05:54:00.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.679+0000 7fbe46c00700 1 --2- 172.21.15.112:0/1519346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe40114670 0x7fbe40116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.679+0000 7fbe46c00700 1 --2- 172.21.15.112:0/1519346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe400756d0 0x7fbe40075ab0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.679+0000 7fbe46c00700 1 --2- 172.21.15.112:0/1519346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe40075ff0 0x7fbe40114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.679+0000 7fbe46c00700 1 -- 172.21.15.112:0/1519346 >> 172.21.15.112:0/1519346 conn(0x7fbe4007a600 msgr2=0x7fbe4007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:00.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.680+0000 7fbe46c00700 1 -- 172.21.15.112:0/1519346 shutdown_connections 2024-05-08T05:54:00.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.680+0000 7fbe46c00700 1 -- 172.21.15.112:0/1519346 wait complete. 2024-05-08T05:54:00.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.681+0000 7fbe46c00700 1 Processor -- start 2024-05-08T05:54:00.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.681+0000 7fbe46c00700 1 -- start start 2024-05-08T05:54:00.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.681+0000 7fbe46c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe400756d0 0x7fbe401a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.681+0000 7fbe46c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe40075ff0 0x7fbe401a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.681+0000 7fbe46c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe40114670 0x7fbe401a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.681+0000 7fbe46c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbe40119c80 con 0x7fbe40075ff0 2024-05-08T05:54:00.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.681+0000 7fbe46c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe40119b00 con 0x7fbe40114670 2024-05-08T05:54:00.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.681+0000 7fbe46c00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbe40119e00 con 0x7fbe400756d0 2024-05-08T05:54:00.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.682+0000 7fbe3fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe400756d0 0x7fbe401a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.682+0000 7fbe3fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe400756d0 0x7fbe401a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:44860/0 (socket says 172.21.15.112:44860) 2024-05-08T05:54:00.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.682+0000 7fbe3fe00700 1 -- 172.21.15.112:0/2412499962 learned_addr learned my addr 172.21.15.112:0/2412499962 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:00.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.682+0000 7fbe37400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe40075ff0 0x7fbe401a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.682+0000 7fbe44c00700 1 --2- 172.21.15.112:0/2412499962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe40114670 0x7fbe401a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.682+0000 7fbe3fe00700 1 -- 172.21.15.112:0/2412499962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe40114670 msgr2=0x7fbe401a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:00.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.682+0000 7fbe3fe00700 1 --2- 172.21.15.112:0/2412499962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe40114670 0x7fbe401a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.682+0000 7fbe3fe00700 1 -- 172.21.15.112:0/2412499962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe40075ff0 msgr2=0x7fbe401a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:00.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.682+0000 7fbe3fe00700 1 --2- 172.21.15.112:0/2412499962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe40075ff0 0x7fbe401a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.682+0000 7fbe3fe00700 1 -- 172.21.15.112:0/2412499962 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe401a51c0 con 0x7fbe400756d0 2024-05-08T05:54:00.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.683+0000 7fbe3fe00700 1 --2- 172.21.15.112:0/2412499962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe400756d0 0x7fbe401a01b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000f990 tx=0x7fbe3000f9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:00.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.683+0000 7fbe3d600700 1 -- 172.21.15.112:0/2412499962 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe30027070 con 0x7fbe400756d0 2024-05-08T05:54:00.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.683+0000 7fbe46c00700 1 -- 172.21.15.112:0/2412499962 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe401a5450 con 0x7fbe400756d0 2024-05-08T05:54:00.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.683+0000 7fbe3d600700 1 -- 172.21.15.112:0/2412499962 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe30005570 con 0x7fbe400756d0 2024-05-08T05:54:00.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.683+0000 7fbe3d600700 1 -- 172.21.15.112:0/2412499962 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe30004930 con 0x7fbe400756d0 2024-05-08T05:54:00.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.683+0000 7fbe46c00700 1 -- 172.21.15.112:0/2412499962 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe401acd30 con 0x7fbe400756d0 2024-05-08T05:54:00.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.688+0000 7fbe3d600700 1 -- 172.21.15.112:0/2412499962 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbe3002a070 con 0x7fbe400756d0 2024-05-08T05:54:00.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.689+0000 7fbe5154db80 1 Processor -- start 2024-05-08T05:54:00.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.689+0000 7fbe5154db80 1 -- start start 2024-05-08T05:54:00.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.689+0000 7fbe5154db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4010 0x55a3d40c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.689+0000 7fbe5154db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4930 0x55a3d40c8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.689+0000 7fbe5154db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c8980 0x55a3d40c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.690+0000 7fbe5154db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a3d40169d0 con 0x55a3d40c8980 2024-05-08T05:54:00.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.690+0000 7fbe5154db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a3d3fb4710 con 0x55a3d40c4930 2024-05-08T05:54:00.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.690+0000 7fbe5154db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a3d4012990 con 0x55a3d40c4010 2024-05-08T05:54:00.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.690+0000 7fbe3fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4930 0x55a3d40c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.690+0000 7fbe44c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4010 0x55a3d40c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.690+0000 7fbe37400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c8980 0x55a3d40c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.690+0000 7fbe44c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4010 0x55a3d40c43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:44876/0 (socket says 172.21.15.112:44876) 2024-05-08T05:54:00.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.690+0000 7fbe44c00700 1 -- 172.21.15.112:0/4289724211 learned_addr learned my addr 172.21.15.112:0/4289724211 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:00.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.690+0000 7fbe3d600700 1 --2- 172.21.15.112:0/2412499962 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbe240b2880 0x7fbe240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.690+0000 7fbe44c00700 1 --2- 172.21.15.112:0/2412499962 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbe240b2880 0x7fbe240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.690+0000 7fbe3d600700 1 -- 172.21.15.112:0/2412499962 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fbe300e2c50 con 0x7fbe400756d0 2024-05-08T05:54:00.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.691+0000 7fbe37400700 1 -- 172.21.15.112:0/4289724211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4010 msgr2=0x55a3d40c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:00.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.691+0000 7fbe37400700 1 --2- 172.21.15.112:0/4289724211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4010 0x55a3d40c43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.691+0000 7fbe37400700 1 -- 172.21.15.112:0/4289724211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4930 msgr2=0x55a3d40c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:00.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.691+0000 7fbe37400700 1 --2- 172.21.15.112:0/4289724211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4930 0x55a3d40c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.691+0000 7fbe37400700 1 -- 172.21.15.112:0/4289724211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d3f63880 con 0x55a3d40c8980 2024-05-08T05:54:00.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.691+0000 7fbe44c00700 1 --2- 172.21.15.112:0/4289724211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4010 0x55a3d40c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:00.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.691+0000 7fbe37400700 1 --2- 172.21.15.112:0/4289724211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c8980 0x55a3d40c8de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbe28005fd0 tx=0x7fbe28011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5580b07b91bb3a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:00.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.691+0000 7fbe35600700 1 -- 172.21.15.112:0/4289724211 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28004250 con 0x55a3d40c8980 2024-05-08T05:54:00.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.691+0000 7fbe35600700 1 -- 172.21.15.112:0/4289724211 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe280043f0 con 0x55a3d40c8980 2024-05-08T05:54:00.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.691+0000 7fbe44c00700 1 --2- 172.21.15.112:0/2412499962 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbe240b2880 0x7fbe240b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbe380060b0 tx=0x7fbe38004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:00.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.691+0000 7fbe35600700 1 -- 172.21.15.112:0/4289724211 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28019810 con 0x55a3d40c8980 2024-05-08T05:54:00.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.692+0000 7fbe5154db80 1 -- 172.21.15.112:0/4289724211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c8980 msgr2=0x55a3d40c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:00.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.692+0000 7fbe5154db80 1 --2- 172.21.15.112:0/4289724211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c8980 0x55a3d40c8de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbe28005fd0 tx=0x7fbe28011720 comp rx=0 tx=0).stop 2024-05-08T05:54:00.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.692+0000 7fbe5154db80 1 -- 172.21.15.112:0/4289724211 shutdown_connections 2024-05-08T05:54:00.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.692+0000 7fbe5154db80 1 --2- 172.21.15.112:0/4289724211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c8980 0x55a3d40c8de0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.692+0000 7fbe5154db80 1 --2- 172.21.15.112:0/4289724211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4010 0x55a3d40c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.692+0000 7fbe5154db80 1 --2- 172.21.15.112:0/4289724211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4930 0x55a3d40c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.692+0000 7fbe5154db80 1 -- 172.21.15.112:0/4289724211 >> 172.21.15.112:0/4289724211 conn(0x55a3d3ff2b10 msgr2=0x55a3d40be750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:00.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.693+0000 7fbe5154db80 1 -- 172.21.15.112:0/4289724211 shutdown_connections 2024-05-08T05:54:00.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.693+0000 7fbe5154db80 1 -- 172.21.15.112:0/4289724211 wait complete. 2024-05-08T05:54:00.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.693+0000 7fbe5154db80 1 -- start start 2024-05-08T05:54:00.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.693+0000 7fbe5154db80 1 Processor -- start 2024-05-08T05:54:00.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.694+0000 7fbe5154db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4010 0x55a3d40cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.694+0000 7fbe5154db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4930 0x55a3d40cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.694+0000 7fbe5154db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c8980 0x55a3d40cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.694+0000 7fbe5154db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a3d40d0b90 con 0x55a3d40c8980 2024-05-08T05:54:00.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.694+0000 7fbe5154db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a3d3f7cad0 con 0x55a3d40c4010 2024-05-08T05:54:00.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.694+0000 7fbe5154db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a3d40d0d10 con 0x55a3d40c4930 2024-05-08T05:54:00.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.694+0000 7fbe5154db80 10 client.0 _ll_register_callbacks cb 0x55a3d3fbdc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:00.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.694+0000 7fbe3fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4010 0x55a3d40cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.694+0000 7fbe37400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4930 0x55a3d40cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.694+0000 7fbe44c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c8980 0x55a3d40cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.694+0000 7fbe37400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4930 0x55a3d40cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:44888/0 (socket says 172.21.15.112:44888) 2024-05-08T05:54:00.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.694+0000 7fbe37400700 1 -- 172.21.15.112:0/1198442798 learned_addr learned my addr 172.21.15.112:0/1198442798 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:00.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.695+0000 7fbe3fe00700 1 -- 172.21.15.112:0/1198442798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4930 msgr2=0x55a3d40cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:00.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.695+0000 7fbe3fe00700 1 --2- 172.21.15.112:0/1198442798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4930 0x55a3d40cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.695+0000 7fbe3fe00700 1 -- 172.21.15.112:0/1198442798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c8980 msgr2=0x55a3d40cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:00.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.695+0000 7fbe3fe00700 1 --2- 172.21.15.112:0/1198442798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c8980 0x55a3d40cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.695+0000 7fbe3fe00700 1 -- 172.21.15.112:0/1198442798 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d3fdc3d0 con 0x55a3d40c4010 2024-05-08T05:54:00.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.695+0000 7fbe3fe00700 1 --2- 172.21.15.112:0/1198442798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4010 0x55a3d40cfe00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbe300734b0 tx=0x7fbe300aa0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4688aec0551a20d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:00.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.695+0000 7fbe2e000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:54:00.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.695+0000 7fbe2e000700 1 -- 172.21.15.112:0/1198442798 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe300a9d80 con 0x55a3d40c4010 2024-05-08T05:54:00.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.695+0000 7fbe2e000700 1 -- 172.21.15.112:0/1198442798 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fbe3006f310 con 0x55a3d40c4010 2024-05-08T05:54:00.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.695+0000 7fbe5154db80 1 -- 172.21.15.112:0/1198442798 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe280043f0 con 0x55a3d40c4010 2024-05-08T05:54:00.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.696+0000 7fbe2e000700 1 -- 172.21.15.112:0/1198442798 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbe300aa970 con 0x55a3d40c4010 2024-05-08T05:54:00.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.696+0000 7fbe5154db80 10 client.24896 fetch_fsmap learned FSMap version 24 2024-05-08T05:54:00.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.696+0000 7fbe5154db80 1 -- 172.21.15.112:0/1198442798 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a3d40db3f0 con 0x55a3d40c4010 2024-05-08T05:54:00.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.696+0000 7fbe2e000700 1 -- 172.21.15.112:0/1198442798 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe280043f0 con 0x55a3d40c4010 2024-05-08T05:54:00.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.696+0000 7fbe2e000700 1 -- 172.21.15.112:0/1198442798 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fbe30069ce0 con 0x55a3d40c4010 2024-05-08T05:54:00.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.697+0000 7fbe5154db80 10 client.24896 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T05:54:00.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.697+0000 7fbe5154db80 10 client.24896 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:54:00.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.697+0000 7fbe5154db80 20 client.24896 populate_metadata read hostname 'smithi112' 2024-05-08T05:54:00.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.697+0000 7fbe5154db80 1 --2- 172.21.15.112:0/1198442798 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a3d40ddad0 0x55a3d40def40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.697+0000 7fbe5154db80 4 client.24896 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:54:00.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.697+0000 7fbe5154db80 1 -- 172.21.15.112:0/1198442798 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3d4091050 con 0x55a3d40ddad0 2024-05-08T05:54:00.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.697+0000 7fbe37400700 1 --2- 172.21.15.112:0/1198442798 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a3d40ddad0 0x55a3d40def40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.698+0000 7fbe2e000700 10 client.24896 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:54:00.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.698+0000 7fbe37400700 1 --2- 172.21.15.112:0/1198442798 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a3d40ddad0 0x55a3d40def40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d73453787bda3791 server_cookie=2c5862680be85ad9 in_seq=0 out_seq=0 2024-05-08T05:54:00.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.699+0000 7fbe2e000700 1 -- 172.21.15.112:0/1198442798 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a3d4091050 con 0x55a3d40ddad0 2024-05-08T05:54:00.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.699+0000 7fbe2e000700 10 client.24896 handle_command_reply: tid=0 2024-05-08T05:54:00.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.699+0000 7fbe5154db80 1 client.24896 shutdown 2024-05-08T05:54:00.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.699+0000 7fbe5154db80 1 -- 172.21.15.112:0/1198442798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4010 msgr2=0x55a3d40cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:00.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.699+0000 7fbe5154db80 1 --2- 172.21.15.112:0/1198442798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4010 0x55a3d40cfe00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbe300734b0 tx=0x7fbe300aa0c0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.699+0000 7fbe5154db80 1 -- 172.21.15.112:0/1198442798 shutdown_connections 2024-05-08T05:54:00.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.699+0000 7fbe5154db80 1 --2- 172.21.15.112:0/1198442798 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a3d40ddad0 0x55a3d40def40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.699+0000 7fbe5154db80 1 --2- 172.21.15.112:0/1198442798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c8980 0x55a3d40cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.699+0000 7fbe5154db80 1 --2- 172.21.15.112:0/1198442798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4010 0x55a3d40cfe00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.699+0000 7fbe5154db80 1 --2- 172.21.15.112:0/1198442798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4930 0x55a3d40cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.699+0000 7fbe2e000700 0 client.24896 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:54:00.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.699+0000 7fbe5154db80 1 -- 172.21.15.112:0/1198442798 >> 172.21.15.112:0/1198442798 conn(0x55a3d3ff2b10 msgr2=0x55a3d40b9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:00.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.699+0000 7fbe5154db80 1 -- 172.21.15.112:0/1198442798 shutdown_connections 2024-05-08T05:54:00.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.699+0000 7fbe5154db80 1 -- 172.21.15.112:0/1198442798 wait complete. 2024-05-08T05:54:00.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.700+0000 7fbe5154db80 20 client.24896 trim_cache size 0 max 16384 2024-05-08T05:54:00.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.722+0000 7fbe5154db80 1 Processor -- start 2024-05-08T05:54:00.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.722+0000 7fbe5154db80 1 -- start start 2024-05-08T05:54:00.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.723+0000 7fbe5154db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c4010 0x55a3d40cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.723+0000 7fbe5154db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4930 0x55a3d40cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.723+0000 7fbe5154db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c8980 0x55a3d40c6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.723+0000 7fbe5154db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a3d40dd280 con 0x55a3d40c4010 2024-05-08T05:54:00.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.723+0000 7fbe5154db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a3d40dd950 con 0x55a3d40c4930 2024-05-08T05:54:00.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.723+0000 7fbe5154db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a3d40de020 con 0x55a3d40c8980 2024-05-08T05:54:00.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.723+0000 7fbe3fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c4010 0x55a3d40cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.723+0000 7fbe3fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c4010 0x55a3d40cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41366/0 (socket says 172.21.15.112:41366) 2024-05-08T05:54:00.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.723+0000 7fbe3fe00700 1 -- 172.21.15.112:0/2011403228 learned_addr learned my addr 172.21.15.112:0/2011403228 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:00.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.723+0000 7fbe37400700 1 --2- 172.21.15.112:0/2011403228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4930 0x55a3d40cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.723+0000 7fbe44c00700 1 --2- 172.21.15.112:0/2011403228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c8980 0x55a3d40c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.724+0000 7fbe37400700 1 -- 172.21.15.112:0/2011403228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c8980 msgr2=0x55a3d40c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:00.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.724+0000 7fbe37400700 1 --2- 172.21.15.112:0/2011403228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c8980 0x55a3d40c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.724+0000 7fbe37400700 1 -- 172.21.15.112:0/2011403228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c4010 msgr2=0x55a3d40cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:00.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.724+0000 7fbe37400700 1 --2- 172.21.15.112:0/2011403228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c4010 0x55a3d40cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.724+0000 7fbe37400700 1 -- 172.21.15.112:0/2011403228 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d40bf220 con 0x55a3d40c4930 2024-05-08T05:54:00.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.724+0000 7fbe37400700 1 --2- 172.21.15.112:0/2011403228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4930 0x55a3d40cbff0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55a3d40cdcf0 tx=0x7fbe28009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8177a9e9a8c4d85 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:00.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.724+0000 7fbe34c00700 1 -- 172.21.15.112:0/2011403228 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe280051d0 con 0x55a3d40c4930 2024-05-08T05:54:00.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.724+0000 7fbe34c00700 1 -- 172.21.15.112:0/2011403228 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe28003680 con 0x55a3d40c4930 2024-05-08T05:54:00.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.724+0000 7fbe34c00700 1 -- 172.21.15.112:0/2011403228 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2801a6a0 con 0x55a3d40c4930 2024-05-08T05:54:00.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.725+0000 7fbe5154db80 1 -- 172.21.15.112:0/2011403228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4930 msgr2=0x55a3d40cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:00.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.725+0000 7fbe5154db80 1 --2- 172.21.15.112:0/2011403228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4930 0x55a3d40cbff0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55a3d40cdcf0 tx=0x7fbe28009990 comp rx=0 tx=0).stop 2024-05-08T05:54:00.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.725+0000 7fbe5154db80 1 -- 172.21.15.112:0/2011403228 shutdown_connections 2024-05-08T05:54:00.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.725+0000 7fbe5154db80 1 --2- 172.21.15.112:0/2011403228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c8980 0x55a3d40c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.725+0000 7fbe5154db80 1 --2- 172.21.15.112:0/2011403228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c4010 0x55a3d40cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.725+0000 7fbe5154db80 1 --2- 172.21.15.112:0/2011403228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c4930 0x55a3d40cbff0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.725+0000 7fbe5154db80 1 -- 172.21.15.112:0/2011403228 >> 172.21.15.112:0/2011403228 conn(0x55a3d3ff2b10 msgr2=0x55a3d40cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.725+0000 7fbe5154db80 1 -- 172.21.15.112:0/2011403228 shutdown_connections 2024-05-08T05:54:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.725+0000 7fbe5154db80 1 -- 172.21.15.112:0/2011403228 wait complete. 2024-05-08T05:54:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.726+0000 7fbe5154db80 1 -- start start 2024-05-08T05:54:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.726+0000 7fbe5154db80 1 Processor -- start 2024-05-08T05:54:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.726+0000 7fbe5154db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4010 0x55a3d40e83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.726+0000 7fbe5154db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c4930 0x55a3d40e8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.726+0000 7fbe5154db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c8980 0x55a3d40e1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.726+0000 7fbe5154db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a3d40dedc0 con 0x55a3d40c4930 2024-05-08T05:54:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.726+0000 7fbe5154db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a3d40de6f0 con 0x55a3d40c8980 2024-05-08T05:54:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.726+0000 7fbe5154db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a3d40df490 con 0x55a3d40c4010 2024-05-08T05:54:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.727+0000 7fbe5154db80 10 client.0 _ll_register_callbacks cb 0x55a3d3fbdc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:00.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.727+0000 7fbe37400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c4930 0x55a3d40e8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.727+0000 7fbe3fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4010 0x55a3d40e83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.727+0000 7fbe37400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c4930 0x55a3d40e8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41378/0 (socket says 172.21.15.112:41378) 2024-05-08T05:54:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.727+0000 7fbe37400700 1 -- 172.21.15.112:0/3962990666 learned_addr learned my addr 172.21.15.112:0/3962990666 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.727+0000 7fbe44c00700 1 --2- 172.21.15.112:0/3962990666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c8980 0x55a3d40e1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.727+0000 7fbe3fe00700 1 -- 172.21.15.112:0/3962990666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c8980 msgr2=0x55a3d40e1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.727+0000 7fbe3fe00700 1 --2- 172.21.15.112:0/3962990666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c8980 0x55a3d40e1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.727+0000 7fbe3fe00700 1 -- 172.21.15.112:0/3962990666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c4930 msgr2=0x55a3d40e8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.727+0000 7fbe3fe00700 1 --2- 172.21.15.112:0/3962990666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c4930 0x55a3d40e8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.727+0000 7fbe3fe00700 1 -- 172.21.15.112:0/3962990666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d40e1a10 con 0x55a3d40c4010 2024-05-08T05:54:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.727+0000 7fbe37400700 1 --2- 172.21.15.112:0/3962990666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c4930 0x55a3d40e8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:00.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.727+0000 7fbe44c00700 1 --2- 172.21.15.112:0/3962990666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c8980 0x55a3d40e1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.728+0000 7fbe3fe00700 1 --2- 172.21.15.112:0/3962990666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4010 0x55a3d40e83c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbe30005e90 tx=0x7fbe30005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa5d0550d89eb85b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.728+0000 7fbe2e000700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:54:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.728+0000 7fbe2e000700 1 -- 172.21.15.112:0/3962990666 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3006d040 con 0x55a3d40c4010 2024-05-08T05:54:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.728+0000 7fbe2e000700 1 -- 172.21.15.112:0/3962990666 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fbe300e0740 con 0x55a3d40c4010 2024-05-08T05:54:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.728+0000 7fbe5154db80 1 -- 172.21.15.112:0/3962990666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe28003680 con 0x55a3d40c4010 2024-05-08T05:54:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.728+0000 7fbe2e000700 1 -- 172.21.15.112:0/3962990666 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbe300a9d80 con 0x55a3d40c4010 2024-05-08T05:54:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.728+0000 7fbe5154db80 10 client.24925 fetch_fsmap learned FSMap version 24 2024-05-08T05:54:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.728+0000 7fbe5154db80 1 -- 172.21.15.112:0/3962990666 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a3d41498e0 con 0x55a3d40c4010 2024-05-08T05:54:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.728+0000 7fbe2e000700 1 -- 172.21.15.112:0/3962990666 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe28003680 con 0x55a3d40c4010 2024-05-08T05:54:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.728+0000 7fbe2e000700 1 -- 172.21.15.112:0/3962990666 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fbe30069ce0 con 0x55a3d40c4010 2024-05-08T05:54:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.729+0000 7fbe5154db80 10 client.24925 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T05:54:00.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.729+0000 7fbe5154db80 10 client.24925 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:54:00.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.729+0000 7fbe5154db80 20 client.24925 populate_metadata read hostname 'smithi112' 2024-05-08T05:54:00.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.729+0000 7fbe5154db80 1 --2- 172.21.15.112:0/3962990666 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a3d414e360 0x55a3d414d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.729+0000 7fbe5154db80 4 client.24925 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:54:00.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.729+0000 7fbe5154db80 1 -- 172.21.15.112:0/3962990666 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "status"}) -- 0x55a3d40243b0 con 0x55a3d414e360 2024-05-08T05:54:00.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.729+0000 7fbe37400700 1 --2- 172.21.15.112:0/3962990666 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a3d414e360 0x55a3d414d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.730+0000 7fbe37400700 1 --2- 172.21.15.112:0/3962990666 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a3d414e360 0x55a3d414d070 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f9ad34bfd19938 server_cookie=939ae309ecf621d7 in_seq=0 out_seq=0 2024-05-08T05:54:00.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.730+0000 7fbe2e000700 10 client.24925 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:54:00.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.730+0000 7fbe2e000700 1 -- 172.21.15.112:0/3962990666 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a3d40243b0 con 0x55a3d414e360 2024-05-08T05:54:00.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.730+0000 7fbe2e000700 10 client.24925 handle_command_reply: tid=0 2024-05-08T05:54:00.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.730+0000 7fbe5154db80 1 client.24925 shutdown 2024-05-08T05:54:00.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.730+0000 7fbe5154db80 1 -- 172.21.15.112:0/3962990666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4010 msgr2=0x55a3d40e83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:00.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.730+0000 7fbe5154db80 1 --2- 172.21.15.112:0/3962990666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4010 0x55a3d40e83c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbe30005e90 tx=0x7fbe30005720 comp rx=0 tx=0).stop 2024-05-08T05:54:00.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.730+0000 7fbe5154db80 1 -- 172.21.15.112:0/3962990666 shutdown_connections 2024-05-08T05:54:00.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.730+0000 7fbe5154db80 1 --2- 172.21.15.112:0/3962990666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3d40c8980 0x55a3d40e1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.730+0000 7fbe5154db80 1 --2- 172.21.15.112:0/3962990666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3d40c4010 0x55a3d40e83c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.730+0000 7fbe5154db80 1 --2- 172.21.15.112:0/3962990666 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a3d414e360 0x55a3d414d070 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.730+0000 7fbe5154db80 1 --2- 172.21.15.112:0/3962990666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a3d40c4930 0x55a3d40e8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.730+0000 7fbe2e000700 0 client.24925 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:54:00.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.730+0000 7fbe5154db80 1 -- 172.21.15.112:0/3962990666 >> 172.21.15.112:0/3962990666 conn(0x55a3d3ff2b10 msgr2=0x55a3d40b89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:00.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.731+0000 7fbe5154db80 1 -- 172.21.15.112:0/3962990666 shutdown_connections 2024-05-08T05:54:00.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.731+0000 7fbe5154db80 1 -- 172.21.15.112:0/3962990666 wait complete. 2024-05-08T05:54:00.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.731+0000 7fbe5154db80 20 client.24925 trim_cache size 0 max 16384 2024-05-08T05:54:00.735 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:54:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "7a07ff60-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:54:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-05-08T05:54:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24529, 2024-05-08T05:54:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-08T05:54:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-08T05:54:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-08T05:54:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 173.405858669, 2024-05-08T05:54:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 24, 2024-05-08T05:54:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 84, 2024-05-08T05:54:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 80, 2024-05-08T05:54:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 197.45216054100001 2024-05-08T05:54:00.736 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:54:00.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.731+0000 7fbe46c00700 1 -- 172.21.15.112:0/2412499962 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbe240b2880 msgr2=0x7fbe240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:00.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.731+0000 7fbe46c00700 1 --2- 172.21.15.112:0/2412499962 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbe240b2880 0x7fbe240b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbe380060b0 tx=0x7fbe38004b40 comp rx=0 tx=0).stop 2024-05-08T05:54:00.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.731+0000 7fbe46c00700 1 -- 172.21.15.112:0/2412499962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe400756d0 msgr2=0x7fbe401a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:00.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.731+0000 7fbe46c00700 1 --2- 172.21.15.112:0/2412499962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe400756d0 0x7fbe401a01b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000f990 tx=0x7fbe3000f9c0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.732+0000 7fbe46c00700 1 -- 172.21.15.112:0/2412499962 shutdown_connections 2024-05-08T05:54:00.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.732+0000 7fbe46c00700 1 --2- 172.21.15.112:0/2412499962 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbe240b2880 0x7fbe240b4d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.732+0000 7fbe46c00700 1 --2- 172.21.15.112:0/2412499962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe40114670 0x7fbe401a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.732+0000 7fbe46c00700 1 --2- 172.21.15.112:0/2412499962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe400756d0 0x7fbe401a01b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.732+0000 7fbe46c00700 1 --2- 172.21.15.112:0/2412499962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe40075ff0 0x7fbe401a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.732+0000 7fbe46c00700 1 -- 172.21.15.112:0/2412499962 >> 172.21.15.112:0/2412499962 conn(0x7fbe4007a600 msgr2=0x7fbe40111510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:00.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.732+0000 7fbe46c00700 1 -- 172.21.15.112:0/2412499962 shutdown_connections 2024-05-08T05:54:00.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:00.732+0000 7fbe46c00700 1 -- 172.21.15.112:0/2412499962 wait complete. 2024-05-08T05:54:01.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:01 smithi112 ceph-mon[30003]: pgmap v402: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:01.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:01 smithi162 ceph-mon[34800]: pgmap v402: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:01.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:01 smithi150 ceph-mon[34315]: pgmap v402: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:01.780 INFO:tasks.cephadm:Setup complete, yielding 2024-05-08T05:54:01.781 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T05:54:01.791 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-05-08T05:54:01.791 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-08T05:54:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:03.409 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:03 smithi112 ceph-mon[30003]: pgmap v403: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:03.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:03 smithi162 ceph-mon[34800]: pgmap v403: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:03.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:03 smithi150 ceph-mon[34315]: pgmap v403: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:04.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.430+0000 7f4e73400700 1 -- 172.21.15.112:0/2043196280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c106940 msgr2=0x7f4e6c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:04.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.430+0000 7f4e73400700 1 --2- 172.21.15.112:0/2043196280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c106940 0x7f4e6c106da0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4e580099c0 tx=0x7f4e58009cd0 comp rx=0 tx=0).stop 2024-05-08T05:54:04.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.430+0000 7f4e73400700 1 -- 172.21.15.112:0/2043196280 shutdown_connections 2024-05-08T05:54:04.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.430+0000 7f4e73400700 1 --2- 172.21.15.112:0/2043196280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e6c1072e0 0x7f4e6c10db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:04.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.430+0000 7f4e73400700 1 --2- 172.21.15.112:0/2043196280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c105740 0x7f4e6c105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:04.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.430+0000 7f4e73400700 1 --2- 172.21.15.112:0/2043196280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c106940 0x7f4e6c106da0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:04.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.430+0000 7f4e73400700 1 -- 172.21.15.112:0/2043196280 >> 172.21.15.112:0/2043196280 conn(0x7f4e6c100d00 msgr2=0x7f4e6c103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:04.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.430+0000 7f4e73400700 1 -- 172.21.15.112:0/2043196280 shutdown_connections 2024-05-08T05:54:04.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.430+0000 7f4e73400700 1 -- 172.21.15.112:0/2043196280 wait complete. 2024-05-08T05:54:04.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.432+0000 7f4e73400700 1 Processor -- start 2024-05-08T05:54:04.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.432+0000 7f4e73400700 1 -- start start 2024-05-08T05:54:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.432+0000 7f4e73400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c105740 0x7f4e6c19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.432+0000 7f4e73400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c106940 0x7f4e6c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.433+0000 7f4e73400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e6c1072e0 0x7f4e6c1a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.433+0000 7f4e73400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1111b0 con 0x7f4e6c106940 2024-05-08T05:54:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.433+0000 7f4e73400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c111030 con 0x7f4e6c105740 2024-05-08T05:54:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.433+0000 7f4e73400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c111330 con 0x7f4e6c1072e0 2024-05-08T05:54:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.433+0000 7f4e70a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c105740 0x7f4e6c19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.433+0000 7f4e70a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c105740 0x7f4e6c19b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:37430/0 (socket says 172.21.15.112:37430) 2024-05-08T05:54:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.433+0000 7f4e70a00700 1 -- 172.21.15.112:0/1983118247 learned_addr learned my addr 172.21.15.112:0/1983118247 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.433+0000 7f4e6be00700 1 --2- 172.21.15.112:0/1983118247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c106940 0x7f4e6c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.433+0000 7f4e71400700 1 --2- 172.21.15.112:0/1983118247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e6c1072e0 0x7f4e6c1a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:04.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.433+0000 7f4e70a00700 1 -- 172.21.15.112:0/1983118247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e6c1072e0 msgr2=0x7f4e6c1a2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.433+0000 7f4e70a00700 1 --2- 172.21.15.112:0/1983118247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e6c1072e0 0x7f4e6c1a2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.433+0000 7f4e70a00700 1 -- 172.21.15.112:0/1983118247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c106940 msgr2=0x7f4e6c19bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.433+0000 7f4e70a00700 1 --2- 172.21.15.112:0/1983118247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c106940 0x7f4e6c19bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.433+0000 7f4e70a00700 1 -- 172.21.15.112:0/1983118247 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6c1a3450 con 0x7f4e6c105740 2024-05-08T05:54:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.434+0000 7f4e70a00700 1 --2- 172.21.15.112:0/1983118247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c105740 0x7f4e6c19b7f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000ea50 tx=0x7f4e6000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.434+0000 7f4e69600700 1 -- 172.21.15.112:0/1983118247 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6000cbd0 con 0x7f4e6c105740 2024-05-08T05:54:04.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.434+0000 7f4e69600700 1 -- 172.21.15.112:0/1983118247 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e6000cd70 con 0x7f4e6c105740 2024-05-08T05:54:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.434+0000 7f4e73400700 1 -- 172.21.15.112:0/1983118247 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6c1a3740 con 0x7f4e6c105740 2024-05-08T05:54:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.434+0000 7f4e69600700 1 -- 172.21.15.112:0/1983118247 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e600186f0 con 0x7f4e6c105740 2024-05-08T05:54:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.434+0000 7f4e73400700 1 -- 172.21.15.112:0/1983118247 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e6c1a3cd0 con 0x7f4e6c105740 2024-05-08T05:54:04.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.435+0000 7f4e73400700 1 -- 172.21.15.112:0/1983118247 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e6c06a190 con 0x7f4e6c105740 2024-05-08T05:54:04.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.437+0000 7f4e69600700 1 -- 172.21.15.112:0/1983118247 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4e6001f020 con 0x7f4e6c105740 2024-05-08T05:54:04.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.439+0000 7f4e69600700 1 --2- 172.21.15.112:0/1983118247 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4e540b2900 0x7f4e540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:04.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.439+0000 7f4e69600700 1 -- 172.21.15.112:0/1983118247 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f4e600e2240 con 0x7f4e6c105740 2024-05-08T05:54:04.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.439+0000 7f4e6be00700 1 --2- 172.21.15.112:0/1983118247 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4e540b2900 0x7f4e540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:04.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.440+0000 7f4e6be00700 1 --2- 172.21.15.112:0/1983118247 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4e540b2900 0x7f4e540b4dc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4e58000c00 tx=0x7f4e58005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:04.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.443+0000 7f4e69600700 1 -- 172.21.15.112:0/1983118247 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4e600a9ed0 con 0x7f4e6c105740 2024-05-08T05:54:04.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.697+0000 7f4e73400700 1 -- 172.21.15.112:0/1983118247 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f4e6c0638e0 con 0x7f4e540b2900 2024-05-08T05:54:04.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.700+0000 7f4e69600700 1 -- 172.21.15.112:0/1983118247 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f4e6c0638e0 con 0x7f4e540b2900 2024-05-08T05:54:04.700 INFO:teuthology.orchestra.run.smithi112.stdout:Backend: cephadm 2024-05-08T05:54:04.700 INFO:teuthology.orchestra.run.smithi112.stdout:Available: Yes 2024-05-08T05:54:04.700 INFO:teuthology.orchestra.run.smithi112.stdout:Paused: No 2024-05-08T05:54:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.704+0000 7f4e73400700 1 -- 172.21.15.112:0/1983118247 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4e540b2900 msgr2=0x7f4e540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.704+0000 7f4e73400700 1 --2- 172.21.15.112:0/1983118247 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4e540b2900 0x7f4e540b4dc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4e58000c00 tx=0x7f4e58005fd0 comp rx=0 tx=0).stop 2024-05-08T05:54:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.704+0000 7f4e73400700 1 -- 172.21.15.112:0/1983118247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c105740 msgr2=0x7f4e6c19b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.704+0000 7f4e73400700 1 --2- 172.21.15.112:0/1983118247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c105740 0x7f4e6c19b7f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000ea50 tx=0x7f4e6000ed60 comp rx=0 tx=0).stop 2024-05-08T05:54:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.704+0000 7f4e73400700 1 -- 172.21.15.112:0/1983118247 shutdown_connections 2024-05-08T05:54:04.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.704+0000 7f4e73400700 1 --2- 172.21.15.112:0/1983118247 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4e540b2900 0x7f4e540b4dc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:04.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.704+0000 7f4e73400700 1 --2- 172.21.15.112:0/1983118247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e6c1072e0 0x7f4e6c1a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:04.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.704+0000 7f4e73400700 1 --2- 172.21.15.112:0/1983118247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c105740 0x7f4e6c19b7f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:04.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.704+0000 7f4e73400700 1 --2- 172.21.15.112:0/1983118247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c106940 0x7f4e6c19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:04.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.704+0000 7f4e73400700 1 -- 172.21.15.112:0/1983118247 >> 172.21.15.112:0/1983118247 conn(0x7f4e6c100d00 msgr2=0x7f4e6c109c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:04.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.704+0000 7f4e73400700 1 -- 172.21.15.112:0/1983118247 shutdown_connections 2024-05-08T05:54:04.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:04.704+0000 7f4e73400700 1 -- 172.21.15.112:0/1983118247 wait complete. 2024-05-08T05:54:05.434 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:05 smithi112 ceph-mon[30003]: pgmap v404: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:05.500 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-08T05:54:05.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:05 smithi162 ceph-mon[34800]: pgmap v404: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:05.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:05 smithi150 ceph-mon[34315]: pgmap v404: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:05.938 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:06.447 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:06 smithi112 ceph-mon[30003]: from='client.24908 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:54:06.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:06 smithi162 ceph-mon[34800]: from='client.24908 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:54:06.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:06 smithi150 ceph-mon[34315]: from='client.24908 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:54:07.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:07 smithi112 ceph-mon[30003]: pgmap v405: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:07.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:07 smithi162 ceph-mon[34800]: pgmap v405: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:07 smithi150 ceph-mon[34315]: pgmap v405: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:08.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.310+0000 7f6f87a00700 1 -- 172.21.15.112:0/843946768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f80103f90 msgr2=0x7f6f80106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:08.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.310+0000 7f6f87a00700 1 --2- 172.21.15.112:0/843946768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f80103f90 0x7f6f80106380 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6f74009990 tx=0x7f6f74009ca0 comp rx=0 tx=0).stop 2024-05-08T05:54:08.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.310+0000 7f6f87a00700 1 -- 172.21.15.112:0/843946768 shutdown_connections 2024-05-08T05:54:08.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.310+0000 7f6f87a00700 1 --2- 172.21.15.112:0/843946768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f801091f0 0x7f6f8010b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:08.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.310+0000 7f6f87a00700 1 --2- 172.21.15.112:0/843946768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f80103f90 0x7f6f80106380 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:08.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.310+0000 7f6f87a00700 1 --2- 172.21.15.112:0/843946768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f801068c0 0x7f6f80108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:08.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.310+0000 7f6f87a00700 1 -- 172.21.15.112:0/843946768 >> 172.21.15.112:0/843946768 conn(0x7f6f800fd9a0 msgr2=0x7f6f800ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:08.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.310+0000 7f6f87a00700 1 -- 172.21.15.112:0/843946768 shutdown_connections 2024-05-08T05:54:08.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.310+0000 7f6f87a00700 1 -- 172.21.15.112:0/843946768 wait complete. 2024-05-08T05:54:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.311+0000 7f6f87a00700 1 Processor -- start 2024-05-08T05:54:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.312+0000 7f6f87a00700 1 -- start start 2024-05-08T05:54:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.312+0000 7f6f87a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f80103f90 0x7f6f801038f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.312+0000 7f6f87a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f801068c0 0x7f6f80101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.312+0000 7f6f87a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f801091f0 0x7f6f80102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:08.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.312+0000 7f6f87a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8010f010 con 0x7f6f80103f90 2024-05-08T05:54:08.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.312+0000 7f6f87a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8010ee90 con 0x7f6f801068c0 2024-05-08T05:54:08.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.312+0000 7f6f87a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8010f190 con 0x7f6f801091f0 2024-05-08T05:54:08.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.312+0000 7f6f85000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f80103f90 0x7f6f801038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:08.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.312+0000 7f6f7fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f801068c0 0x7f6f80101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:08.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.312+0000 7f6f85000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f80103f90 0x7f6f801038f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56364/0 (socket says 172.21.15.112:56364) 2024-05-08T05:54:08.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.312+0000 7f6f7fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f801068c0 0x7f6f80101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:37456/0 (socket says 172.21.15.112:37456) 2024-05-08T05:54:08.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.312+0000 7f6f85000700 1 -- 172.21.15.112:0/2230991376 learned_addr learned my addr 172.21.15.112:0/2230991376 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:08.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.312+0000 7f6f85a00700 1 --2- 172.21.15.112:0/2230991376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f801091f0 0x7f6f80102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:08.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.313+0000 7f6f7fe00700 1 -- 172.21.15.112:0/2230991376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f801091f0 msgr2=0x7f6f80102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:08.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.313+0000 7f6f7fe00700 1 --2- 172.21.15.112:0/2230991376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f801091f0 0x7f6f80102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:08.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.313+0000 7f6f7fe00700 1 -- 172.21.15.112:0/2230991376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f80103f90 msgr2=0x7f6f801038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:08.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.313+0000 7f6f7fe00700 1 --2- 172.21.15.112:0/2230991376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f80103f90 0x7f6f801038f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:08.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.313+0000 7f6f7fe00700 1 -- 172.21.15.112:0/2230991376 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f80102d30 con 0x7f6f801068c0 2024-05-08T05:54:08.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.313+0000 7f6f85a00700 1 --2- 172.21.15.112:0/2230991376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f801091f0 0x7f6f80102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:08.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.313+0000 7f6f85000700 1 --2- 172.21.15.112:0/2230991376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f80103f90 0x7f6f801038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:08.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.313+0000 7f6f7fe00700 1 --2- 172.21.15.112:0/2230991376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f801068c0 0x7f6f80101f40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000e9f0 tx=0x7f6f7000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.313+0000 7f6f7d600700 1 -- 172.21.15.112:0/2230991376 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70004220 con 0x7f6f801068c0 2024-05-08T05:54:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.313+0000 7f6f7d600700 1 -- 172.21.15.112:0/2230991376 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f700043c0 con 0x7f6f801068c0 2024-05-08T05:54:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.313+0000 7f6f87a00700 1 -- 172.21.15.112:0/2230991376 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f801a3890 con 0x7f6f801068c0 2024-05-08T05:54:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.314+0000 7f6f7d600700 1 -- 172.21.15.112:0/2230991376 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7000ce60 con 0x7f6f801068c0 2024-05-08T05:54:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.314+0000 7f6f87a00700 1 -- 172.21.15.112:0/2230991376 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f801a3da0 con 0x7f6f801068c0 2024-05-08T05:54:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.315+0000 7f6f87a00700 1 -- 172.21.15.112:0/2230991376 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f8006a190 con 0x7f6f801068c0 2024-05-08T05:54:08.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.317+0000 7f6f7d600700 1 -- 172.21.15.112:0/2230991376 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6f700052f0 con 0x7f6f801068c0 2024-05-08T05:54:08.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.319+0000 7f6f7d600700 1 --2- 172.21.15.112:0/2230991376 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6f680b2900 0x7f6f680b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:08.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.319+0000 7f6f7d600700 1 -- 172.21.15.112:0/2230991376 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f6f700e2390 con 0x7f6f801068c0 2024-05-08T05:54:08.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.319+0000 7f6f85000700 1 --2- 172.21.15.112:0/2230991376 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6f680b2900 0x7f6f680b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:08.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.320+0000 7f6f85000700 1 --2- 172.21.15.112:0/2230991376 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6f680b2900 0x7f6f680b4dc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6f7400b2e0 tx=0x7f6f7401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:08.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.323+0000 7f6f7d600700 1 -- 172.21.15.112:0/2230991376 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6f700aa020 con 0x7f6f801068c0 2024-05-08T05:54:08.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.566+0000 7f6f87a00700 1 -- 172.21.15.112:0/2230991376 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6f800638e0 con 0x7f6f680b2900 2024-05-08T05:54:08.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.583+0000 7f6f7d600700 1 -- 172.21.15.112:0/2230991376 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f6f800638e0 con 0x7f6f680b2900 2024-05-08T05:54:08.584 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-08T05:54:08.584 INFO:teuthology.orchestra.run.smithi112.stdout:mds.a smithi112 running (3m) 46s ago 3m 25.9M - 19.0.0-3538-gbee299f4 352576a2df49 b7a24dd92d92 2024-05-08T05:54:08.584 INFO:teuthology.orchestra.run.smithi112.stdout:mds.b smithi150 running (3m) 65s ago 3m 21.3M - 19.0.0-3538-gbee299f4 352576a2df49 7dc66aadba57 2024-05-08T05:54:08.584 INFO:teuthology.orchestra.run.smithi112.stdout:mds.c smithi162 running (3m) 65s ago 3m 19.4M - 19.0.0-3538-gbee299f4 352576a2df49 e053083a8d5f 2024-05-08T05:54:08.584 INFO:teuthology.orchestra.run.smithi112.stdout:mds.d smithi112 running (3m) 46s ago 3m 13.3M - 19.0.0-3538-gbee299f4 352576a2df49 cb81d29a3c70 2024-05-08T05:54:08.584 INFO:teuthology.orchestra.run.smithi112.stdout:mds.e smithi150 running (3m) 65s ago 3m 13.7M - 19.0.0-3538-gbee299f4 352576a2df49 92a9e02d224d 2024-05-08T05:54:08.584 INFO:teuthology.orchestra.run.smithi112.stdout:mds.f smithi162 running (3m) 65s ago 3m 18.7M - 19.0.0-3538-gbee299f4 352576a2df49 f14a5f6fd940 2024-05-08T05:54:08.584 INFO:teuthology.orchestra.run.smithi112.stdout:mds.g smithi112 running (3m) 46s ago 3m 13.2M - 19.0.0-3538-gbee299f4 352576a2df49 f5f0933b3f22 2024-05-08T05:54:08.584 INFO:teuthology.orchestra.run.smithi112.stdout:mds.h smithi150 running (3m) 65s ago 3m 21.1M - 19.0.0-3538-gbee299f4 352576a2df49 cb34aefd585e 2024-05-08T05:54:08.584 INFO:teuthology.orchestra.run.smithi112.stdout:mds.i smithi162 running (3m) 65s ago 3m 21.4M - 19.0.0-3538-gbee299f4 352576a2df49 2f5505b532fa 2024-05-08T05:54:08.585 INFO:teuthology.orchestra.run.smithi112.stdout:mds.j smithi112 running (3m) 46s ago 3m 13.3M - 19.0.0-3538-gbee299f4 352576a2df49 340658b200f7 2024-05-08T05:54:08.585 INFO:teuthology.orchestra.run.smithi112.stdout:mds.k smithi150 running (3m) 65s ago 3m 14.7M - 19.0.0-3538-gbee299f4 352576a2df49 16086b0e30d0 2024-05-08T05:54:08.585 INFO:teuthology.orchestra.run.smithi112.stdout:mds.l smithi162 running (4m) 65s ago 4m 17.0M - 19.0.0-3538-gbee299f4 352576a2df49 0ecda3dd22a5 2024-05-08T05:54:08.585 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.x smithi112 *:9283,8765 running (12m) 46s ago 12m 530M - 19.0.0-3538-gbee299f4 352576a2df49 935065eb06df 2024-05-08T05:54:08.585 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.y smithi150 *:8443,8765 running (9m) 65s ago 9m 469M - 19.0.0-3538-gbee299f4 352576a2df49 fb487a1da7df 2024-05-08T05:54:08.585 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.z smithi162 *:8443,8765 running (9m) 65s ago 9m 470M - 19.0.0-3538-gbee299f4 352576a2df49 fadf6de37aa0 2024-05-08T05:54:08.585 INFO:teuthology.orchestra.run.smithi112.stdout:mon.a smithi112 running (12m) 46s ago 12m 49.3M 2048M 19.0.0-3538-gbee299f4 352576a2df49 11e2e06de9b6 2024-05-08T05:54:08.585 INFO:teuthology.orchestra.run.smithi112.stdout:mon.b smithi150 running (10m) 65s ago 10m 41.5M 2048M 19.0.0-3538-gbee299f4 352576a2df49 7241b4923745 2024-05-08T05:54:08.585 INFO:teuthology.orchestra.run.smithi112.stdout:mon.c smithi162 running (10m) 65s ago 10m 40.1M 2048M 19.0.0-3538-gbee299f4 352576a2df49 9d8c1bd591f2 2024-05-08T05:54:08.585 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (9m) 46s ago 9m 48.6M 4272M 19.0.0-3538-gbee299f4 352576a2df49 c1d9f0ef0df4 2024-05-08T05:54:08.585 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi150 running (8m) 65s ago 8m 58.6M 4272M 19.0.0-3538-gbee299f4 352576a2df49 4c5a4a89b254 2024-05-08T05:54:08.585 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi162 running (8m) 65s ago 8m 57.5M 4272M 19.0.0-3538-gbee299f4 352576a2df49 e810072f7718 2024-05-08T05:54:08.585 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi112 running (7m) 46s ago 7m 46.7M 4272M 19.0.0-3538-gbee299f4 352576a2df49 48d5c4d03c79 2024-05-08T05:54:08.585 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi150 running (7m) 65s ago 7m 46.7M 4272M 19.0.0-3538-gbee299f4 352576a2df49 ee01b6b84637 2024-05-08T05:54:08.586 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi162 running (7m) 65s ago 7m 54.1M 4272M 19.0.0-3538-gbee299f4 352576a2df49 f10d6d2b9330 2024-05-08T05:54:08.586 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 smithi112 running (6m) 46s ago 6m 47.4M 4272M 19.0.0-3538-gbee299f4 352576a2df49 a97c5b3e4592 2024-05-08T05:54:08.586 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 smithi150 running (6m) 65s ago 6m 47.1M 4272M 19.0.0-3538-gbee299f4 352576a2df49 94f25f634d84 2024-05-08T05:54:08.586 INFO:teuthology.orchestra.run.smithi112.stdout:osd.8 smithi162 running (5m) 65s ago 5m 57.7M 4272M 19.0.0-3538-gbee299f4 352576a2df49 4853414ed854 2024-05-08T05:54:08.586 INFO:teuthology.orchestra.run.smithi112.stdout:osd.9 smithi112 running (5m) 46s ago 5m 45.2M 4272M 19.0.0-3538-gbee299f4 352576a2df49 e0450fefd79c 2024-05-08T05:54:08.586 INFO:teuthology.orchestra.run.smithi112.stdout:osd.10 smithi150 running (4m) 65s ago 4m 48.2M 4272M 19.0.0-3538-gbee299f4 352576a2df49 b0b81ec35de9 2024-05-08T05:54:08.586 INFO:teuthology.orchestra.run.smithi112.stdout:osd.11 smithi162 running (4m) 65s ago 4m 44.2M 4272M 19.0.0-3538-gbee299f4 352576a2df49 0e0f0951de72 2024-05-08T05:54:08.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.586+0000 7f6f87a00700 1 -- 172.21.15.112:0/2230991376 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6f680b2900 msgr2=0x7f6f680b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:08.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.586+0000 7f6f87a00700 1 --2- 172.21.15.112:0/2230991376 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6f680b2900 0x7f6f680b4dc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6f7400b2e0 tx=0x7f6f7401a040 comp rx=0 tx=0).stop 2024-05-08T05:54:08.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.586+0000 7f6f87a00700 1 -- 172.21.15.112:0/2230991376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f801068c0 msgr2=0x7f6f80101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:08.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.586+0000 7f6f87a00700 1 --2- 172.21.15.112:0/2230991376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f801068c0 0x7f6f80101f40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000e9f0 tx=0x7f6f7000ed00 comp rx=0 tx=0).stop 2024-05-08T05:54:08.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.586+0000 7f6f87a00700 1 -- 172.21.15.112:0/2230991376 shutdown_connections 2024-05-08T05:54:08.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.586+0000 7f6f87a00700 1 --2- 172.21.15.112:0/2230991376 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6f680b2900 0x7f6f680b4dc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:08.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.586+0000 7f6f87a00700 1 --2- 172.21.15.112:0/2230991376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f801091f0 0x7f6f80102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:08.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.586+0000 7f6f87a00700 1 --2- 172.21.15.112:0/2230991376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f80103f90 0x7f6f801038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:08.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.586+0000 7f6f87a00700 1 --2- 172.21.15.112:0/2230991376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f801068c0 0x7f6f80101f40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:08.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.586+0000 7f6f87a00700 1 -- 172.21.15.112:0/2230991376 >> 172.21.15.112:0/2230991376 conn(0x7f6f800fd9a0 msgr2=0x7f6f801048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:08.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.586+0000 7f6f87a00700 1 -- 172.21.15.112:0/2230991376 shutdown_connections 2024-05-08T05:54:08.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:08.586+0000 7f6f87a00700 1 -- 172.21.15.112:0/2230991376 wait complete. 2024-05-08T05:54:09.330 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-08T05:54:09.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:09 smithi112 ceph-mon[30003]: pgmap v406: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:09.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:09 smithi162 ceph-mon[34800]: pgmap v406: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:09 smithi150 ceph-mon[34315]: pgmap v406: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:09.778 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:10.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:10 smithi112 ceph-mon[30003]: from='client.24914 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:54:10.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:10 smithi162 ceph-mon[34800]: from='client.24914 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:54:10.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:10 smithi150 ceph-mon[34315]: from='client.24914 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:54:11.522 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:11 smithi112 ceph-mon[30003]: pgmap v407: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:11.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:11 smithi162 ceph-mon[34800]: pgmap v407: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:11 smithi150 ceph-mon[34315]: pgmap v407: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:11.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.805+0000 7f019da00700 1 -- 172.21.15.112:0/98472162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0198105640 msgr2=0x7f0198105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:11.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.805+0000 7f019da00700 1 --2- 172.21.15.112:0/98472162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0198105640 0x7f0198105a60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f01880099c0 tx=0x7f0188009cd0 comp rx=0 tx=0).stop 2024-05-08T05:54:11.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.805+0000 7f019da00700 1 -- 172.21.15.112:0/98472162 shutdown_connections 2024-05-08T05:54:11.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.805+0000 7f019da00700 1 --2- 172.21.15.112:0/98472162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01981071e0 0x7f019810da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:11.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.805+0000 7f019da00700 1 --2- 172.21.15.112:0/98472162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0198105640 0x7f0198105a60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:11.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.805+0000 7f019da00700 1 --2- 172.21.15.112:0/98472162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0198106840 0x7f0198106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:11.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.805+0000 7f019da00700 1 -- 172.21.15.112:0/98472162 >> 172.21.15.112:0/98472162 conn(0x7f0198100bc0 msgr2=0x7f0198103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:11.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.805+0000 7f019da00700 1 -- 172.21.15.112:0/98472162 shutdown_connections 2024-05-08T05:54:11.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.806+0000 7f019da00700 1 -- 172.21.15.112:0/98472162 wait complete. 2024-05-08T05:54:11.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.806+0000 7f019da00700 1 Processor -- start 2024-05-08T05:54:11.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.807+0000 7f019da00700 1 -- start start 2024-05-08T05:54:11.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.807+0000 7f019da00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0198105640 0x7f019819b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:11.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.807+0000 7f019da00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0198106840 0x7f019819bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:11.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.807+0000 7f019da00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01981071e0 0x7f01981a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:11.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.807+0000 7f019da00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f01981110e0 con 0x7f0198106840 2024-05-08T05:54:11.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.807+0000 7f019da00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0198110f60 con 0x7f01981071e0 2024-05-08T05:54:11.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.807+0000 7f019da00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0198111260 con 0x7f0198105640 2024-05-08T05:54:11.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.807+0000 7f0197400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0198106840 0x7f019819bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:11.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.807+0000 7f0197400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0198106840 0x7f019819bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56392/0 (socket says 172.21.15.112:56392) 2024-05-08T05:54:11.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.807+0000 7f0197e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0198105640 0x7f019819b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:11.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.807+0000 7f0197400700 1 -- 172.21.15.112:0/395224750 learned_addr learned my addr 172.21.15.112:0/395224750 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:11.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.807+0000 7f019d000700 1 --2- 172.21.15.112:0/395224750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01981071e0 0x7f01981a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:11.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.808+0000 7f0197400700 1 -- 172.21.15.112:0/395224750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0198105640 msgr2=0x7f019819b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:11.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.808+0000 7f0197400700 1 --2- 172.21.15.112:0/395224750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0198105640 0x7f019819b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:11.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.808+0000 7f0197400700 1 -- 172.21.15.112:0/395224750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01981071e0 msgr2=0x7f01981a2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:11.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.808+0000 7f0197400700 1 --2- 172.21.15.112:0/395224750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01981071e0 0x7f01981a2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:11.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.808+0000 7f0197400700 1 -- 172.21.15.112:0/395224750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01981a3380 con 0x7f0198106840 2024-05-08T05:54:11.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.808+0000 7f0197400700 1 --2- 172.21.15.112:0/395224750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0198106840 0x7f019819bc60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f018400d9b0 tx=0x7f018400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:11.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.808+0000 7f0194c00700 1 -- 172.21.15.112:0/395224750 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0184016070 con 0x7f0198106840 2024-05-08T05:54:11.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.809+0000 7f0194c00700 1 -- 172.21.15.112:0/395224750 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0184004830 con 0x7f0198106840 2024-05-08T05:54:11.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.809+0000 7f019da00700 1 -- 172.21.15.112:0/395224750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01981a3670 con 0x7f0198106840 2024-05-08T05:54:11.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.809+0000 7f0194c00700 1 -- 172.21.15.112:0/395224750 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0184005440 con 0x7f0198106840 2024-05-08T05:54:11.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.809+0000 7f019da00700 1 -- 172.21.15.112:0/395224750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01981a3c00 con 0x7f0198106840 2024-05-08T05:54:11.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.810+0000 7f019da00700 1 -- 172.21.15.112:0/395224750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f019806a190 con 0x7f0198106840 2024-05-08T05:54:11.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.813+0000 7f0194c00700 1 -- 172.21.15.112:0/395224750 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f01840055e0 con 0x7f0198106840 2024-05-08T05:54:11.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.814+0000 7f0194c00700 1 --2- 172.21.15.112:0/395224750 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f017c0b2900 0x7f017c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:11.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.814+0000 7f0194c00700 1 -- 172.21.15.112:0/395224750 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f01840e1fb0 con 0x7f0198106840 2024-05-08T05:54:11.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.814+0000 7f0197e00700 1 --2- 172.21.15.112:0/395224750 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f017c0b2900 0x7f017c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:11.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.815+0000 7f0197e00700 1 --2- 172.21.15.112:0/395224750 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f017c0b2900 0x7f017c0b4dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f018800b580 tx=0x7f0188005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:11.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:11.819+0000 7f0194c00700 1 -- 172.21.15.112:0/395224750 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f018401f020 con 0x7f0198106840 2024-05-08T05:54:12.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:12.060+0000 7f019da00700 1 -- 172.21.15.112:0/395224750 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f01980638e0 con 0x7f017c0b2900 2024-05-08T05:54:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:12.082+0000 7f0194c00700 1 -- 172.21.15.112:0/395224750 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f01980638e0 con 0x7f017c0b2900 2024-05-08T05:54:12.083 INFO:teuthology.orchestra.run.smithi112.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-08T05:54:12.083 INFO:teuthology.orchestra.run.smithi112.stdout:mds.all 12/12 68s ago 4m smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l;count:12 2024-05-08T05:54:12.083 INFO:teuthology.orchestra.run.smithi112.stdout:mgr 3/3 68s ago 9m smithi112=x;smithi150=y;smithi162=z;count:3 2024-05-08T05:54:12.083 INFO:teuthology.orchestra.run.smithi112.stdout:mon 3/3 68s ago 10m smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi162:172.21.15.162=c;count:3 2024-05-08T05:54:12.083 INFO:teuthology.orchestra.run.smithi112.stdout:osd 12 68s ago - 2024-05-08T05:54:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:12.086+0000 7f019da00700 1 -- 172.21.15.112:0/395224750 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f017c0b2900 msgr2=0x7f017c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:12.086+0000 7f019da00700 1 --2- 172.21.15.112:0/395224750 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f017c0b2900 0x7f017c0b4dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f018800b580 tx=0x7f0188005fb0 comp rx=0 tx=0).stop 2024-05-08T05:54:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:12.086+0000 7f019da00700 1 -- 172.21.15.112:0/395224750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0198106840 msgr2=0x7f019819bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:12.086+0000 7f019da00700 1 --2- 172.21.15.112:0/395224750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0198106840 0x7f019819bc60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f018400d9b0 tx=0x7f018400dcc0 comp rx=0 tx=0).stop 2024-05-08T05:54:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:12.086+0000 7f019da00700 1 -- 172.21.15.112:0/395224750 shutdown_connections 2024-05-08T05:54:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:12.086+0000 7f019da00700 1 --2- 172.21.15.112:0/395224750 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f017c0b2900 0x7f017c0b4dc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:12.086+0000 7f019da00700 1 --2- 172.21.15.112:0/395224750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01981071e0 0x7f01981a2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:12.086+0000 7f019da00700 1 --2- 172.21.15.112:0/395224750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0198105640 0x7f019819b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:12.086+0000 7f019da00700 1 --2- 172.21.15.112:0/395224750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0198106840 0x7f019819bc60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:12.086+0000 7f019da00700 1 -- 172.21.15.112:0/395224750 >> 172.21.15.112:0/395224750 conn(0x7f0198100bc0 msgr2=0x7f0198109bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:12.086+0000 7f019da00700 1 -- 172.21.15.112:0/395224750 shutdown_connections 2024-05-08T05:54:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:12.086+0000 7f019da00700 1 -- 172.21.15.112:0/395224750 wait complete. 2024-05-08T05:54:12.887 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-08T05:54:13.343 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:13.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:13 smithi112 ceph-mon[30003]: pgmap v408: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:13.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:13 smithi112 ceph-mon[30003]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:54:13.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:13 smithi162 ceph-mon[34800]: pgmap v408: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:13.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:13 smithi162 ceph-mon[34800]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:54:13.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:13 smithi150 ceph-mon[34315]: pgmap v408: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:13.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:13 smithi150 ceph-mon[34315]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:54:15.292 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:15 smithi112 ceph-mon[30003]: pgmap v409: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:15.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.539+0000 7fc7cb800700 1 -- 172.21.15.112:0/4026991833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c4075e80 msgr2=0x7fc7c4076300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:15.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.539+0000 7fc7cb800700 1 --2- 172.21.15.112:0/4026991833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c4075e80 0x7fc7c4076300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009990 tx=0x7fc7b0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.539+0000 7fc7cb800700 1 -- 172.21.15.112:0/4026991833 shutdown_connections 2024-05-08T05:54:15.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.539+0000 7fc7cb800700 1 --2- 172.21.15.112:0/4026991833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7c410b400 0x7fc7c410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.539+0000 7fc7cb800700 1 --2- 172.21.15.112:0/4026991833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7c4077450 0x7fc7c40758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.539+0000 7fc7cb800700 1 --2- 172.21.15.112:0/4026991833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c4075e80 0x7fc7c4076300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.539+0000 7fc7cb800700 1 -- 172.21.15.112:0/4026991833 >> 172.21.15.112:0/4026991833 conn(0x7fc7c40feaa0 msgr2=0x7fc7c4100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:15.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.539+0000 7fc7cb800700 1 -- 172.21.15.112:0/4026991833 shutdown_connections 2024-05-08T05:54:15.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.539+0000 7fc7cb800700 1 -- 172.21.15.112:0/4026991833 wait complete. 2024-05-08T05:54:15.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.540+0000 7fc7cb800700 1 Processor -- start 2024-05-08T05:54:15.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.540+0000 7fc7cb800700 1 -- start start 2024-05-08T05:54:15.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.541+0000 7fc7cb800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c4075e80 0x7fc7c419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.541+0000 7fc7cb800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7c4077450 0x7fc7c419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.541+0000 7fc7cb800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7c410b400 0x7fc7c41a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.541+0000 7fc7cb800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c4111160 con 0x7fc7c410b400 2024-05-08T05:54:15.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.541+0000 7fc7cb800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c4110fe0 con 0x7fc7c4077450 2024-05-08T05:54:15.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.541+0000 7fc7cb800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c41112e0 con 0x7fc7c4075e80 2024-05-08T05:54:15.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.541+0000 7fc7c9800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7c410b400 0x7fc7c41a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.541+0000 7fc7c8e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c4075e80 0x7fc7c419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.541+0000 7fc7c3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7c4077450 0x7fc7c419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.541+0000 7fc7c8e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c4075e80 0x7fc7c419b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:53508/0 (socket says 172.21.15.112:53508) 2024-05-08T05:54:15.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.541+0000 7fc7c9800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7c410b400 0x7fc7c41a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39400/0 (socket says 172.21.15.112:39400) 2024-05-08T05:54:15.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.541+0000 7fc7c8e00700 1 -- 172.21.15.112:0/3059094137 learned_addr learned my addr 172.21.15.112:0/3059094137 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:15.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.542+0000 7fc7c8e00700 1 -- 172.21.15.112:0/3059094137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7c4077450 msgr2=0x7fc7c419bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:15.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.542+0000 7fc7c8e00700 1 --2- 172.21.15.112:0/3059094137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7c4077450 0x7fc7c419bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.542+0000 7fc7c8e00700 1 -- 172.21.15.112:0/3059094137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7c410b400 msgr2=0x7fc7c41a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:15.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.542+0000 7fc7c8e00700 1 --2- 172.21.15.112:0/3059094137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7c410b400 0x7fc7c41a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.542+0000 7fc7c8e00700 1 -- 172.21.15.112:0/3059094137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c41a34c0 con 0x7fc7c4075e80 2024-05-08T05:54:15.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.542+0000 7fc7c8e00700 1 --2- 172.21.15.112:0/3059094137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c4075e80 0x7fc7c419b860 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800e9d0 tx=0x7fc7b800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:15.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.542+0000 7fc7c1600700 1 -- 172.21.15.112:0/3059094137 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b800cbd0 con 0x7fc7c4075e80 2024-05-08T05:54:15.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.542+0000 7fc7c1600700 1 -- 172.21.15.112:0/3059094137 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b800ee60 con 0x7fc7c4075e80 2024-05-08T05:54:15.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.542+0000 7fc7c1600700 1 -- 172.21.15.112:0/3059094137 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8018680 con 0x7fc7c4075e80 2024-05-08T05:54:15.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.542+0000 7fc7cb800700 1 -- 172.21.15.112:0/3059094137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c41a37b0 con 0x7fc7c4075e80 2024-05-08T05:54:15.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.542+0000 7fc7cb800700 1 -- 172.21.15.112:0/3059094137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c41a3ca0 con 0x7fc7c4075e80 2024-05-08T05:54:15.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.543+0000 7fc7cb800700 1 -- 172.21.15.112:0/3059094137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c406a190 con 0x7fc7c4075e80 2024-05-08T05:54:15.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.546+0000 7fc7c1600700 1 -- 172.21.15.112:0/3059094137 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc7b8010430 con 0x7fc7c4075e80 2024-05-08T05:54:15.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.547+0000 7fc7c1600700 1 --2- 172.21.15.112:0/3059094137 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc7ac0b2900 0x7fc7ac0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.547+0000 7fc7c1600700 1 -- 172.21.15.112:0/3059094137 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fc7b80e2430 con 0x7fc7c4075e80 2024-05-08T05:54:15.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.547+0000 7fc7c3e00700 1 --2- 172.21.15.112:0/3059094137 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc7ac0b2900 0x7fc7ac0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.548+0000 7fc7c3e00700 1 --2- 172.21.15.112:0/3059094137 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc7ac0b2900 0x7fc7ac0b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000b2e0 tx=0x7fc7b0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:15.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.552+0000 7fc7c1600700 1 -- 172.21.15.112:0/3059094137 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc7b80aa0c0 con 0x7fc7c4075e80 2024-05-08T05:54:15.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:15 smithi162 ceph-mon[34800]: pgmap v409: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:15 smithi150 ceph-mon[34315]: pgmap v409: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:15.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.789+0000 7fc7cb800700 1 -- 172.21.15.112:0/3059094137 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fc7c40638e0 con 0x7fc7ac0b2900 2024-05-08T05:54:15.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.793+0000 7fc7c1600700 1 -- 172.21.15.112:0/3059094137 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fc7c40638e0 con 0x7fc7ac0b2900 2024-05-08T05:54:15.793 INFO:teuthology.orchestra.run.smithi112.stdout:HOST ADDR LABELS STATUS 2024-05-08T05:54:15.793 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 172.21.15.112 2024-05-08T05:54:15.793 INFO:teuthology.orchestra.run.smithi112.stdout:smithi150 172.21.15.150 2024-05-08T05:54:15.793 INFO:teuthology.orchestra.run.smithi112.stdout:smithi162 172.21.15.162 2024-05-08T05:54:15.793 INFO:teuthology.orchestra.run.smithi112.stdout:3 hosts in cluster 2024-05-08T05:54:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.797+0000 7fc7cb800700 1 -- 172.21.15.112:0/3059094137 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc7ac0b2900 msgr2=0x7fc7ac0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.797+0000 7fc7cb800700 1 --2- 172.21.15.112:0/3059094137 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc7ac0b2900 0x7fc7ac0b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000b2e0 tx=0x7fc7b0005fb0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.797+0000 7fc7cb800700 1 -- 172.21.15.112:0/3059094137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c4075e80 msgr2=0x7fc7c419b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.797+0000 7fc7cb800700 1 --2- 172.21.15.112:0/3059094137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c4075e80 0x7fc7c419b860 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800e9d0 tx=0x7fc7b800ece0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.797+0000 7fc7cb800700 1 -- 172.21.15.112:0/3059094137 shutdown_connections 2024-05-08T05:54:15.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.797+0000 7fc7cb800700 1 --2- 172.21.15.112:0/3059094137 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc7ac0b2900 0x7fc7ac0b4dc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.797+0000 7fc7cb800700 1 --2- 172.21.15.112:0/3059094137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7c410b400 0x7fc7c41a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.797+0000 7fc7cb800700 1 --2- 172.21.15.112:0/3059094137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c4075e80 0x7fc7c419b860 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.797+0000 7fc7cb800700 1 --2- 172.21.15.112:0/3059094137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7c4077450 0x7fc7c419bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.797+0000 7fc7cb800700 1 -- 172.21.15.112:0/3059094137 >> 172.21.15.112:0/3059094137 conn(0x7fc7c40feaa0 msgr2=0x7fc7c4107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:15.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.797+0000 7fc7cb800700 1 -- 172.21.15.112:0/3059094137 shutdown_connections 2024-05-08T05:54:15.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:15.797+0000 7fc7cb800700 1 -- 172.21.15.112:0/3059094137 wait complete. 2024-05-08T05:54:16.668 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-08T05:54:17.128 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:17.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:17 smithi112 ceph-mon[30003]: pgmap v410: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:17.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:17 smithi112 ceph-mon[30003]: from='client.24949 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:54:17.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:17 smithi162 ceph-mon[34800]: pgmap v410: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:17.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:17 smithi162 ceph-mon[34800]: from='client.24949 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:54:17.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:17 smithi150 ceph-mon[34315]: pgmap v410: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:17.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:17 smithi150 ceph-mon[34315]: from='client.24949 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:54:19.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.332+0000 7f1620000700 1 -- 172.21.15.112:0/2914473622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f161810b6e0 msgr2=0x7f1618111f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:19.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.332+0000 7f1620000700 1 --2- 172.21.15.112:0/2914473622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f161810b6e0 0x7f1618111f90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f161000b0d0 tx=0x7f161000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:54:19.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.332+0000 7f1620000700 1 -- 172.21.15.112:0/2914473622 shutdown_connections 2024-05-08T05:54:19.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.332+0000 7f1620000700 1 --2- 172.21.15.112:0/2914473622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f161810b6e0 0x7f1618111f90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:19.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.332+0000 7f1620000700 1 --2- 172.21.15.112:0/2914473622 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1618109b40 0x7f1618109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:19.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.332+0000 7f1620000700 1 --2- 172.21.15.112:0/2914473622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f161810ad40 0x7f161810b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:19.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.332+0000 7f1620000700 1 -- 172.21.15.112:0/2914473622 >> 172.21.15.112:0/2914473622 conn(0x7f1618078b10 msgr2=0x7f161807af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:19.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.332+0000 7f1620000700 1 -- 172.21.15.112:0/2914473622 shutdown_connections 2024-05-08T05:54:19.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.332+0000 7f1620000700 1 -- 172.21.15.112:0/2914473622 wait complete. 2024-05-08T05:54:19.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.333+0000 7f1620000700 1 Processor -- start 2024-05-08T05:54:19.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.333+0000 7f1620000700 1 -- start start 2024-05-08T05:54:19.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.333+0000 7f1620000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1618109b40 0x7f161819fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:19.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f1620000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f161810ad40 0x7f16181a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:19.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f1620000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f161810b6e0 0x7f16181a7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:19.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f1620000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16181155c0 con 0x7f161810b6e0 2024-05-08T05:54:19.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f1620000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1618115440 con 0x7f1618109b40 2024-05-08T05:54:19.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f1620000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1618115740 con 0x7f161810ad40 2024-05-08T05:54:19.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f161d600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1618109b40 0x7f161819fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:19.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f161e000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f161810b6e0 0x7f16181a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:19.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f161e000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f161810b6e0 0x7f16181a7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39422/0 (socket says 172.21.15.112:39422) 2024-05-08T05:54:19.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f161d600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1618109b40 0x7f161819fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:59236/0 (socket says 172.21.15.112:59236) 2024-05-08T05:54:19.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f161e000700 1 -- 172.21.15.112:0/3584203557 learned_addr learned my addr 172.21.15.112:0/3584203557 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:19.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f161cc00700 1 --2- 172.21.15.112:0/3584203557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f161810ad40 0x7f16181a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:19.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f161d600700 1 -- 172.21.15.112:0/3584203557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f161810ad40 msgr2=0x7f16181a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f161d600700 1 --2- 172.21.15.112:0/3584203557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f161810ad40 0x7f16181a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f161d600700 1 -- 172.21.15.112:0/3584203557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f161810b6e0 msgr2=0x7f16181a7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f161d600700 1 --2- 172.21.15.112:0/3584203557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f161810b6e0 0x7f16181a7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.334+0000 7f161d600700 1 -- 172.21.15.112:0/3584203557 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16181a7810 con 0x7f1618109b40 2024-05-08T05:54:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.335+0000 7f161d600700 1 --2- 172.21.15.112:0/3584203557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1618109b40 0x7f161819fbb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f160c00e9c0 tx=0x7f160c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.335+0000 7f1616000700 1 -- 172.21.15.112:0/3584203557 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c004220 con 0x7f1618109b40 2024-05-08T05:54:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.335+0000 7f1616000700 1 -- 172.21.15.112:0/3584203557 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f160c0043c0 con 0x7f1618109b40 2024-05-08T05:54:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.335+0000 7f1616000700 1 -- 172.21.15.112:0/3584203557 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c003c30 con 0x7f1618109b40 2024-05-08T05:54:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.335+0000 7f1620000700 1 -- 172.21.15.112:0/3584203557 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16181a7b00 con 0x7f1618109b40 2024-05-08T05:54:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.335+0000 7f1620000700 1 -- 172.21.15.112:0/3584203557 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16181a8090 con 0x7f1618109b40 2024-05-08T05:54:19.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.336+0000 7f1620000700 1 -- 172.21.15.112:0/3584203557 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161806a190 con 0x7f1618109b40 2024-05-08T05:54:19.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.338+0000 7f1616000700 1 -- 172.21.15.112:0/3584203557 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f160c046910 con 0x7f1618109b40 2024-05-08T05:54:19.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.340+0000 7f1616000700 1 --2- 172.21.15.112:0/3584203557 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f16000b2900 0x7f16000b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:19.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.340+0000 7f1616000700 1 -- 172.21.15.112:0/3584203557 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f160c0e23f0 con 0x7f1618109b40 2024-05-08T05:54:19.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.340+0000 7f161cc00700 1 --2- 172.21.15.112:0/3584203557 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f16000b2900 0x7f16000b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:19.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.341+0000 7f161cc00700 1 --2- 172.21.15.112:0/3584203557 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f16000b2900 0x7f16000b4dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1604009680 tx=0x7f1604009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:19.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.342+0000 7f1616000700 1 -- 172.21.15.112:0/3584203557 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f160c0aa080 con 0x7f1618109b40 2024-05-08T05:54:19.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:19 smithi112 ceph-mon[30003]: pgmap v411: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:19.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:19 smithi162 ceph-mon[34800]: pgmap v411: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:19.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.585+0000 7f1620000700 1 -- 172.21.15.112:0/3584203557 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f16180638e0 con 0x7f16000b2900 2024-05-08T05:54:19.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.588+0000 7f1616000700 1 -- 172.21.15.112:0/3584203557 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f16180638e0 con 0x7f16000b2900 2024-05-08T05:54:19.589 INFO:teuthology.orchestra.run.smithi112.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-08T05:54:19.589 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NV400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-08T05:54:19.589 INFO:teuthology.orchestra.run.smithi112.stdout:smithi150 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GX400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-08T05:54:19.589 INFO:teuthology.orchestra.run.smithi112.stdout:smithi162 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JU400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-08T05:54:19.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.591+0000 7f1620000700 1 -- 172.21.15.112:0/3584203557 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f16000b2900 msgr2=0x7f16000b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:19.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.591+0000 7f1620000700 1 --2- 172.21.15.112:0/3584203557 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f16000b2900 0x7f16000b4dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1604009680 tx=0x7f1604009260 comp rx=0 tx=0).stop 2024-05-08T05:54:19.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.592+0000 7f1620000700 1 -- 172.21.15.112:0/3584203557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1618109b40 msgr2=0x7f161819fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:19.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.592+0000 7f1620000700 1 --2- 172.21.15.112:0/3584203557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1618109b40 0x7f161819fbb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f160c00e9c0 tx=0x7f160c00ecd0 comp rx=0 tx=0).stop 2024-05-08T05:54:19.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.592+0000 7f1620000700 1 -- 172.21.15.112:0/3584203557 shutdown_connections 2024-05-08T05:54:19.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.592+0000 7f1620000700 1 --2- 172.21.15.112:0/3584203557 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f16000b2900 0x7f16000b4dc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:19.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.592+0000 7f1620000700 1 --2- 172.21.15.112:0/3584203557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f161810b6e0 0x7f16181a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:19.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.592+0000 7f1620000700 1 --2- 172.21.15.112:0/3584203557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1618109b40 0x7f161819fbb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:19.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.592+0000 7f1620000700 1 --2- 172.21.15.112:0/3584203557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f161810ad40 0x7f16181a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:19.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.592+0000 7f1620000700 1 -- 172.21.15.112:0/3584203557 >> 172.21.15.112:0/3584203557 conn(0x7f1618078b10 msgr2=0x7f161810e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:19.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.592+0000 7f1620000700 1 -- 172.21.15.112:0/3584203557 shutdown_connections 2024-05-08T05:54:19.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:19.592+0000 7f1620000700 1 -- 172.21.15.112:0/3584203557 wait complete. 2024-05-08T05:54:19.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:19 smithi150 ceph-mon[34315]: pgmap v411: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:20.351 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T05:54:20.360 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-05-08T05:54:20.360 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-08T05:54:20.809 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:21.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:21 smithi112 ceph-mon[30003]: pgmap v412: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:21.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:21 smithi112 ceph-mon[30003]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:54:21.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:21 smithi162 ceph-mon[34800]: pgmap v412: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:21.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:21 smithi162 ceph-mon[34800]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:54:21.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:21 smithi150 ceph-mon[34315]: pgmap v412: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:21.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:21 smithi150 ceph-mon[34315]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:54:22.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.991+0000 7f1acee00700 1 -- 172.21.15.112:0/3088079373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac8106940 msgr2=0x7f1ac8106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:22.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.991+0000 7f1acee00700 1 --2- 172.21.15.112:0/3088079373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac8106940 0x7f1ac8106da0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009990 tx=0x7f1ab0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:54:22.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.991+0000 7f1acee00700 1 -- 172.21.15.112:0/3088079373 shutdown_connections 2024-05-08T05:54:22.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.991+0000 7f1acee00700 1 --2- 172.21.15.112:0/3088079373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac81072e0 0x7f1ac810db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:22.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.991+0000 7f1acee00700 1 --2- 172.21.15.112:0/3088079373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac8105740 0x7f1ac8105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:22.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.991+0000 7f1acee00700 1 --2- 172.21.15.112:0/3088079373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac8106940 0x7f1ac8106da0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:22.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.991+0000 7f1acee00700 1 -- 172.21.15.112:0/3088079373 >> 172.21.15.112:0/3088079373 conn(0x7f1ac8100d00 msgr2=0x7f1ac8103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:22.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.992+0000 7f1acee00700 1 -- 172.21.15.112:0/3088079373 shutdown_connections 2024-05-08T05:54:22.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.992+0000 7f1acee00700 1 -- 172.21.15.112:0/3088079373 wait complete. 2024-05-08T05:54:22.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.993+0000 7f1acee00700 1 Processor -- start 2024-05-08T05:54:22.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.994+0000 7f1acee00700 1 -- start start 2024-05-08T05:54:22.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.994+0000 7f1acee00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac8105740 0x7f1ac819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:22.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.994+0000 7f1acee00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac8106940 0x7f1ac819bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:22.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.994+0000 7f1acee00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac81072e0 0x7f1ac81a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:22.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.994+0000 7f1acee00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8111200 con 0x7f1ac8105740 2024-05-08T05:54:22.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.994+0000 7f1acee00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8111080 con 0x7f1ac8106940 2024-05-08T05:54:22.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.994+0000 7f1acee00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8111380 con 0x7f1ac81072e0 2024-05-08T05:54:22.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.995+0000 7f1abfe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac8106940 0x7f1ac819bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:22.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.995+0000 7f1abfe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac8106940 0x7f1ac819bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:59266/0 (socket says 172.21.15.112:59266) 2024-05-08T05:54:22.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.995+0000 7f1acce00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac81072e0 0x7f1ac81a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:22.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.995+0000 7f1abfe00700 1 -- 172.21.15.112:0/1142970959 learned_addr learned my addr 172.21.15.112:0/1142970959 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:22.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.995+0000 7f1acce00700 1 -- 172.21.15.112:0/1142970959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac8106940 msgr2=0x7f1ac819bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:22.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.995+0000 7f1acce00700 1 --2- 172.21.15.112:0/1142970959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac8106940 0x7f1ac819bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:22.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.995+0000 7f1acce00700 1 -- 172.21.15.112:0/1142970959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac8105740 msgr2=0x7f1ac819b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.995+0000 7f1acce00700 1 --2- 172.21.15.112:0/1142970959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac8105740 0x7f1ac819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.995+0000 7f1acce00700 1 -- 172.21.15.112:0/1142970959 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac81a34d0 con 0x7f1ac81072e0 2024-05-08T05:54:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.995+0000 7f1acce00700 1 --2- 172.21.15.112:0/1142970959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac81072e0 0x7f1ac81a2e30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009f60 tx=0x7f1ac0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.996+0000 7f1ac5600700 1 -- 172.21.15.112:0/1142970959 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00092e0 con 0x7f1ac81072e0 2024-05-08T05:54:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.996+0000 7f1ac5600700 1 -- 172.21.15.112:0/1142970959 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac0004bb0 con 0x7f1ac81072e0 2024-05-08T05:54:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.996+0000 7f1ac5600700 1 -- 172.21.15.112:0/1142970959 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0004280 con 0x7f1ac81072e0 2024-05-08T05:54:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.996+0000 7f1acee00700 1 -- 172.21.15.112:0/1142970959 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac81a3760 con 0x7f1ac81072e0 2024-05-08T05:54:22.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.996+0000 7f1acee00700 1 -- 172.21.15.112:0/1142970959 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac81a3c50 con 0x7f1ac81072e0 2024-05-08T05:54:22.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:22.997+0000 7f1acee00700 1 -- 172.21.15.112:0/1142970959 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac806a190 con 0x7f1ac81072e0 2024-05-08T05:54:23.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.000+0000 7f1ac5600700 1 -- 172.21.15.112:0/1142970959 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1ac001b030 con 0x7f1ac81072e0 2024-05-08T05:54:23.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.001+0000 7f1ac5600700 1 --2- 172.21.15.112:0/1142970959 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1aac0b2900 0x7f1aac0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:23.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.001+0000 7f1ac5600700 1 -- 172.21.15.112:0/1142970959 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f1ac00e1cf0 con 0x7f1ac81072e0 2024-05-08T05:54:23.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.002+0000 7f1ac7e00700 1 --2- 172.21.15.112:0/1142970959 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1aac0b2900 0x7f1aac0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:23.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.002+0000 7f1ac7e00700 1 --2- 172.21.15.112:0/1142970959 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1aac0b2900 0x7f1aac0b4dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1ac81067a0 tx=0x7f1ab80096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:23.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.005+0000 7f1ac5600700 1 -- 172.21.15.112:0/1142970959 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1ac0022020 con 0x7f1ac81072e0 2024-05-08T05:54:23.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.285+0000 7f1acee00700 1 -- 172.21.15.112:0/1142970959 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1ac80640b0 con 0x7f1ac81072e0 2024-05-08T05:54:23.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.286+0000 7f1ac5600700 1 -- 172.21.15.112:0/1142970959 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3064 (secure 0 0 0) 0x7f1ac0010040 con 0x7f1ac81072e0 2024-05-08T05:54:23.288 INFO:teuthology.orchestra.run.smithi112.stdout:e24 2024-05-08T05:54:23.288 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T05:54:23.288 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T05:54:23.288 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-05-08T05:54:23.288 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:54:23.288 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-05-08T05:54:23.288 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-05-08T05:54:23.288 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 24 2024-05-08T05:54:23.288 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-08T05:54:23.288 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-05-08T05:51:07.312542+0000 2024-05-08T05:54:23.288 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-05-08T05:51:31.368725+0000 2024-05-08T05:54:23.288 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-05-08T05:54:23.288 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-05-08T05:54:23.289 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 300 2024-05-08T05:54:23.289 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-05-08T05:54:23.289 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-05-08T05:54:23.289 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-05-08T05:54:23.289 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-05-08T05:54:23.289 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-05-08T05:54:23.289 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 0 2024-05-08T05:54:23.289 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T05:54:23.289 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 3 2024-05-08T05:54:23.289 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1,2 2024-05-08T05:54:23.289 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=24529,1=24518,2=24511} 2024-05-08T05:54:23.289 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-05-08T05:54:23.289 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-05-08T05:54:23.289 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-05-08T05:54:23.290 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-05-08T05:54:23.290 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-05-08T05:54:23.290 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data disabled 2024-05-08T05:54:23.290 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-05-08T05:54:23.290 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-05-08T05:54:23.290 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 0 2024-05-08T05:54:23.290 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 24529 members: 24529,24511,24518 2024-05-08T05:54:23.290 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a{0:24529} state up:active seq 8 addr [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:54:23.290 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c{0:24496} state up:standby-replay seq 1 addr [v2:172.21.15.162:6836/3092620994,v1:172.21.15.162:6837/3092620994] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:54:23.290 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b{1:24518} state up:active seq 11 addr [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:54:23.290 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.f{1:24484} state up:standby-replay seq 1 addr [v2:172.21.15.162:6834/3060051414,v1:172.21.15.162:6835/3060051414] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:54:23.290 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.i{2:24511} state up:active seq 14 addr [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:54:23.291 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.h{2:24479} state up:standby-replay seq 1 addr [v2:172.21.15.150:6836/4103977076,v1:172.21.15.150:6837/4103977076] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:54:23.291 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:54:23.291 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:54:23.291 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-05-08T05:54:23.291 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:54:23.291 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.j{-1:14361} state up:standby seq 1 addr [v2:172.21.15.112:6834/3258450798,v1:172.21.15.112:6835/3258450798] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:54:23.291 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d{-1:14367} state up:standby seq 1 addr [v2:172.21.15.112:6836/2849670124,v1:172.21.15.112:6837/2849670124] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:54:23.291 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.g{-1:14391} state up:standby seq 1 addr [v2:172.21.15.112:6838/3014879788,v1:172.21.15.112:6839/3014879788] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:54:23.291 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.k{-1:24434} state up:standby seq 1 addr [v2:172.21.15.150:6832/442303489,v1:172.21.15.150:6833/442303489] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:54:23.291 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.l{-1:24448} state up:standby seq 1 addr [v2:172.21.15.162:6832/3668390959,v1:172.21.15.162:6833/3668390959] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:54:23.291 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.e{-1:24464} state up:standby seq 1 addr [v2:172.21.15.150:6834/1945132252,v1:172.21.15.150:6835/1945132252] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:54:23.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.290+0000 7f1acee00700 1 -- 172.21.15.112:0/1142970959 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1aac0b2900 msgr2=0x7f1aac0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:23.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.290+0000 7f1acee00700 1 --2- 172.21.15.112:0/1142970959 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1aac0b2900 0x7f1aac0b4dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1ac81067a0 tx=0x7f1ab80096c0 comp rx=0 tx=0).stop 2024-05-08T05:54:23.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.290+0000 7f1acee00700 1 -- 172.21.15.112:0/1142970959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac81072e0 msgr2=0x7f1ac81a2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:23.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.290+0000 7f1acee00700 1 --2- 172.21.15.112:0/1142970959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac81072e0 0x7f1ac81a2e30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009f60 tx=0x7f1ac0009f90 comp rx=0 tx=0).stop 2024-05-08T05:54:23.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.290+0000 7f1acee00700 1 -- 172.21.15.112:0/1142970959 shutdown_connections 2024-05-08T05:54:23.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.290+0000 7f1acee00700 1 --2- 172.21.15.112:0/1142970959 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1aac0b2900 0x7f1aac0b4dc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:23.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.290+0000 7f1acee00700 1 --2- 172.21.15.112:0/1142970959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac81072e0 0x7f1ac81a2e30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:23.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.290+0000 7f1acee00700 1 --2- 172.21.15.112:0/1142970959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ac8105740 0x7f1ac819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:23.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.290+0000 7f1acee00700 1 --2- 172.21.15.112:0/1142970959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac8106940 0x7f1ac819bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:23.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.290+0000 7f1acee00700 1 -- 172.21.15.112:0/1142970959 >> 172.21.15.112:0/1142970959 conn(0x7f1ac8100d00 msgr2=0x7f1ac8109cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:23.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.290+0000 7f1acee00700 1 -- 172.21.15.112:0/1142970959 shutdown_connections 2024-05-08T05:54:23.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:23.290+0000 7f1acee00700 1 -- 172.21.15.112:0/1142970959 wait complete. 2024-05-08T05:54:23.293 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 24 2024-05-08T05:54:23.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:23 smithi112 ceph-mon[30003]: pgmap v413: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:23.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:23 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1142970959' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T05:54:23.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:23 smithi162 ceph-mon[34800]: pgmap v413: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:23.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:23 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1142970959' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T05:54:23.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:23 smithi150 ceph-mon[34315]: pgmap v413: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:23.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:23 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1142970959' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T05:54:24.099 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-08T05:54:24.550 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:24.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:54:24.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:54:24.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:24 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:54:24.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:54:24.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:54:24.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:24 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:54:24.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:54:24.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:54:24.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:24 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:54:25.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:25 smithi112 ceph-mon[30003]: pgmap v414: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:25.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:25 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:54:25.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:25 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:54:25.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:25 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:54:25.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:25 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:54:25.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:25 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:54:25.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:25 smithi162 ceph-mon[34800]: pgmap v414: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:25.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:25 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:54:25.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:25 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:54:25.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:25 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:54:25.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:25 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:54:25.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:25 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:54:25.584 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:25 smithi150 ceph-mon[34315]: pgmap v414: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:25.584 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:25 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:54:25.584 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:25 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:54:25.584 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:25 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:54:25.584 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:25 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:54:25.584 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:25 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:54:26.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.616+0000 7f4614e00700 1 -- 172.21.15.112:0/1831856441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4610105cf0 msgr2=0x7f461010db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:26.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.616+0000 7f4614e00700 1 --2- 172.21.15.112:0/1831856441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4610105cf0 0x7f461010db80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4608009fc0 tx=0x7f460800b330 comp rx=0 tx=0).stop 2024-05-08T05:54:26.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.616+0000 7f4614e00700 1 -- 172.21.15.112:0/1831856441 shutdown_connections 2024-05-08T05:54:26.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.616+0000 7f4614e00700 1 --2- 172.21.15.112:0/1831856441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4610105cf0 0x7f461010db80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:26.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.616+0000 7f4614e00700 1 --2- 172.21.15.112:0/1831856441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4610106e10 0x7f4610107230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:26.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.616+0000 7f4614e00700 1 --2- 172.21.15.112:0/1831856441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610105350 0x7f46101057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:26.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.616+0000 7f4614e00700 1 -- 172.21.15.112:0/1831856441 >> 172.21.15.112:0/1831856441 conn(0x7f4610100cf0 msgr2=0x7f4610103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:26.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.616+0000 7f4614e00700 1 -- 172.21.15.112:0/1831856441 shutdown_connections 2024-05-08T05:54:26.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.617+0000 7f4614e00700 1 -- 172.21.15.112:0/1831856441 wait complete. 2024-05-08T05:54:26.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.618+0000 7f4614e00700 1 Processor -- start 2024-05-08T05:54:26.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.618+0000 7f4614e00700 1 -- start start 2024-05-08T05:54:26.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.618+0000 7f4614e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4610105350 0x7f461019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:26.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.618+0000 7f4614e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4610105cf0 0x7f461019be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:26.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.618+0000 7f4614e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610106e10 0x7f46101a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:26.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.618+0000 7f4614e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4610111320 con 0x7f4610106e10 2024-05-08T05:54:26.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.618+0000 7f4614e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f46101111a0 con 0x7f4610105cf0 2024-05-08T05:54:26.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.618+0000 7f4614e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f46101114a0 con 0x7f4610105350 2024-05-08T05:54:26.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.619+0000 7f4607e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4610105cf0 0x7f461019be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:26.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.619+0000 7f4607e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4610105cf0 0x7f461019be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:57202/0 (socket says 172.21.15.112:57202) 2024-05-08T05:54:26.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.619+0000 7f4607e00700 1 -- 172.21.15.112:0/3733136979 learned_addr learned my addr 172.21.15.112:0/3733136979 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:26.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.619+0000 7f460de00700 1 --2- 172.21.15.112:0/3733136979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4610105350 0x7f461019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:26.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.619+0000 7f460e800700 1 --2- 172.21.15.112:0/3733136979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610106e10 0x7f46101a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:26.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.619+0000 7f4607e00700 1 -- 172.21.15.112:0/3733136979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4610105350 msgr2=0x7f461019b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:26.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.619+0000 7f4607e00700 1 --2- 172.21.15.112:0/3733136979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4610105350 0x7f461019b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:26.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.619+0000 7f4607e00700 1 -- 172.21.15.112:0/3733136979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610106e10 msgr2=0x7f46101a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:26.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.619+0000 7f4607e00700 1 --2- 172.21.15.112:0/3733136979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610106e10 0x7f46101a2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:26.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.619+0000 7f4607e00700 1 -- 172.21.15.112:0/3733136979 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46101a3560 con 0x7f4610105cf0 2024-05-08T05:54:26.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.619+0000 7f460e800700 1 --2- 172.21.15.112:0/3733136979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610106e10 0x7f46101a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:26.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.620+0000 7f4607e00700 1 --2- 172.21.15.112:0/3733136979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4610105cf0 0x7f461019be40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f45f800ec10 tx=0x7f45f800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:26.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.620+0000 7f4606a00700 1 -- 172.21.15.112:0/3733136979 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f800cc90 con 0x7f4610105cf0 2024-05-08T05:54:26.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.620+0000 7f4606a00700 1 -- 172.21.15.112:0/3733136979 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f800ce30 con 0x7f4610105cf0 2024-05-08T05:54:26.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.620+0000 7f4606a00700 1 -- 172.21.15.112:0/3733136979 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8010430 con 0x7f4610105cf0 2024-05-08T05:54:26.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.620+0000 7f4614e00700 1 -- 172.21.15.112:0/3733136979 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46101a3850 con 0x7f4610105cf0 2024-05-08T05:54:26.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.620+0000 7f4614e00700 1 -- 172.21.15.112:0/3733136979 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46101a3de0 con 0x7f4610105cf0 2024-05-08T05:54:26.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.621+0000 7f4614e00700 1 -- 172.21.15.112:0/3733136979 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f461006a190 con 0x7f4610105cf0 2024-05-08T05:54:26.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.624+0000 7f4606a00700 1 -- 172.21.15.112:0/3733136979 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f45f8003fe0 con 0x7f4610105cf0 2024-05-08T05:54:26.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.625+0000 7f4606a00700 1 --2- 172.21.15.112:0/3733136979 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f45f40b2900 0x7f45f40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:26.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.626+0000 7f4606a00700 1 -- 172.21.15.112:0/3733136979 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f45f80e24f0 con 0x7f4610105cf0 2024-05-08T05:54:26.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.626+0000 7f460de00700 1 --2- 172.21.15.112:0/3733136979 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f45f40b2900 0x7f45f40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:26.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.626+0000 7f460de00700 1 --2- 172.21.15.112:0/3733136979 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f45f40b2900 0x7f45f40b4dc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4610106790 tx=0x7f4600009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:26.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.630+0000 7f4606a00700 1 -- 172.21.15.112:0/3733136979 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45f80aa180 con 0x7f4610105cf0 2024-05-08T05:54:26.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.864+0000 7f4614e00700 1 -- 172.21.15.112:0/3733136979 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f46100640b0 con 0x7f4610105cf0 2024-05-08T05:54:26.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.865+0000 7f4606a00700 1 -- 172.21.15.112:0/3733136979 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v84) ==== 56+0+5364 (secure 0 0 0) 0x7f45f80b2020 con 0x7f4610105cf0 2024-05-08T05:54:26.868 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 84 2024-05-08T05:54:26.868 INFO:teuthology.orchestra.run.smithi112.stdout:fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:54:26.868 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-05-08T05:41:48.498871+0000 2024-05-08T05:54:26.868 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-05-08T05:53:05.251932+0000 2024-05-08T05:54:26.868 INFO:teuthology.orchestra.run.smithi112.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-08T05:54:26.868 INFO:teuthology.orchestra.run.smithi112.stdout:crush_version 26 2024-05-08T05:54:26.868 INFO:teuthology.orchestra.run.smithi112.stdout:full_ratio 0.95 2024-05-08T05:54:26.868 INFO:teuthology.orchestra.run.smithi112.stdout:backfillfull_ratio 0.9 2024-05-08T05:54:26.868 INFO:teuthology.orchestra.run.smithi112.stdout:nearfull_ratio 0.85 2024-05-08T05:54:26.868 INFO:teuthology.orchestra.run.smithi112.stdout:require_min_compat_client luminous 2024-05-08T05:54:26.868 INFO:teuthology.orchestra.run.smithi112.stdout:min_compat_client luminous 2024-05-08T05:54:26.868 INFO:teuthology.orchestra.run.smithi112.stdout:require_osd_release squid 2024-05-08T05:54:26.868 INFO:teuthology.orchestra.run.smithi112.stdout:stretch_mode_enabled false 2024-05-08T05:54:26.869 INFO:teuthology.orchestra.run.smithi112.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-08T05:54:26.869 INFO:teuthology.orchestra.run.smithi112.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-08T05:54:26.869 INFO:teuthology.orchestra.run.smithi112.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-08T05:54:26.869 INFO:teuthology.orchestra.run.smithi112.stdout:max_osd 12 2024-05-08T05:54:26.869 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6802/3419091820,v1:172.21.15.112:6803/3419091820] [v2:172.21.15.112:6804/3419091820,v1:172.21.15.112:6805/3419091820] exists,up 0b378542-1bbe-4614-8805-8a6b875ada55 2024-05-08T05:54:26.869 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/2728419797,v1:172.21.15.150:6801/2728419797] [v2:172.21.15.150:6802/2728419797,v1:172.21.15.150:6803/2728419797] exists,up 826ef036-2db7-470e-9956-d9a58b9181e9 2024-05-08T05:54:26.869 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6800/1556203215,v1:172.21.15.162:6801/1556203215] [v2:172.21.15.162:6802/1556203215,v1:172.21.15.162:6803/1556203215] exists,up 32e710e7-2a87-4863-bc7a-906d31aa4ef7 2024-05-08T05:54:26.869 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 up in weight 1 up_from 25 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6810/3325718755,v1:172.21.15.112:6811/3325718755] [v2:172.21.15.112:6812/3325718755,v1:172.21.15.112:6813/3325718755] exists,up ba1d4a5f-6a95-4b9e-8a48-8524a899b004 2024-05-08T05:54:26.869 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 up in weight 1 up_from 31 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/1548914683,v1:172.21.15.150:6809/1548914683] [v2:172.21.15.150:6810/1548914683,v1:172.21.15.150:6811/1548914683] exists,up cb4f1a42-8de8-4b50-b3bc-602de9f4803a 2024-05-08T05:54:26.869 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 up in weight 1 up_from 37 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6808/409836750,v1:172.21.15.162:6809/409836750] [v2:172.21.15.162:6810/409836750,v1:172.21.15.162:6811/409836750] exists,up a26c03b8-050c-4a2c-8f23-5801fffb7c78 2024-05-08T05:54:26.869 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 up in weight 1 up_from 43 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6818/1098984186,v1:172.21.15.112:6819/1098984186] [v2:172.21.15.112:6820/1098984186,v1:172.21.15.112:6821/1098984186] exists,up 8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0 2024-05-08T05:54:26.869 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 up in weight 1 up_from 48 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/4009836564,v1:172.21.15.150:6817/4009836564] [v2:172.21.15.150:6818/4009836564,v1:172.21.15.150:6819/4009836564] exists,up af3629b4-9bed-4669-97a6-48deba503a96 2024-05-08T05:54:26.869 INFO:teuthology.orchestra.run.smithi112.stdout:osd.8 up in weight 1 up_from 55 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6816/852791606,v1:172.21.15.162:6817/852791606] [v2:172.21.15.162:6818/852791606,v1:172.21.15.162:6819/852791606] exists,up 9333e87d-dd78-4467-bbde-c2b44f4ea07f 2024-05-08T05:54:26.869 INFO:teuthology.orchestra.run.smithi112.stdout:osd.9 up in weight 1 up_from 61 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6826/2186764674,v1:172.21.15.112:6827/2186764674] [v2:172.21.15.112:6828/2186764674,v1:172.21.15.112:6829/2186764674] exists,up 09723581-1995-42e0-8db7-776ea98a918c 2024-05-08T05:54:26.870 INFO:teuthology.orchestra.run.smithi112.stdout:osd.10 up in weight 1 up_from 66 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/2838995474,v1:172.21.15.150:6825/2838995474] [v2:172.21.15.150:6826/2838995474,v1:172.21.15.150:6827/2838995474] exists,up 34e08e3e-1231-4899-a5b3-76d449105bf4 2024-05-08T05:54:26.870 INFO:teuthology.orchestra.run.smithi112.stdout:osd.11 up in weight 1 up_from 72 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6824/3702614331,v1:172.21.15.162:6825/3702614331] [v2:172.21.15.162:6826/3702614331,v1:172.21.15.162:6827/3702614331] exists,up 6fc34ec0-78ca-403c-acb7-b29d83ff64e8 2024-05-08T05:54:26.870 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.0 [8,4,7,11] 2024-05-08T05:54:26.870 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.b [2,4] 2024-05-08T05:54:26.870 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.f [8,11] 2024-05-08T05:54:26.870 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.10 [2,1] 2024-05-08T05:54:26.870 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.0 [5,9] 2024-05-08T05:54:26.870 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.2 [3,9] 2024-05-08T05:54:26.870 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.e [4,8] 2024-05-08T05:54:26.870 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.14 [4,6] 2024-05-08T05:54:26.870 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.22 [3,9] 2024-05-08T05:54:26.870 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.23 [4,7] 2024-05-08T05:54:26.870 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/3667148497 expires 2024-05-09T05:43:03.313546+0000 2024-05-08T05:54:26.871 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6800/164150456 expires 2024-05-09T05:43:03.313546+0000 2024-05-08T05:54:26.871 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/3425327158 expires 2024-05-09T05:42:29.819307+0000 2024-05-08T05:54:26.871 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/1404611205 expires 2024-05-09T05:43:03.313546+0000 2024-05-08T05:54:26.871 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/3765334043 expires 2024-05-09T05:42:29.819307+0000 2024-05-08T05:54:26.871 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6800/772108158 expires 2024-05-09T05:42:29.819307+0000 2024-05-08T05:54:26.871 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/2814303790 expires 2024-05-09T05:43:03.313546+0000 2024-05-08T05:54:26.871 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6801/164150456 expires 2024-05-09T05:43:03.313546+0000 2024-05-08T05:54:26.871 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6801/772108158 expires 2024-05-09T05:42:29.819307+0000 2024-05-08T05:54:26.871 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/1091751583 expires 2024-05-09T05:42:29.819307+0000 2024-05-08T05:54:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.869+0000 7f4614e00700 1 -- 172.21.15.112:0/3733136979 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f45f40b2900 msgr2=0x7f45f40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.869+0000 7f4614e00700 1 --2- 172.21.15.112:0/3733136979 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f45f40b2900 0x7f45f40b4dc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4610106790 tx=0x7f4600009260 comp rx=0 tx=0).stop 2024-05-08T05:54:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.869+0000 7f4614e00700 1 -- 172.21.15.112:0/3733136979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4610105cf0 msgr2=0x7f461019be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.869+0000 7f4614e00700 1 --2- 172.21.15.112:0/3733136979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4610105cf0 0x7f461019be40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f45f800ec10 tx=0x7f45f800ef20 comp rx=0 tx=0).stop 2024-05-08T05:54:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.869+0000 7f4614e00700 1 -- 172.21.15.112:0/3733136979 shutdown_connections 2024-05-08T05:54:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.869+0000 7f4614e00700 1 --2- 172.21.15.112:0/3733136979 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f45f40b2900 0x7f45f40b4dc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.869+0000 7f4614e00700 1 --2- 172.21.15.112:0/3733136979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610106e10 0x7f46101a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.869+0000 7f4614e00700 1 --2- 172.21.15.112:0/3733136979 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4610105350 0x7f461019b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.869+0000 7f4614e00700 1 --2- 172.21.15.112:0/3733136979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4610105cf0 0x7f461019be40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:26.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.869+0000 7f4614e00700 1 -- 172.21.15.112:0/3733136979 >> 172.21.15.112:0/3733136979 conn(0x7f4610100cf0 msgr2=0x7f4610109c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.869+0000 7f4614e00700 1 -- 172.21.15.112:0/3733136979 shutdown_connections 2024-05-08T05:54:26.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:26.869+0000 7f4614e00700 1 -- 172.21.15.112:0/3733136979 wait complete. 2024-05-08T05:54:27.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:27 smithi112 ceph-mon[30003]: pgmap v415: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:27.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:27 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3733136979' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T05:54:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:27 smithi162 ceph-mon[34800]: pgmap v415: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:27 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3733136979' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T05:54:27.653 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-08T05:54:27.664 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:54:27.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:27 smithi150 ceph-mon[34315]: pgmap v415: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:27.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:27 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3733136979' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T05:54:28.107 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:29.484 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:29 smithi112 ceph-mon[30003]: pgmap v416: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:29.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:29 smithi150 ceph-mon[34315]: pgmap v416: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:29.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:29 smithi162 ceph-mon[34800]: pgmap v416: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:30.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.322+0000 7f5113200700 1 -- 172.21.15.112:0/2763102080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f510c101e60 msgr2=0x7f510c1022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:30.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.322+0000 7f5113200700 1 --2- 172.21.15.112:0/2763102080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f510c101e60 0x7f510c1022e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009990 tx=0x7f50f8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:54:30.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.322+0000 7f5113200700 1 -- 172.21.15.112:0/2763102080 shutdown_connections 2024-05-08T05:54:30.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.322+0000 7f5113200700 1 --2- 172.21.15.112:0/2763102080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c102820 0x7f510c110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:30.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.322+0000 7f5113200700 1 --2- 172.21.15.112:0/2763102080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f510c108580 0x7f510c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:30.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.322+0000 7f5113200700 1 --2- 172.21.15.112:0/2763102080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f510c101e60 0x7f510c1022e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:30.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.322+0000 7f5113200700 1 -- 172.21.15.112:0/2763102080 >> 172.21.15.112:0/2763102080 conn(0x7f510c0776c0 msgr2=0x7f510c077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:30.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.322+0000 7f5113200700 1 -- 172.21.15.112:0/2763102080 shutdown_connections 2024-05-08T05:54:30.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.322+0000 7f5113200700 1 -- 172.21.15.112:0/2763102080 wait complete. 2024-05-08T05:54:30.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.323+0000 7f5113200700 1 Processor -- start 2024-05-08T05:54:30.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.324+0000 7f5113200700 1 -- start start 2024-05-08T05:54:30.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.324+0000 7f5113200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f510c101e60 0x7f510c103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:30.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.324+0000 7f5113200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c102820 0x7f510c103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:30.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.324+0000 7f5113200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f510c108580 0x7f510c103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:30.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.324+0000 7f5113200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f510c1136e0 con 0x7f510c102820 2024-05-08T05:54:30.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.324+0000 7f5113200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f510c113560 con 0x7f510c108580 2024-05-08T05:54:30.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.324+0000 7f5113200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f510c113860 con 0x7f510c101e60 2024-05-08T05:54:30.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.324+0000 7f510b400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c102820 0x7f510c103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:30.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.325+0000 7f510be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f510c101e60 0x7f510c103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:30.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.325+0000 7f510be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f510c101e60 0x7f510c103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:48046/0 (socket says 172.21.15.112:48046) 2024-05-08T05:54:30.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.325+0000 7f510be00700 1 -- 172.21.15.112:0/2323763384 learned_addr learned my addr 172.21.15.112:0/2323763384 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:30.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.325+0000 7f5111200700 1 --2- 172.21.15.112:0/2323763384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f510c108580 0x7f510c103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:30.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.325+0000 7f510be00700 1 -- 172.21.15.112:0/2323763384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f510c108580 msgr2=0x7f510c103ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:30.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.325+0000 7f510be00700 1 --2- 172.21.15.112:0/2323763384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f510c108580 0x7f510c103ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:30.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.325+0000 7f510be00700 1 -- 172.21.15.112:0/2323763384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c102820 msgr2=0x7f510c103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:30.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.325+0000 7f510be00700 1 --2- 172.21.15.112:0/2323763384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c102820 0x7f510c103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:30.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.325+0000 7f510be00700 1 -- 172.21.15.112:0/2323763384 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c1a8140 con 0x7f510c101e60 2024-05-08T05:54:30.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.325+0000 7f510b400700 1 --2- 172.21.15.112:0/2323763384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c102820 0x7f510c103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:30.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.325+0000 7f510be00700 1 --2- 172.21.15.112:0/2323763384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f510c101e60 0x7f510c103040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f510000e950 tx=0x7f510000ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:30.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.326+0000 7f5108c00700 1 -- 172.21.15.112:0/2323763384 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100004230 con 0x7f510c101e60 2024-05-08T05:54:30.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.326+0000 7f5108c00700 1 -- 172.21.15.112:0/2323763384 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51000043d0 con 0x7f510c101e60 2024-05-08T05:54:30.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.326+0000 7f5108c00700 1 -- 172.21.15.112:0/2323763384 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100003d60 con 0x7f510c101e60 2024-05-08T05:54:30.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.326+0000 7f5113200700 1 -- 172.21.15.112:0/2323763384 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510c1a8430 con 0x7f510c101e60 2024-05-08T05:54:30.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.326+0000 7f5113200700 1 -- 172.21.15.112:0/2323763384 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510c1a8920 con 0x7f510c101e60 2024-05-08T05:54:30.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.327+0000 7f5113200700 1 -- 172.21.15.112:0/2323763384 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510c06a190 con 0x7f510c101e60 2024-05-08T05:54:30.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.329+0000 7f5108c00700 1 -- 172.21.15.112:0/2323763384 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f510000c7e0 con 0x7f510c101e60 2024-05-08T05:54:30.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.331+0000 7f5108c00700 1 --2- 172.21.15.112:0/2323763384 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f50f40b2900 0x7f50f40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:30.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.331+0000 7f5108c00700 1 -- 172.21.15.112:0/2323763384 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f51000e2400 con 0x7f510c101e60 2024-05-08T05:54:30.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.331+0000 7f510b400700 1 --2- 172.21.15.112:0/2323763384 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f50f40b2900 0x7f50f40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:30.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.332+0000 7f510b400700 1 --2- 172.21.15.112:0/2323763384 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f50f40b2900 0x7f50f40b4dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f510c077d70 tx=0x7f50f800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:30.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.335+0000 7f5108c00700 1 -- 172.21.15.112:0/2323763384 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f51000aa090 con 0x7f510c101e60 2024-05-08T05:54:30.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.619+0000 7f5113200700 1 -- 172.21.15.112:0/2323763384 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f510c02e510 con 0x7f510c101e60 2024-05-08T05:54:30.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.620+0000 7f5108c00700 1 -- 172.21.15.112:0/2323763384 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11891 (secure 0 0 0) 0x7f51000a9990 con 0x7f510c101e60 2024-05-08T05:54:30.621 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:54:30.622 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T05:51:31.368725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24511,24518]},"id":1}]} 2024-05-08T05:54:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.624+0000 7f5113200700 1 -- 172.21.15.112:0/2323763384 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f50f40b2900 msgr2=0x7f50f40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.624+0000 7f5113200700 1 --2- 172.21.15.112:0/2323763384 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f50f40b2900 0x7f50f40b4dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f510c077d70 tx=0x7f50f800b260 comp rx=0 tx=0).stop 2024-05-08T05:54:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.624+0000 7f5113200700 1 -- 172.21.15.112:0/2323763384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f510c101e60 msgr2=0x7f510c103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.624+0000 7f5113200700 1 --2- 172.21.15.112:0/2323763384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f510c101e60 0x7f510c103040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f510000e950 tx=0x7f510000ec60 comp rx=0 tx=0).stop 2024-05-08T05:54:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.624+0000 7f5113200700 1 -- 172.21.15.112:0/2323763384 shutdown_connections 2024-05-08T05:54:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.624+0000 7f5113200700 1 --2- 172.21.15.112:0/2323763384 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f50f40b2900 0x7f50f40b4dc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.624+0000 7f5113200700 1 --2- 172.21.15.112:0/2323763384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f510c108580 0x7f510c103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.624+0000 7f5113200700 1 --2- 172.21.15.112:0/2323763384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f510c101e60 0x7f510c103040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.624+0000 7f5113200700 1 --2- 172.21.15.112:0/2323763384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f510c102820 0x7f510c103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.624+0000 7f5113200700 1 -- 172.21.15.112:0/2323763384 >> 172.21.15.112:0/2323763384 conn(0x7f510c0776c0 msgr2=0x7f510c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.624+0000 7f5113200700 1 -- 172.21.15.112:0/2323763384 shutdown_connections 2024-05-08T05:54:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:30.624+0000 7f5113200700 1 -- 172.21.15.112:0/2323763384 wait complete. 2024-05-08T05:54:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 24 2024-05-08T05:54:31.463 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:54:31.636 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:31 smithi112 ceph-mon[30003]: pgmap v417: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:31.636 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:31 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2323763384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:31.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:31 smithi150 ceph-mon[34315]: pgmap v417: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:31.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:31 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/2323763384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:31.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:31 smithi162 ceph-mon[34800]: pgmap v417: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:31.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:31 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/2323763384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:33.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:33 smithi112 ceph-mon[30003]: pgmap v418: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:33.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:33 smithi150 ceph-mon[34315]: pgmap v418: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:33.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:33 smithi162 ceph-mon[34800]: pgmap v418: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:34.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.104+0000 7ff1a6600700 1 -- 172.21.15.112:0/1836530562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1a010a490 msgr2=0x7ff1a01169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:34.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.104+0000 7ff1a6600700 1 --2- 172.21.15.112:0/1836530562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1a010a490 0x7ff1a01169a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff19800b0a0 tx=0x7ff19800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:54:34.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.104+0000 7ff1a6600700 1 -- 172.21.15.112:0/1836530562 shutdown_connections 2024-05-08T05:54:34.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.104+0000 7ff1a6600700 1 --2- 172.21.15.112:0/1836530562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1a010a490 0x7ff1a01169a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:34.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.104+0000 7ff1a6600700 1 --2- 172.21.15.112:0/1836530562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1a010fb60 0x7ff1a010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:34.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.104+0000 7ff1a6600700 1 --2- 172.21.15.112:0/1836530562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1a0109ad0 0x7ff1a0109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:34.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.104+0000 7ff1a6600700 1 -- 172.21.15.112:0/1836530562 >> 172.21.15.112:0/1836530562 conn(0x7ff1a0076c70 msgr2=0x7ff1a01082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:34.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.104+0000 7ff1a6600700 1 -- 172.21.15.112:0/1836530562 shutdown_connections 2024-05-08T05:54:34.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.104+0000 7ff1a6600700 1 -- 172.21.15.112:0/1836530562 wait complete. 2024-05-08T05:54:34.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.105+0000 7ff1a6600700 1 Processor -- start 2024-05-08T05:54:34.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.105+0000 7ff1a6600700 1 -- start start 2024-05-08T05:54:34.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.106+0000 7ff1a6600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1a0109ad0 0x7ff1a01a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:34.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.106+0000 7ff1a6600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1a010a490 0x7ff1a01a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:34.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.106+0000 7ff1a6600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1a010fb60 0x7ff1a01a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:34.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.106+0000 7ff1a6600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0119cd0 con 0x7ff1a010fb60 2024-05-08T05:54:34.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.106+0000 7ff1a6600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0119b50 con 0x7ff1a010a490 2024-05-08T05:54:34.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.106+0000 7ff1a6600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0119e50 con 0x7ff1a0109ad0 2024-05-08T05:54:34.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.106+0000 7ff19fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1a010fb60 0x7ff1a01a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:34.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.106+0000 7ff19f400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1a0109ad0 0x7ff1a01a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:34.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.106+0000 7ff19f400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1a0109ad0 0x7ff1a01a0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:54310/0 (socket says 172.21.15.112:54310) 2024-05-08T05:54:34.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.106+0000 7ff19f400700 1 -- 172.21.15.112:0/3097876034 learned_addr learned my addr 172.21.15.112:0/3097876034 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:34.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.106+0000 7ff19ea00700 1 --2- 172.21.15.112:0/3097876034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1a010a490 0x7ff1a01a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:34.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.107+0000 7ff19fe00700 1 -- 172.21.15.112:0/3097876034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1a0109ad0 msgr2=0x7ff1a01a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:34.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.107+0000 7ff19fe00700 1 --2- 172.21.15.112:0/3097876034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1a0109ad0 0x7ff1a01a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:34.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.107+0000 7ff19fe00700 1 -- 172.21.15.112:0/3097876034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1a010a490 msgr2=0x7ff1a01a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:34.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.107+0000 7ff19fe00700 1 --2- 172.21.15.112:0/3097876034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1a010a490 0x7ff1a01a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:34.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.107+0000 7ff19fe00700 1 -- 172.21.15.112:0/3097876034 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a01a5100 con 0x7ff1a010fb60 2024-05-08T05:54:34.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.107+0000 7ff19ea00700 1 --2- 172.21.15.112:0/3097876034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1a010a490 0x7ff1a01a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:34.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.107+0000 7ff19fe00700 1 --2- 172.21.15.112:0/3097876034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1a010fb60 0x7ff1a01a49c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff19800b070 tx=0x7ff198004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.107+0000 7ff197e00700 1 -- 172.21.15.112:0/3097876034 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff198021070 con 0x7ff1a010fb60 2024-05-08T05:54:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.107+0000 7ff197e00700 1 -- 172.21.15.112:0/3097876034 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff198007e30 con 0x7ff1a010fb60 2024-05-08T05:54:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.107+0000 7ff1a6600700 1 -- 172.21.15.112:0/3097876034 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a01a5390 con 0x7ff1a010fb60 2024-05-08T05:54:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.107+0000 7ff197e00700 1 -- 172.21.15.112:0/3097876034 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1980045d0 con 0x7ff1a010fb60 2024-05-08T05:54:34.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.107+0000 7ff1a6600700 1 -- 172.21.15.112:0/3097876034 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a01acc70 con 0x7ff1a010fb60 2024-05-08T05:54:34.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.108+0000 7ff1a6600700 1 -- 172.21.15.112:0/3097876034 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a006a190 con 0x7ff1a010fb60 2024-05-08T05:54:34.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.111+0000 7ff197e00700 1 -- 172.21.15.112:0/3097876034 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff198003c00 con 0x7ff1a010fb60 2024-05-08T05:54:34.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.113+0000 7ff197e00700 1 --2- 172.21.15.112:0/3097876034 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff1840b2900 0x7ff1840b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:34.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.114+0000 7ff19f400700 1 --2- 172.21.15.112:0/3097876034 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff1840b2900 0x7ff1840b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:34.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.114+0000 7ff197e00700 1 -- 172.21.15.112:0/3097876034 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7ff19800f020 con 0x7ff1a010fb60 2024-05-08T05:54:34.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.114+0000 7ff19f400700 1 --2- 172.21.15.112:0/3097876034 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff1840b2900 0x7ff1840b4dc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff190009ca0 tx=0x7ff190009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:34.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.116+0000 7ff197e00700 1 -- 172.21.15.112:0/3097876034 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff19802a020 con 0x7ff1a010fb60 2024-05-08T05:54:34.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.400+0000 7ff1a6600700 1 -- 172.21.15.112:0/3097876034 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1a01a1450 con 0x7ff1a010fb60 2024-05-08T05:54:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.402+0000 7ff197e00700 1 -- 172.21.15.112:0/3097876034 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11891 (secure 0 0 0) 0x7ff1980ab5e0 con 0x7ff1a010fb60 2024-05-08T05:54:34.403 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:54:34.403 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T05:51:31.368725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24511,24518]},"id":1}]} 2024-05-08T05:54:34.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.405+0000 7ff1a6600700 1 -- 172.21.15.112:0/3097876034 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff1840b2900 msgr2=0x7ff1840b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:34.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.405+0000 7ff1a6600700 1 --2- 172.21.15.112:0/3097876034 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff1840b2900 0x7ff1840b4dc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff190009ca0 tx=0x7ff190009260 comp rx=0 tx=0).stop 2024-05-08T05:54:34.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.405+0000 7ff1a6600700 1 -- 172.21.15.112:0/3097876034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1a010fb60 msgr2=0x7ff1a01a49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:34.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.405+0000 7ff1a6600700 1 --2- 172.21.15.112:0/3097876034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1a010fb60 0x7ff1a01a49c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff19800b070 tx=0x7ff198004d00 comp rx=0 tx=0).stop 2024-05-08T05:54:34.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.405+0000 7ff1a6600700 1 -- 172.21.15.112:0/3097876034 shutdown_connections 2024-05-08T05:54:34.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.405+0000 7ff1a6600700 1 --2- 172.21.15.112:0/3097876034 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff1840b2900 0x7ff1840b4dc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:34.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.405+0000 7ff1a6600700 1 --2- 172.21.15.112:0/3097876034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1a010fb60 0x7ff1a01a49c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.405+0000 7ff1a6600700 1 --2- 172.21.15.112:0/3097876034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1a0109ad0 0x7ff1a01a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.406+0000 7ff1a6600700 1 --2- 172.21.15.112:0/3097876034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1a010a490 0x7ff1a01a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.406+0000 7ff1a6600700 1 -- 172.21.15.112:0/3097876034 >> 172.21.15.112:0/3097876034 conn(0x7ff1a0076c70 msgr2=0x7ff1a0077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.406+0000 7ff1a6600700 1 -- 172.21.15.112:0/3097876034 shutdown_connections 2024-05-08T05:54:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:34.406+0000 7ff1a6600700 1 -- 172.21.15.112:0/3097876034 wait complete. 2024-05-08T05:54:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 24 2024-05-08T05:54:34.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:34 smithi150 ceph-mon[34315]: pgmap v419: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:34.789 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:34 smithi112 ceph-mon[30003]: pgmap v419: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:34.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:34 smithi162 ceph-mon[34800]: pgmap v419: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:35.230 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:54:35.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:35 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3097876034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:35.668 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:35.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:35 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3097876034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:35.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:35 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3097876034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:36.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:36 smithi112 ceph-mon[30003]: pgmap v420: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:36.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:36 smithi150 ceph-mon[34315]: pgmap v420: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:36.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:36 smithi162 ceph-mon[34800]: pgmap v420: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:37.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.953+0000 7f96fd000700 1 -- 172.21.15.112:0/608908738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f810a490 msgr2=0x7f96f81169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:37.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.953+0000 7f96fd000700 1 --2- 172.21.15.112:0/608908738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f810a490 0x7f96f81169a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f96f000b0a0 tx=0x7f96f000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.953+0000 7f96fd000700 1 -- 172.21.15.112:0/608908738 shutdown_connections 2024-05-08T05:54:37.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.953+0000 7f96fd000700 1 --2- 172.21.15.112:0/608908738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f810a490 0x7f96f81169a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.953+0000 7f96fd000700 1 --2- 172.21.15.112:0/608908738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f810fb60 0x7f96f810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.953+0000 7f96fd000700 1 --2- 172.21.15.112:0/608908738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96f8109ad0 0x7f96f8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.953+0000 7f96fd000700 1 -- 172.21.15.112:0/608908738 >> 172.21.15.112:0/608908738 conn(0x7f96f8076c70 msgr2=0x7f96f81082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:37.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.953+0000 7f96fd000700 1 -- 172.21.15.112:0/608908738 shutdown_connections 2024-05-08T05:54:37.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.953+0000 7f96fd000700 1 -- 172.21.15.112:0/608908738 wait complete. 2024-05-08T05:54:37.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.955+0000 7f96fd000700 1 Processor -- start 2024-05-08T05:54:37.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.955+0000 7f96fd000700 1 -- start start 2024-05-08T05:54:37.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.955+0000 7f96fd000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f8109ad0 0x7f96f81a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.955+0000 7f96fd000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96f810a490 0x7f96f81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.955+0000 7f96fd000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f810fb60 0x7f96f81a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.955+0000 7f96fd000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96f8119cd0 con 0x7f96f810fb60 2024-05-08T05:54:37.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.955+0000 7f96fd000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96f8119b50 con 0x7f96f8109ad0 2024-05-08T05:54:37.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.955+0000 7f96fd000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f96f8119e50 con 0x7f96f810a490 2024-05-08T05:54:37.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.956+0000 7f96f6800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f810fb60 0x7f96f81a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.956+0000 7f96f5400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96f810a490 0x7f96f81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.956+0000 7f96f6800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f810fb60 0x7f96f81a49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45834/0 (socket says 172.21.15.112:45834) 2024-05-08T05:54:37.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.956+0000 7f96f6800700 1 -- 172.21.15.112:0/3404591719 learned_addr learned my addr 172.21.15.112:0/3404591719 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:37.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.956+0000 7f96f5e00700 1 --2- 172.21.15.112:0/3404591719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f8109ad0 0x7f96f81a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.956+0000 7f96f5400700 1 -- 172.21.15.112:0/3404591719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f8109ad0 msgr2=0x7f96f81a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:37.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.956+0000 7f96f5400700 1 --2- 172.21.15.112:0/3404591719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f8109ad0 0x7f96f81a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.956+0000 7f96f5400700 1 -- 172.21.15.112:0/3404591719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f810fb60 msgr2=0x7f96f81a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:37.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.956+0000 7f96f5400700 1 --2- 172.21.15.112:0/3404591719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f810fb60 0x7f96f81a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.956+0000 7f96f5400700 1 -- 172.21.15.112:0/3404591719 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f81a5110 con 0x7f96f810a490 2024-05-08T05:54:37.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.956+0000 7f96f5400700 1 --2- 172.21.15.112:0/3404591719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96f810a490 0x7f96f81a0640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f96e400e9f0 tx=0x7f96e400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:37.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.957+0000 7f96eea00700 1 -- 172.21.15.112:0/3404591719 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e400cb70 con 0x7f96f810a490 2024-05-08T05:54:37.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.957+0000 7f96fd000700 1 -- 172.21.15.112:0/3404591719 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f81a5400 con 0x7f96f810a490 2024-05-08T05:54:37.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.957+0000 7f96eea00700 1 -- 172.21.15.112:0/3404591719 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96e400ee90 con 0x7f96f810a490 2024-05-08T05:54:37.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.957+0000 7f96eea00700 1 -- 172.21.15.112:0/3404591719 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e4003d40 con 0x7f96f810a490 2024-05-08T05:54:37.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.957+0000 7f96fd000700 1 -- 172.21.15.112:0/3404591719 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f81acc80 con 0x7f96f810a490 2024-05-08T05:54:37.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.958+0000 7f96fd000700 1 -- 172.21.15.112:0/3404591719 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f806a190 con 0x7f96f810a490 2024-05-08T05:54:37.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.961+0000 7f96eea00700 1 -- 172.21.15.112:0/3404591719 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f96e4003ee0 con 0x7f96f810a490 2024-05-08T05:54:37.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.963+0000 7f96eea00700 1 --2- 172.21.15.112:0/3404591719 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f96dc0b2880 0x7f96dc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.963+0000 7f96eea00700 1 -- 172.21.15.112:0/3404591719 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f96e40e2600 con 0x7f96f810a490 2024-05-08T05:54:37.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.963+0000 7f96f5e00700 1 --2- 172.21.15.112:0/3404591719 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f96dc0b2880 0x7f96dc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.964+0000 7f96f5e00700 1 --2- 172.21.15.112:0/3404591719 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f96dc0b2880 0x7f96dc0b4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009680 tx=0x7f96e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:37.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:37.964+0000 7f96eea00700 1 -- 172.21.15.112:0/3404591719 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f96e4054df0 con 0x7f96f810a490 2024-05-08T05:54:38.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:38.197+0000 7f96fd000700 1 -- 172.21.15.112:0/3404591719 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96f802e510 con 0x7f96f810a490 2024-05-08T05:54:38.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:38.199+0000 7f96eea00700 1 -- 172.21.15.112:0/3404591719 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22829 (secure 0 0 0) 0x7f96e40aa290 con 0x7f96f810a490 2024-05-08T05:54:38.199 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:54:38.200 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":84,"fsid":"7a07ff60-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:48.498871+0000","modified":"2024-05-08T05:53:05.251932+0000","last_up_change":"2024-05-08T05:49:54.039380+0000","last_in_change":"2024-05-08T05:49:34.443850+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:47.485105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:50:58.918909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.175022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b378542-1bbe-4614-8805-8a6b875ada55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6803","nonce":3419091820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6805","nonce":3419091820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6809","nonce":3419091820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6807","nonce":3419091820}]},"public_addr":"172.21.15.112:6803/3419091820","cluster_addr":"172.21.15.112:6805/3419091820","heartbeat_back_addr":"172.21.15.112:6809/3419091820","heartbeat_front_addr":"172.21.15.112:6807/3419091820","state":["exists","up"]},{"osd":1,"uuid":"826ef036-2db7-470e-9956-d9a58b9181e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6801","nonce":2728419797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6803","nonce":2728419797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6807","nonce":2728419797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6805","nonce":2728419797}]},"public_addr":"172.21.15.150:6801/2728419797","cluster_addr":"172.21.15.150:6803/2728419797","heartbeat_back_addr":"172.21.15.150:6807/2728419797","heartbeat_front_addr":"172.21.15.150:6805/2728419797","state":["exists","up"]},{"osd":2,"uuid":"32e710e7-2a87-4863-bc7a-906d31aa4ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6801","nonce":1556203215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6803","nonce":1556203215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6807","nonce":1556203215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6805","nonce":1556203215}]},"public_addr":"172.21.15.162:6801/1556203215","cluster_addr":"172.21.15.162:6803/1556203215","heartbeat_back_addr":"172.21.15.162:6807/1556203215","heartbeat_front_addr":"172.21.15.162:6805/1556203215","state":["exists","up"]},{"osd":3,"uuid":"ba1d4a5f-6a95-4b9e-8a48-8524a899b004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6811","nonce":3325718755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6813","nonce":3325718755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6817","nonce":3325718755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6815","nonce":3325718755}]},"public_addr":"172.21.15.112:6811/3325718755","cluster_addr":"172.21.15.112:6813/3325718755","heartbeat_back_addr":"172.21.15.112:6817/3325718755","heartbeat_front_addr":"172.21.15.112:6815/3325718755","state":["exists","up"]},{"osd":4,"uuid":"cb4f1a42-8de8-4b50-b3bc-602de9f4803a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6809","nonce":1548914683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6811","nonce":1548914683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6815","nonce":1548914683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6813","nonce":1548914683}]},"public_addr":"172.21.15.150:6809/1548914683","cluster_addr":"172.21.15.150:6811/1548914683","heartbeat_back_addr":"172.21.15.150:6815/1548914683","heartbeat_front_addr":"172.21.15.150:6813/1548914683","state":["exists","up"]},{"osd":5,"uuid":"a26c03b8-050c-4a2c-8f23-5801fffb7c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6809","nonce":409836750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6811","nonce":409836750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6815","nonce":409836750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6813","nonce":409836750}]},"public_addr":"172.21.15.162:6809/409836750","cluster_addr":"172.21.15.162:6811/409836750","heartbeat_back_addr":"172.21.15.162:6815/409836750","heartbeat_front_addr":"172.21.15.162:6813/409836750","state":["exists","up"]},{"osd":6,"uuid":"8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6819","nonce":1098984186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6821","nonce":1098984186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6825","nonce":1098984186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6823","nonce":1098984186}]},"public_addr":"172.21.15.112:6819/1098984186","cluster_addr":"172.21.15.112:6821/1098984186","heartbeat_back_addr":"172.21.15.112:6825/1098984186","heartbeat_front_addr":"172.21.15.112:6823/1098984186","state":["exists","up"]},{"osd":7,"uuid":"af3629b4-9bed-4669-97a6-48deba503a96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6817","nonce":4009836564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6819","nonce":4009836564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6823","nonce":4009836564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6821","nonce":4009836564}]},"public_addr":"172.21.15.150:6817/4009836564","cluster_addr":"172.21.15.150:6819/4009836564","heartbeat_back_addr":"172.21.15.150:6823/4009836564","heartbeat_front_addr":"172.21.15.150:6821/4009836564","state":["exists","up"]},{"osd":8,"uuid":"9333e87d-dd78-4467-bbde-c2b44f4ea07f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6817","nonce":852791606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6819","nonce":852791606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6823","nonce":852791606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6821","nonce":852791606}]},"public_addr":"172.21.15.162:6817/852791606","cluster_addr":"172.21.15.162:6819/852791606","heartbeat_back_addr":"172.21.15.162:6823/852791606","heartbeat_front_addr":"172.21.15.162:6821/852791606","state":["exists","up"]},{"osd":9,"uuid":"09723581-1995-42e0-8db7-776ea98a918c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6827","nonce":2186764674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6829","nonce":2186764674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6833","nonce":2186764674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6831","nonce":2186764674}]},"public_addr":"172.21.15.112:6827/2186764674","cluster_addr":"172.21.15.112:6829/2186764674","heartbeat_back_addr":"172.21.15.112:6833/2186764674","heartbeat_front_addr":"172.21.15.112:6831/2186764674","state":["exists","up"]},{"osd":10,"uuid":"34e08e3e-1231-4899-a5b3-76d449105bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6825","nonce":2838995474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6827","nonce":2838995474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6831","nonce":2838995474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6829","nonce":2838995474}]},"public_addr":"172.21.15.150:6825/2838995474","cluster_addr":"172.21.15.150:6827/2838995474","heartbeat_back_addr":"172.21.15.150:6831/2838995474","heartbeat_front_addr":"172.21.15.150:6829/2838995474","state":["exists","up"]},{"osd":11,"uuid":"6fc34ec0-78ca-403c-acb7-b29d83ff64e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6825","nonce":3702614331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6827","nonce":3702614331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6831","nonce":3702614331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6829","nonce":3702614331}]},"public_addr":"172.21.15.162:6825/3702614331","cluster_addr":"172.21.15.162:6827/3702614331","heartbeat_back_addr":"172.21.15.162:6831/3702614331","heartbeat_front_addr":"172.21.15.162:6829/3702614331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:54.518702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:19.939543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:44.425586+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:16.126471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:41.696071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:05.517405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:38.549318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:03.799649+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:28.007282+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:01.395459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:27.963637+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:52.181212+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3667148497":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6800/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3425327158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1404611205":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3765334043":"2024-05-09T05:42:29.819307+0000","172.21.15.112:6800/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/2814303790":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1091751583":"2024-05-09T05:42:29.819307+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:54:38.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:38.202+0000 7f96fd000700 1 -- 172.21.15.112:0/3404591719 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f96dc0b2880 msgr2=0x7f96dc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:38.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:38.202+0000 7f96fd000700 1 --2- 172.21.15.112:0/3404591719 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f96dc0b2880 0x7f96dc0b4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009680 tx=0x7f96e8009260 comp rx=0 tx=0).stop 2024-05-08T05:54:38.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:38.202+0000 7f96fd000700 1 -- 172.21.15.112:0/3404591719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96f810a490 msgr2=0x7f96f81a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:38.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:38.202+0000 7f96fd000700 1 --2- 172.21.15.112:0/3404591719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96f810a490 0x7f96f81a0640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f96e400e9f0 tx=0x7f96e400ed00 comp rx=0 tx=0).stop 2024-05-08T05:54:38.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:38.203+0000 7f96fd000700 1 -- 172.21.15.112:0/3404591719 shutdown_connections 2024-05-08T05:54:38.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:38.203+0000 7f96fd000700 1 --2- 172.21.15.112:0/3404591719 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f96dc0b2880 0x7f96dc0b4d40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:38.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:38.203+0000 7f96fd000700 1 --2- 172.21.15.112:0/3404591719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f810fb60 0x7f96f81a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:38.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:38.203+0000 7f96fd000700 1 --2- 172.21.15.112:0/3404591719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f8109ad0 0x7f96f81a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:38.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:38.203+0000 7f96fd000700 1 --2- 172.21.15.112:0/3404591719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96f810a490 0x7f96f81a0640 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:38.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:38.203+0000 7f96fd000700 1 -- 172.21.15.112:0/3404591719 >> 172.21.15.112:0/3404591719 conn(0x7f96f8076c70 msgr2=0x7f96f8077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:38.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:38.203+0000 7f96fd000700 1 -- 172.21.15.112:0/3404591719 shutdown_connections 2024-05-08T05:54:38.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:38.203+0000 7f96fd000700 1 -- 172.21.15.112:0/3404591719 wait complete. 2024-05-08T05:54:38.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:38 smithi150 ceph-mon[34315]: pgmap v421: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:38.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:38 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3404591719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:54:39.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:38 smithi112 ceph-mon[30003]: pgmap v421: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:39.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:38 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3404591719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:54:39.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:38 smithi162 ceph-mon[34800]: pgmap v421: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:39.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:38 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3404591719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:54:39.093 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T05:51:07.312542+0000', 'modified': '2024-05-08T05:51:31.368725+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24529, 'mds_1': 24518, 'mds_2': 24511}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6837/4103977076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 4103977076}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 4103977076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.162:6835/3060051414', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6834', 'nonce': 3060051414}, {'type': 'v1', 'addr': '172.21.15.162:6835', 'nonce': 3060051414}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.162:6837/3092620994', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6836', 'nonce': 3092620994}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 3092620994}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.162:6839/539986796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6838', 'nonce': 539986796}, {'type': 'v1', 'addr': '172.21.15.162:6839', 'nonce': 539986796}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'b', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.150:6839/1284437777', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 1284437777}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 1284437777}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24529': {'gid': 24529, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.112:6841/2816461111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6840', 'nonce': 2816461111}, {'type': 'v1', 'addr': '172.21.15.112:6841', 'nonce': 2816461111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24529, 'qdb_cluster': [24529, 24511, 24518]} 2024-05-08T05:54:39.093 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T05:54:39.093 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-08T05:54:39.541 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:40.922 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:40 smithi112 ceph-mon[30003]: pgmap v422: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:40.932 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:40 smithi150 ceph-mon[34315]: pgmap v422: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:41.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:40 smithi162 ceph-mon[34800]: pgmap v422: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:41.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.653+0000 7f0c4ce00700 1 -- 172.21.15.112:0/2024656389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c48108700 msgr2=0x7f0c481102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:41.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.653+0000 7f0c4ce00700 1 --2- 172.21.15.112:0/2024656389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c48108700 0x7f0c481102d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000b0d0 tx=0x7f0c4000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.653+0000 7f0c4ce00700 1 -- 172.21.15.112:0/2024656389 shutdown_connections 2024-05-08T05:54:41.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.653+0000 7f0c4ce00700 1 --2- 172.21.15.112:0/2024656389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c48108700 0x7f0c481102d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.653+0000 7f0c4ce00700 1 --2- 172.21.15.112:0/2024656389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c48103620 0x7f0c48103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.653+0000 7f0c4ce00700 1 --2- 172.21.15.112:0/2024656389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c48103fd0 0x7f0c48108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.653+0000 7f0c4ce00700 1 -- 172.21.15.112:0/2024656389 >> 172.21.15.112:0/2024656389 conn(0x7f0c480feeb0 msgr2=0x7f0c481012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:41.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.653+0000 7f0c4ce00700 1 -- 172.21.15.112:0/2024656389 shutdown_connections 2024-05-08T05:54:41.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.653+0000 7f0c4ce00700 1 -- 172.21.15.112:0/2024656389 wait complete. 2024-05-08T05:54:41.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.654+0000 7f0c4ce00700 1 Processor -- start 2024-05-08T05:54:41.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.654+0000 7f0c4ce00700 1 -- start start 2024-05-08T05:54:41.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.654+0000 7f0c4ce00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c48103620 0x7f0c481a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c4ce00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c48103fd0 0x7f0c481a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c4ce00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c48108700 0x7f0c4819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c4ce00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481135f0 con 0x7f0c48103620 2024-05-08T05:54:41.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c4ce00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0c48113470 con 0x7f0c48108700 2024-05-08T05:54:41.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c4ce00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0c48113770 con 0x7f0c48103fd0 2024-05-08T05:54:41.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c47400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c48103620 0x7f0c481a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c47400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c48103620 0x7f0c481a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45852/0 (socket says 172.21.15.112:45852) 2024-05-08T05:54:41.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c47400700 1 -- 172.21.15.112:0/1740530559 learned_addr learned my addr 172.21.15.112:0/1740530559 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:41.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c46a00700 1 --2- 172.21.15.112:0/1740530559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c48103fd0 0x7f0c481a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c47e00700 1 --2- 172.21.15.112:0/1740530559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c48108700 0x7f0c4819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c47400700 1 -- 172.21.15.112:0/1740530559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c48103fd0 msgr2=0x7f0c481a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:41.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c47400700 1 --2- 172.21.15.112:0/1740530559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c48103fd0 0x7f0c481a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c47400700 1 -- 172.21.15.112:0/1740530559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c48108700 msgr2=0x7f0c4819bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:41.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c47400700 1 --2- 172.21.15.112:0/1740530559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c48108700 0x7f0c4819bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c47400700 1 -- 172.21.15.112:0/1740530559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4819c290 con 0x7f0c48103620 2024-05-08T05:54:41.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.655+0000 7f0c46a00700 1 --2- 172.21.15.112:0/1740530559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c48103fd0 0x7f0c481a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:41.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.656+0000 7f0c47400700 1 --2- 172.21.15.112:0/1740530559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c48103620 0x7f0c481a1920 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0c3800e9a0 tx=0x7f0c3800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:41.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.656+0000 7f0c3fe00700 1 -- 172.21.15.112:0/1740530559 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3800ca10 con 0x7f0c48103620 2024-05-08T05:54:41.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.656+0000 7f0c4ce00700 1 -- 172.21.15.112:0/1740530559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4819c580 con 0x7f0c48103620 2024-05-08T05:54:41.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.656+0000 7f0c3fe00700 1 -- 172.21.15.112:0/1740530559 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c3800cbb0 con 0x7f0c48103620 2024-05-08T05:54:41.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.656+0000 7f0c3fe00700 1 -- 172.21.15.112:0/1740530559 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38018680 con 0x7f0c48103620 2024-05-08T05:54:41.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.656+0000 7f0c4ce00700 1 -- 172.21.15.112:0/1740530559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c481a87f0 con 0x7f0c48103620 2024-05-08T05:54:41.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.660+0000 7f0c3fe00700 1 -- 172.21.15.112:0/1740530559 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0c38010430 con 0x7f0c48103620 2024-05-08T05:54:41.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.661+0000 7f0c58de4b80 1 Processor -- start 2024-05-08T05:54:41.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.662+0000 7f0c58de4b80 1 -- start start 2024-05-08T05:54:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.662+0000 7f0c58de4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 0x56409fdcd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.662+0000 7f0c58de4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdcd670 0x56409fdc73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.662+0000 7f0c58de4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 0x56409fdc7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.662+0000 7f0c58de4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56409fd199d0 con 0x56409fdccd50 2024-05-08T05:54:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.662+0000 7f0c58de4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56409fcb7710 con 0x56409fdc7a00 2024-05-08T05:54:41.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.662+0000 7f0c58de4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56409fd15990 con 0x56409fdcd670 2024-05-08T05:54:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.662+0000 7f0c3fe00700 1 --2- 172.21.15.112:0/1740530559 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0c2c0b2880 0x7f0c2c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.663+0000 7f0c47400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdcd670 0x56409fdc73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.663+0000 7f0c46a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 0x56409fdc7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.663+0000 7f0c46a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 0x56409fdc7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:42488/0 (socket says 172.21.15.112:42488) 2024-05-08T05:54:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.663+0000 7f0c47400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdcd670 0x56409fdc73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:54354/0 (socket says 172.21.15.112:54354) 2024-05-08T05:54:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.663+0000 7f0c46a00700 1 -- 172.21.15.112:0/746381806 learned_addr learned my addr 172.21.15.112:0/746381806 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.663+0000 7f0c3fe00700 1 -- 172.21.15.112:0/1740530559 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f0c380e23f0 con 0x7f0c48103620 2024-05-08T05:54:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.663+0000 7f0c47e00700 1 --2- 172.21.15.112:0/746381806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 0x56409fdcd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.663+0000 7f0c47e00700 1 --2- 172.21.15.112:0/1740530559 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0c2c0b2880 0x7f0c2c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.663+0000 7f0c46a00700 1 -- 172.21.15.112:0/746381806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdcd670 msgr2=0x56409fdc73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:41.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.663+0000 7f0c46a00700 1 --2- 172.21.15.112:0/746381806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdcd670 0x56409fdc73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.663+0000 7f0c46a00700 1 -- 172.21.15.112:0/746381806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 msgr2=0x56409fdcd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:41.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.663+0000 7f0c46a00700 1 --2- 172.21.15.112:0/746381806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 0x56409fdcd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.663+0000 7f0c46a00700 1 -- 172.21.15.112:0/746381806 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56409fc66880 con 0x56409fdc7a00 2024-05-08T05:54:41.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.664+0000 7f0c46a00700 1 --2- 172.21.15.112:0/746381806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 0x56409fdc7e80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0c340114f0 tx=0x7f0c34011800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3183195ce5e0a1f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:41.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.664+0000 7f0c47e00700 1 --2- 172.21.15.112:0/1740530559 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0c2c0b2880 0x7f0c2c0b4d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0c400060b0 tx=0x7f0c4000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:41.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.664+0000 7f0c3cc00700 1 -- 172.21.15.112:0/746381806 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34004a30 con 0x56409fdc7a00 2024-05-08T05:54:41.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.664+0000 7f0c3cc00700 1 -- 172.21.15.112:0/746381806 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c340054d0 con 0x56409fdc7a00 2024-05-08T05:54:41.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.664+0000 7f0c3cc00700 1 -- 172.21.15.112:0/746381806 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400e970 con 0x56409fdc7a00 2024-05-08T05:54:41.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.665+0000 7f0c58de4b80 1 -- 172.21.15.112:0/746381806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 msgr2=0x56409fdc7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:41.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.665+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/746381806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 0x56409fdc7e80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0c340114f0 tx=0x7f0c34011800 comp rx=0 tx=0).stop 2024-05-08T05:54:41.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.665+0000 7f0c58de4b80 1 -- 172.21.15.112:0/746381806 shutdown_connections 2024-05-08T05:54:41.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.665+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/746381806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 0x56409fdc7e80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.665+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/746381806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 0x56409fdcd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.665+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/746381806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdcd670 0x56409fdc73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.665+0000 7f0c58de4b80 1 -- 172.21.15.112:0/746381806 >> 172.21.15.112:0/746381806 conn(0x56409fd3f680 msgr2=0x56409fdc1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:41.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.665+0000 7f0c58de4b80 1 -- 172.21.15.112:0/746381806 shutdown_connections 2024-05-08T05:54:41.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.665+0000 7f0c58de4b80 1 -- 172.21.15.112:0/746381806 wait complete. 2024-05-08T05:54:41.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.666+0000 7f0c58de4b80 1 -- start start 2024-05-08T05:54:41.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.666+0000 7f0c58de4b80 1 Processor -- start 2024-05-08T05:54:41.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.666+0000 7f0c58de4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 0x56409fdd3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.666+0000 7f0c58de4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdccd50 0x56409fdd0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.666+0000 7f0c58de4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdcd670 0x56409fdd0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.666+0000 7f0c58de4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56409fdd4100 con 0x56409fdcd670 2024-05-08T05:54:41.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.666+0000 7f0c58de4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56409fc7fad0 con 0x56409fdc7a00 2024-05-08T05:54:41.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.666+0000 7f0c58de4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56409fdd4280 con 0x56409fdccd50 2024-05-08T05:54:41.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.667+0000 7f0c58de4b80 10 client.0 _ll_register_callbacks cb 0x56409fcc0c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:41.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.667+0000 7f0c46a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdccd50 0x56409fdd0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.667+0000 7f0c47400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 0x56409fdd3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.667+0000 7f0c46a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdccd50 0x56409fdd0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:54356/0 (socket says 172.21.15.112:54356) 2024-05-08T05:54:41.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.667+0000 7f0c46a00700 1 -- 172.21.15.112:0/3623856882 learned_addr learned my addr 172.21.15.112:0/3623856882 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:41.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.667+0000 7f0c47e00700 1 --2- 172.21.15.112:0/3623856882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdcd670 0x56409fdd0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.667+0000 7f0c47400700 1 -- 172.21.15.112:0/3623856882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdccd50 msgr2=0x56409fdd0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:41.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.667+0000 7f0c47400700 1 --2- 172.21.15.112:0/3623856882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdccd50 0x56409fdd0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.667+0000 7f0c47400700 1 -- 172.21.15.112:0/3623856882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdcd670 msgr2=0x56409fdd0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:41.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.667+0000 7f0c47400700 1 --2- 172.21.15.112:0/3623856882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdcd670 0x56409fdd0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.667+0000 7f0c47400700 1 -- 172.21.15.112:0/3623856882 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56409fcdf3d0 con 0x56409fdc7a00 2024-05-08T05:54:41.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.667+0000 7f0c47400700 1 --2- 172.21.15.112:0/3623856882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 0x56409fdd3360 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0c380a5a00 tx=0x7f0c380cc390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69e059e268da97d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:41.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.667+0000 7f0c31600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:54:41.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.667+0000 7f0c31600700 1 -- 172.21.15.112:0/3623856882 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38026040 con 0x56409fdc7a00 2024-05-08T05:54:41.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.668+0000 7f0c58de4b80 1 -- 172.21.15.112:0/3623856882 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c340054d0 con 0x56409fdc7a00 2024-05-08T05:54:41.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.668+0000 7f0c31600700 1 -- 172.21.15.112:0/3623856882 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f0c380b2560 con 0x56409fdc7a00 2024-05-08T05:54:41.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.668+0000 7f0c31600700 1 -- 172.21.15.112:0/3623856882 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0c380b2120 con 0x56409fdc7a00 2024-05-08T05:54:41.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.668+0000 7f0c58de4b80 10 client.24974 fetch_fsmap learned FSMap version 24 2024-05-08T05:54:41.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.668+0000 7f0c58de4b80 1 -- 172.21.15.112:0/3623856882 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56409fdde920 con 0x56409fdc7a00 2024-05-08T05:54:41.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.668+0000 7f0c31600700 1 -- 172.21.15.112:0/3623856882 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c340054d0 con 0x56409fdc7a00 2024-05-08T05:54:41.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.668+0000 7f0c31600700 1 -- 172.21.15.112:0/3623856882 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0c3806f110 con 0x56409fdc7a00 2024-05-08T05:54:41.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.668+0000 7f0c58de4b80 10 client.24974 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T05:54:41.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.669+0000 7f0c58de4b80 10 client.24974 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:54:41.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.669+0000 7f0c58de4b80 20 client.24974 populate_metadata read hostname 'smithi112' 2024-05-08T05:54:41.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.669+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/3623856882 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x56409fde0f60 0x56409fde23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.669+0000 7f0c58de4b80 4 client.24974 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:54:41.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.669+0000 7f0c58de4b80 1 -- 172.21.15.112:0/3623856882 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56409fd940f0 con 0x56409fde0f60 2024-05-08T05:54:41.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.669+0000 7f0c46a00700 1 --2- 172.21.15.112:0/3623856882 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x56409fde0f60 0x56409fde23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.670+0000 7f0c46a00700 1 --2- 172.21.15.112:0/3623856882 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x56409fde0f60 0x56409fde23d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d1131c0f36ee6ba server_cookie=93c1400619a1f6d6 in_seq=0 out_seq=0 2024-05-08T05:54:41.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.670+0000 7f0c31600700 10 client.24974 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:54:41.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.671+0000 7f0c31600700 1 -- 172.21.15.112:0/3623856882 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56409fd940f0 con 0x56409fde0f60 2024-05-08T05:54:41.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.671+0000 7f0c31600700 10 client.24974 handle_command_reply: tid=0 2024-05-08T05:54:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.671+0000 7f0c58de4b80 1 client.24974 shutdown 2024-05-08T05:54:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.671+0000 7f0c58de4b80 1 -- 172.21.15.112:0/3623856882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 msgr2=0x56409fdd3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.671+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/3623856882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 0x56409fdd3360 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0c380a5a00 tx=0x7f0c380cc390 comp rx=0 tx=0).stop 2024-05-08T05:54:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.671+0000 7f0c58de4b80 1 -- 172.21.15.112:0/3623856882 shutdown_connections 2024-05-08T05:54:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.671+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/3623856882 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x56409fde0f60 0x56409fde23d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.671+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/3623856882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdcd670 0x56409fdd0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.671+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/3623856882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 0x56409fdd3360 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.671+0000 7f0c31600700 0 client.24974 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:54:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.671+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/3623856882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdccd50 0x56409fdd0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.671+0000 7f0c58de4b80 1 -- 172.21.15.112:0/3623856882 >> 172.21.15.112:0/3623856882 conn(0x56409fd3f680 msgr2=0x56409fdbc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.671+0000 7f0c58de4b80 1 -- 172.21.15.112:0/3623856882 shutdown_connections 2024-05-08T05:54:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.671+0000 7f0c58de4b80 1 -- 172.21.15.112:0/3623856882 wait complete. 2024-05-08T05:54:41.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.671+0000 7f0c58de4b80 20 client.24974 trim_cache size 0 max 16384 2024-05-08T05:54:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.694+0000 7f0c58de4b80 1 Processor -- start 2024-05-08T05:54:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.695+0000 7f0c58de4b80 1 -- start start 2024-05-08T05:54:41.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.695+0000 7f0c58de4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdc7a00 0x56409fdc73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.695+0000 7f0c58de4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 0x56409fdd0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.695+0000 7f0c58de4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdcd670 0x56409fdbc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.695+0000 7f0c58de4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56409fde0710 con 0x56409fdccd50 2024-05-08T05:54:41.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.695+0000 7f0c58de4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56409fde0de0 con 0x56409fdcd670 2024-05-08T05:54:41.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.695+0000 7f0c58de4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56409fde14b0 con 0x56409fdc7a00 2024-05-08T05:54:41.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.695+0000 7f0c47400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdc7a00 0x56409fdc73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.695+0000 7f0c47400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdc7a00 0x56409fdc73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:54368/0 (socket says 172.21.15.112:54368) 2024-05-08T05:54:41.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.695+0000 7f0c47400700 1 -- 172.21.15.112:0/2189200488 learned_addr learned my addr 172.21.15.112:0/2189200488 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:41.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.695+0000 7f0c46a00700 1 --2- 172.21.15.112:0/2189200488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 0x56409fdd0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.695+0000 7f0c47e00700 1 --2- 172.21.15.112:0/2189200488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdcd670 0x56409fdbc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.696+0000 7f0c47400700 1 -- 172.21.15.112:0/2189200488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdcd670 msgr2=0x56409fdbc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:41.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.696+0000 7f0c47400700 1 --2- 172.21.15.112:0/2189200488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdcd670 0x56409fdbc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.696+0000 7f0c47400700 1 -- 172.21.15.112:0/2189200488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 msgr2=0x56409fdd0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:41.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.696+0000 7f0c47400700 1 --2- 172.21.15.112:0/2189200488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 0x56409fdd0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.696+0000 7f0c47400700 1 -- 172.21.15.112:0/2189200488 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56409fdd16e0 con 0x56409fdc7a00 2024-05-08T05:54:41.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.696+0000 7f0c47400700 1 --2- 172.21.15.112:0/2189200488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdc7a00 0x56409fdc73e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0c380a6bf0 tx=0x7f0c380a9400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=825ffc09cdba0885 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:41.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.696+0000 7f0c33e00700 1 -- 172.21.15.112:0/2189200488 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c380a9490 con 0x56409fdc7a00 2024-05-08T05:54:41.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.696+0000 7f0c33e00700 1 -- 172.21.15.112:0/2189200488 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c380aa080 con 0x56409fdc7a00 2024-05-08T05:54:41.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.696+0000 7f0c33e00700 1 -- 172.21.15.112:0/2189200488 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3806f370 con 0x56409fdc7a00 2024-05-08T05:54:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.697+0000 7f0c58de4b80 1 -- 172.21.15.112:0/2189200488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdc7a00 msgr2=0x56409fdc73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.697+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/2189200488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdc7a00 0x56409fdc73e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0c380a6bf0 tx=0x7f0c380a9400 comp rx=0 tx=0).stop 2024-05-08T05:54:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.697+0000 7f0c58de4b80 1 -- 172.21.15.112:0/2189200488 shutdown_connections 2024-05-08T05:54:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.697+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/2189200488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdcd670 0x56409fdbc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.697+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/2189200488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdc7a00 0x56409fdc73e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.697+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/2189200488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 0x56409fdd0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.697+0000 7f0c58de4b80 1 -- 172.21.15.112:0/2189200488 >> 172.21.15.112:0/2189200488 conn(0x56409fd3f680 msgr2=0x56409fdceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.697+0000 7f0c58de4b80 1 -- 172.21.15.112:0/2189200488 shutdown_connections 2024-05-08T05:54:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.697+0000 7f0c58de4b80 1 -- 172.21.15.112:0/2189200488 wait complete. 2024-05-08T05:54:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.698+0000 7f0c58de4b80 1 -- start start 2024-05-08T05:54:41.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.698+0000 7f0c58de4b80 1 Processor -- start 2024-05-08T05:54:41.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c58de4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 0x56409fdbbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c58de4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 0x56409fdbc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c58de4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdcd670 0x56409fdbc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c58de4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56409fde2250 con 0x56409fdccd50 2024-05-08T05:54:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c58de4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56409fde1b80 con 0x56409fdc7a00 2024-05-08T05:54:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c58de4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56409fde2920 con 0x56409fdcd670 2024-05-08T05:54:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c58de4b80 10 client.0 _ll_register_callbacks cb 0x56409fcc0c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c46a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 0x56409fdbc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c47e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdcd670 0x56409fdbc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c46a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 0x56409fdbc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45876/0 (socket says 172.21.15.112:45876) 2024-05-08T05:54:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c46a00700 1 -- 172.21.15.112:0/1612081927 learned_addr learned my addr 172.21.15.112:0/1612081927 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c47400700 1 --2- 172.21.15.112:0/1612081927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 0x56409fdbbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c46a00700 1 -- 172.21.15.112:0/1612081927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdcd670 msgr2=0x56409fdbc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c46a00700 1 --2- 172.21.15.112:0/1612081927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdcd670 0x56409fdbc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c46a00700 1 -- 172.21.15.112:0/1612081927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 msgr2=0x56409fdbbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c46a00700 1 --2- 172.21.15.112:0/1612081927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 0x56409fdbbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.699+0000 7f0c46a00700 1 -- 172.21.15.112:0/1612081927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56409fde9bb0 con 0x56409fdccd50 2024-05-08T05:54:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.700+0000 7f0c46a00700 1 --2- 172.21.15.112:0/1612081927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 0x56409fdbc0e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c340114f0 tx=0x7f0c340058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d817f638bde11e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.700+0000 7f0c31600700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:54:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.700+0000 7f0c31600700 1 -- 172.21.15.112:0/1612081927 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34004220 con 0x56409fdccd50 2024-05-08T05:54:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.700+0000 7f0c31600700 1 -- 172.21.15.112:0/1612081927 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f0c340043c0 con 0x56409fdccd50 2024-05-08T05:54:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.700+0000 7f0c58de4b80 1 -- 172.21.15.112:0/1612081927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c380aa080 con 0x56409fdccd50 2024-05-08T05:54:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.700+0000 7f0c31600700 1 -- 172.21.15.112:0/1612081927 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0c3403a030 con 0x56409fdccd50 2024-05-08T05:54:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.700+0000 7f0c58de4b80 10 client.14871 fetch_fsmap learned FSMap version 24 2024-05-08T05:54:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.700+0000 7f0c58de4b80 1 -- 172.21.15.112:0/1612081927 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56409fe4c6c0 con 0x56409fdccd50 2024-05-08T05:54:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.700+0000 7f0c31600700 1 -- 172.21.15.112:0/1612081927 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c380aa080 con 0x56409fdccd50 2024-05-08T05:54:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.701+0000 7f0c31600700 1 -- 172.21.15.112:0/1612081927 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0c3402bce0 con 0x56409fdccd50 2024-05-08T05:54:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.701+0000 7f0c58de4b80 10 client.14871 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T05:54:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.701+0000 7f0c58de4b80 10 client.14871 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:54:41.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.701+0000 7f0c58de4b80 20 client.14871 populate_metadata read hostname 'smithi112' 2024-05-08T05:54:41.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.701+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/1612081927 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x56409fe51030 0x56409fe4fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:41.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.701+0000 7f0c58de4b80 4 client.14871 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:54:41.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.701+0000 7f0c58de4b80 1 -- 172.21.15.112:0/1612081927 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "status"}) -- 0x56409fd273b0 con 0x56409fe51030 2024-05-08T05:54:41.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.701+0000 7f0c47400700 1 --2- 172.21.15.112:0/1612081927 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x56409fe51030 0x56409fe4fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:41.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.702+0000 7f0c47400700 1 --2- 172.21.15.112:0/1612081927 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x56409fe51030 0x56409fe4fd40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=879f6a1dc763987 server_cookie=5c451fe28b4d8437 in_seq=0 out_seq=0 2024-05-08T05:54:41.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.702+0000 7f0c31600700 10 client.14871 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:54:41.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.702+0000 7f0c31600700 1 -- 172.21.15.112:0/1612081927 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56409fd273b0 con 0x56409fe51030 2024-05-08T05:54:41.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.702+0000 7f0c31600700 10 client.14871 handle_command_reply: tid=0 2024-05-08T05:54:41.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.702+0000 7f0c58de4b80 1 client.14871 shutdown 2024-05-08T05:54:41.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.702+0000 7f0c58de4b80 1 -- 172.21.15.112:0/1612081927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 msgr2=0x56409fdbc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.702+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/1612081927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 0x56409fdbc0e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c340114f0 tx=0x7f0c340058e0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.702+0000 7f0c58de4b80 1 -- 172.21.15.112:0/1612081927 shutdown_connections 2024-05-08T05:54:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.702+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/1612081927 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x56409fe51030 0x56409fe4fd40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.702+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/1612081927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56409fdcd670 0x56409fdbc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.702+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/1612081927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56409fdc7a00 0x56409fdbbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.702+0000 7f0c58de4b80 1 --2- 172.21.15.112:0/1612081927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56409fdccd50 0x56409fdbc0e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.702+0000 7f0c58de4b80 1 -- 172.21.15.112:0/1612081927 >> 172.21.15.112:0/1612081927 conn(0x56409fd3f680 msgr2=0x56409fdcda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.702+0000 7f0c31600700 0 client.14871 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:54:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.703+0000 7f0c58de4b80 1 -- 172.21.15.112:0/1612081927 shutdown_connections 2024-05-08T05:54:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.703+0000 7f0c58de4b80 1 -- 172.21.15.112:0/1612081927 wait complete. 2024-05-08T05:54:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.703+0000 7f0c58de4b80 20 client.14871 trim_cache size 0 max 16384 2024-05-08T05:54:41.706 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:54:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "7a07ff60-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:54:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 2, 2024-05-08T05:54:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24511, 2024-05-08T05:54:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-08T05:54:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-08T05:54:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-08T05:54:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 197.098028927, 2024-05-08T05:54:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 24, 2024-05-08T05:54:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 84, 2024-05-08T05:54:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 80, 2024-05-08T05:54:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 247.30994906399999 2024-05-08T05:54:41.706 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:54:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.704+0000 7f0c4ce00700 1 -- 172.21.15.112:0/1740530559 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0c2c0b2880 msgr2=0x7f0c2c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.704+0000 7f0c4ce00700 1 --2- 172.21.15.112:0/1740530559 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0c2c0b2880 0x7f0c2c0b4d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0c400060b0 tx=0x7f0c4000b040 comp rx=0 tx=0).stop 2024-05-08T05:54:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.704+0000 7f0c4ce00700 1 -- 172.21.15.112:0/1740530559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c48103620 msgr2=0x7f0c481a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.704+0000 7f0c4ce00700 1 --2- 172.21.15.112:0/1740530559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c48103620 0x7f0c481a1920 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0c3800e9a0 tx=0x7f0c3800ecb0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.704+0000 7f0c4ce00700 1 -- 172.21.15.112:0/1740530559 shutdown_connections 2024-05-08T05:54:41.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.704+0000 7f0c4ce00700 1 --2- 172.21.15.112:0/1740530559 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0c2c0b2880 0x7f0c2c0b4d40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.704+0000 7f0c4ce00700 1 --2- 172.21.15.112:0/1740530559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c48108700 0x7f0c4819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.704+0000 7f0c4ce00700 1 --2- 172.21.15.112:0/1740530559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c48103620 0x7f0c481a1920 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.704+0000 7f0c4ce00700 1 --2- 172.21.15.112:0/1740530559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c48103fd0 0x7f0c481a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:41.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.704+0000 7f0c4ce00700 1 -- 172.21.15.112:0/1740530559 >> 172.21.15.112:0/1740530559 conn(0x7f0c480feeb0 msgr2=0x7f0c481001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:41.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.704+0000 7f0c4ce00700 1 -- 172.21.15.112:0/1740530559 shutdown_connections 2024-05-08T05:54:41.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:41.704+0000 7f0c4ce00700 1 -- 172.21.15.112:0/1740530559 wait complete. 2024-05-08T05:54:42.561 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-08T05:54:42.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:42 smithi112 ceph-mon[30003]: pgmap v423: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:42.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:42 smithi150 ceph-mon[34315]: pgmap v423: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:43.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:42 smithi162 ceph-mon[34800]: pgmap v423: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:44.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:44 smithi112 ceph-mon[30003]: pgmap v424: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:44.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:44 smithi150 ceph-mon[34315]: pgmap v424: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:45.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:44 smithi162 ceph-mon[34800]: pgmap v424: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.300+0000 7f252c400700 1 -- 172.21.15.112:0/2095036495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25241023a0 msgr2=0x7f2524102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.300+0000 7f252c400700 1 --2- 172.21.15.112:0/2095036495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25241023a0 0x7f2524102820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f25100098f0 tx=0x7f2510009c00 comp rx=0 tx=0).stop 2024-05-08T05:54:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.300+0000 7f252c400700 1 -- 172.21.15.112:0/2095036495 shutdown_connections 2024-05-08T05:54:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.300+0000 7f252c400700 1 --2- 172.21.15.112:0/2095036495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2524102d60 0x7f252410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.300+0000 7f252c400700 1 --2- 172.21.15.112:0/2095036495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2524108410 0x7f25241087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.300+0000 7f252c400700 1 --2- 172.21.15.112:0/2095036495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25241023a0 0x7f2524102820 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.300+0000 7f252c400700 1 -- 172.21.15.112:0/2095036495 >> 172.21.15.112:0/2095036495 conn(0x7f2524079350 msgr2=0x7f2524079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:45.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.300+0000 7f252c400700 1 -- 172.21.15.112:0/2095036495 shutdown_connections 2024-05-08T05:54:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.300+0000 7f252c400700 1 -- 172.21.15.112:0/2095036495 wait complete. 2024-05-08T05:54:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.302+0000 7f252c400700 1 Processor -- start 2024-05-08T05:54:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.302+0000 7f252c400700 1 -- start start 2024-05-08T05:54:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.302+0000 7f252c400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25241023a0 0x7f252419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.302+0000 7f252c400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2524102d60 0x7f252419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.302+0000 7f252c400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2524108410 0x7f25241a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.302+0000 7f252c400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2524112550 con 0x7f2524102d60 2024-05-08T05:54:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.302+0000 7f252c400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f25241123d0 con 0x7f25241023a0 2024-05-08T05:54:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.302+0000 7f252c400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f25241126d0 con 0x7f2524108410 2024-05-08T05:54:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.303+0000 7f2529a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25241023a0 0x7f252419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.303+0000 7f2529a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25241023a0 0x7f252419bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:42116/0 (socket says 172.21.15.112:42116) 2024-05-08T05:54:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.303+0000 7f2529a00700 1 -- 172.21.15.112:0/2703198517 learned_addr learned my addr 172.21.15.112:0/2703198517 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.303+0000 7f252a400700 1 --2- 172.21.15.112:0/2703198517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2524108410 0x7f25241a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.303+0000 7f2529000700 1 --2- 172.21.15.112:0/2703198517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2524102d60 0x7f252419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.303+0000 7f2529a00700 1 -- 172.21.15.112:0/2703198517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2524108410 msgr2=0x7f25241a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.303+0000 7f2529a00700 1 --2- 172.21.15.112:0/2703198517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2524108410 0x7f25241a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.303+0000 7f2529a00700 1 -- 172.21.15.112:0/2703198517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2524102d60 msgr2=0x7f252419c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.303+0000 7f2529a00700 1 --2- 172.21.15.112:0/2703198517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2524102d60 0x7f252419c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.303+0000 7f2529a00700 1 -- 172.21.15.112:0/2703198517 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25241a0e00 con 0x7f25241023a0 2024-05-08T05:54:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.303+0000 7f2529000700 1 --2- 172.21.15.112:0/2703198517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2524102d60 0x7f252419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:54:45.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.303+0000 7f2529a00700 1 --2- 172.21.15.112:0/2703198517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25241023a0 0x7f252419bdf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f251800e9c0 tx=0x7f251800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:45.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.304+0000 7f2522000700 1 -- 172.21.15.112:0/2703198517 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251800cbd0 con 0x7f25241023a0 2024-05-08T05:54:45.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.304+0000 7f2522000700 1 -- 172.21.15.112:0/2703198517 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f251800cd70 con 0x7f25241023a0 2024-05-08T05:54:45.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.304+0000 7f2522000700 1 -- 172.21.15.112:0/2703198517 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518005610 con 0x7f25241023a0 2024-05-08T05:54:45.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.304+0000 7f252c400700 1 -- 172.21.15.112:0/2703198517 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25241a10f0 con 0x7f25241023a0 2024-05-08T05:54:45.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.304+0000 7f252c400700 1 -- 172.21.15.112:0/2703198517 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25241a8970 con 0x7f25241023a0 2024-05-08T05:54:45.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.307+0000 7f2522000700 1 -- 172.21.15.112:0/2703198517 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2518004750 con 0x7f25241023a0 2024-05-08T05:54:45.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.309+0000 7f2536d0ab80 1 Processor -- start 2024-05-08T05:54:45.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.309+0000 7f2536d0ab80 1 -- start start 2024-05-08T05:54:45.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.309+0000 7f2536d0ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a504f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.309+0000 7f2522000700 1 --2- 172.21.15.112:0/2703198517 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f250c0b2900 0x7f250c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.309+0000 7f2536d0ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a505490 0x561e0a4ff280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.309+0000 7f2536d0ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a4ff8a0 0x561e0a4ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.309+0000 7f2536d0ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561e0a4519d0 con 0x561e0a4ff8a0 2024-05-08T05:54:45.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.309+0000 7f2536d0ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561e0a3ef710 con 0x561e0a505490 2024-05-08T05:54:45.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.309+0000 7f2536d0ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561e0a44d990 con 0x561e0a504b70 2024-05-08T05:54:45.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.310+0000 7f2529000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a505490 0x561e0a4ff280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.310+0000 7f2529000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a505490 0x561e0a4ff280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:42124/0 (socket says 172.21.15.112:42124) 2024-05-08T05:54:45.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.310+0000 7f2529000700 1 -- 172.21.15.112:0/1289321321 learned_addr learned my addr 172.21.15.112:0/1289321321 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:45.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.310+0000 7f2529a00700 1 --2- 172.21.15.112:0/1289321321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a504f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.310+0000 7f2522000700 1 -- 172.21.15.112:0/2703198517 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f25180e2500 con 0x7f25241023a0 2024-05-08T05:54:45.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.310+0000 7f252a400700 1 --2- 172.21.15.112:0/2703198517 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f250c0b2900 0x7f250c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.310+0000 7f252a400700 1 --2- 172.21.15.112:0/1289321321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a4ff8a0 0x561e0a4ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.310+0000 7f2529a00700 1 -- 172.21.15.112:0/1289321321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a505490 msgr2=0x561e0a4ff280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:45.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.310+0000 7f2529a00700 1 --2- 172.21.15.112:0/1289321321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a505490 0x561e0a4ff280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.310+0000 7f2529a00700 1 -- 172.21.15.112:0/1289321321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a4ff8a0 msgr2=0x561e0a4ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:45.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.310+0000 7f2529a00700 1 --2- 172.21.15.112:0/1289321321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a4ff8a0 0x561e0a4ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.310+0000 7f2529a00700 1 -- 172.21.15.112:0/1289321321 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0a39e880 con 0x561e0a504b70 2024-05-08T05:54:45.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.310+0000 7f2529000700 1 --2- 172.21.15.112:0/1289321321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a505490 0x561e0a4ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:45.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.311+0000 7f252a400700 1 --2- 172.21.15.112:0/2703198517 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f250c0b2900 0x7f250c0b4dc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x561e0a500260 tx=0x7f251c0099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:45.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.311+0000 7f2529a00700 1 --2- 172.21.15.112:0/1289321321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a504f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f25180a6d00 tx=0x7f25180a9200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=117638300aed2170 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:45.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.311+0000 7f2516a00700 1 -- 172.21.15.112:0/1289321321 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25180a5220 con 0x561e0a504b70 2024-05-08T05:54:45.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.311+0000 7f2516a00700 1 -- 172.21.15.112:0/1289321321 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25180b2230 con 0x561e0a504b70 2024-05-08T05:54:45.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.311+0000 7f2516a00700 1 -- 172.21.15.112:0/1289321321 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25180aaff0 con 0x561e0a504b70 2024-05-08T05:54:45.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.312+0000 7f2536d0ab80 1 -- 172.21.15.112:0/1289321321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 msgr2=0x561e0a504f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:45.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.312+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/1289321321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a504f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f25180a6d00 tx=0x7f25180a9200 comp rx=0 tx=0).stop 2024-05-08T05:54:45.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.312+0000 7f2536d0ab80 1 -- 172.21.15.112:0/1289321321 shutdown_connections 2024-05-08T05:54:45.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.312+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/1289321321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a4ff8a0 0x561e0a4ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.312+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/1289321321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a504f50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.312+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/1289321321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a505490 0x561e0a4ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.312+0000 7f2536d0ab80 1 -- 172.21.15.112:0/1289321321 >> 172.21.15.112:0/1289321321 conn(0x561e0a477680 msgr2=0x561e0a4f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:45.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.312+0000 7f2536d0ab80 1 -- 172.21.15.112:0/1289321321 shutdown_connections 2024-05-08T05:54:45.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.312+0000 7f2536d0ab80 1 -- 172.21.15.112:0/1289321321 wait complete. 2024-05-08T05:54:45.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.312+0000 7f2536d0ab80 1 -- start start 2024-05-08T05:54:45.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.313+0000 7f2536d0ab80 1 Processor -- start 2024-05-08T05:54:45.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.313+0000 7f2536d0ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a4ff8a0 0x561e0a50b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.313+0000 7f2536d0ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a508560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.313+0000 7f2536d0ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a505490 0x561e0a508aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.313+0000 7f2536d0ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561e0a50bed0 con 0x561e0a505490 2024-05-08T05:54:45.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.313+0000 7f2536d0ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561e0a3b7ad0 con 0x561e0a4ff8a0 2024-05-08T05:54:45.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.313+0000 7f2536d0ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561e0a50c050 con 0x561e0a504b70 2024-05-08T05:54:45.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.313+0000 7f2536d0ab80 10 client.0 _ll_register_callbacks cb 0x561e0a3f8c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:45.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.314+0000 7f2529a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a4ff8a0 0x561e0a50b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.314+0000 7f252a400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a505490 0x561e0a508aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.314+0000 7f2529a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a4ff8a0 0x561e0a50b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:42132/0 (socket says 172.21.15.112:42132) 2024-05-08T05:54:45.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.314+0000 7f2529a00700 1 -- 172.21.15.112:0/1837112812 learned_addr learned my addr 172.21.15.112:0/1837112812 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:45.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.314+0000 7f2529000700 1 --2- 172.21.15.112:0/1837112812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a508560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.314+0000 7f2529a00700 1 -- 172.21.15.112:0/1837112812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 msgr2=0x561e0a508560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:45.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.314+0000 7f2529a00700 1 --2- 172.21.15.112:0/1837112812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a508560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.314+0000 7f2529a00700 1 -- 172.21.15.112:0/1837112812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a505490 msgr2=0x561e0a508aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:45.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.314+0000 7f2529a00700 1 --2- 172.21.15.112:0/1837112812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a505490 0x561e0a508aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.314+0000 7f2529a00700 1 -- 172.21.15.112:0/1837112812 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e0a4173d0 con 0x561e0a4ff8a0 2024-05-08T05:54:45.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.314+0000 7f2529000700 1 --2- 172.21.15.112:0/1837112812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a508560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:45.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.314+0000 7f2529a00700 1 --2- 172.21.15.112:0/1837112812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a4ff8a0 0x561e0a50b1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2518009880 tx=0x7f251800cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1582a13c9044e8b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:45.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.314+0000 7f250b400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:54:45.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.315+0000 7f250b400700 1 -- 172.21.15.112:0/1837112812 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25180a5220 con 0x561e0a4ff8a0 2024-05-08T05:54:45.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.315+0000 7f2536d0ab80 1 -- 172.21.15.112:0/1837112812 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561e0a4cc0f0 con 0x561e0a4ff8a0 2024-05-08T05:54:45.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.315+0000 7f250b400700 1 -- 172.21.15.112:0/1837112812 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f25180a9f00 con 0x561e0a4ff8a0 2024-05-08T05:54:45.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.315+0000 7f250b400700 1 -- 172.21.15.112:0/1837112812 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f25180a9a90 con 0x561e0a4ff8a0 2024-05-08T05:54:45.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.315+0000 7f2536d0ab80 10 client.24992 fetch_fsmap learned FSMap version 24 2024-05-08T05:54:45.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.315+0000 7f2536d0ab80 1 -- 172.21.15.112:0/1837112812 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561e0a516780 con 0x561e0a4ff8a0 2024-05-08T05:54:45.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.315+0000 7f250b400700 1 -- 172.21.15.112:0/1837112812 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561e0a4cc0f0 con 0x561e0a4ff8a0 2024-05-08T05:54:45.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.315+0000 7f250b400700 1 -- 172.21.15.112:0/1837112812 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f25180b5c80 con 0x561e0a4ff8a0 2024-05-08T05:54:45.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.315+0000 7f2536d0ab80 10 client.24992 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T05:54:45.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.315+0000 7f2536d0ab80 10 client.24992 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:54:45.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.315+0000 7f2536d0ab80 20 client.24992 populate_metadata read hostname 'smithi112' 2024-05-08T05:54:45.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.315+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/1837112812 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561e0a517ad0 0x561e0a519c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.316+0000 7f2536d0ab80 4 client.24992 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:54:45.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.316+0000 7f2536d0ab80 1 -- 172.21.15.112:0/1837112812 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e0a45f3b0 con 0x561e0a517ad0 2024-05-08T05:54:45.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.316+0000 7f2529000700 1 --2- 172.21.15.112:0/1837112812 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561e0a517ad0 0x561e0a519c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.316+0000 7f2529000700 1 --2- 172.21.15.112:0/1837112812 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561e0a517ad0 0x561e0a519c90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1446e7d3a5209416 server_cookie=806a1f5e6cbc7a6f in_seq=0 out_seq=0 2024-05-08T05:54:45.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.316+0000 7f250b400700 10 client.24992 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T05:54:45.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.317+0000 7f250b400700 1 -- 172.21.15.112:0/1837112812 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561e0a45f3b0 con 0x561e0a517ad0 2024-05-08T05:54:45.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.317+0000 7f250b400700 10 client.24992 handle_command_reply: tid=0 2024-05-08T05:54:45.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.317+0000 7f2536d0ab80 1 client.24992 shutdown 2024-05-08T05:54:45.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.317+0000 7f2536d0ab80 1 -- 172.21.15.112:0/1837112812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a4ff8a0 msgr2=0x561e0a50b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:45.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.318+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/1837112812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a4ff8a0 0x561e0a50b1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2518009880 tx=0x7f251800cf90 comp rx=0 tx=0).stop 2024-05-08T05:54:45.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.318+0000 7f2536d0ab80 1 -- 172.21.15.112:0/1837112812 shutdown_connections 2024-05-08T05:54:45.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.318+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/1837112812 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561e0a517ad0 0x561e0a519c90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.318+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/1837112812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a505490 0x561e0a508aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.318+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/1837112812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a4ff8a0 0x561e0a50b1c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.318+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/1837112812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a508560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.318+0000 7f250b400700 0 client.24992 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T05:54:45.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.318+0000 7f2536d0ab80 1 -- 172.21.15.112:0/1837112812 >> 172.21.15.112:0/1837112812 conn(0x561e0a477680 msgr2=0x561e0a4f9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:45.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.318+0000 7f2536d0ab80 1 -- 172.21.15.112:0/1837112812 shutdown_connections 2024-05-08T05:54:45.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.318+0000 7f2536d0ab80 1 -- 172.21.15.112:0/1837112812 wait complete. 2024-05-08T05:54:45.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.318+0000 7f2536d0ab80 20 client.24992 trim_cache size 0 max 16384 2024-05-08T05:54:45.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.340+0000 7f2536d0ab80 1 Processor -- start 2024-05-08T05:54:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.340+0000 7f2536d0ab80 1 -- start start 2024-05-08T05:54:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f2536d0ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a4ff8a0 0x561e0a5091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f2536d0ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a4f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f2536d0ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a505490 0x561e0a5142e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f2536d0ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561e0a517fd0 con 0x561e0a505490 2024-05-08T05:54:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f2536d0ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561e0a5186a0 con 0x561e0a4ff8a0 2024-05-08T05:54:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f2536d0ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561e0a518d70 con 0x561e0a504b70 2024-05-08T05:54:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f252a400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a505490 0x561e0a5142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f252a400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a505490 0x561e0a5142e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40238/0 (socket says 172.21.15.112:40238) 2024-05-08T05:54:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f252a400700 1 -- 172.21.15.112:0/4194180603 learned_addr learned my addr 172.21.15.112:0/4194180603 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f2529a00700 1 --2- 172.21.15.112:0/4194180603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a4ff8a0 0x561e0a5091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f2529000700 1 --2- 172.21.15.112:0/4194180603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a4f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f252a400700 1 -- 172.21.15.112:0/4194180603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 msgr2=0x561e0a4f98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f252a400700 1 --2- 172.21.15.112:0/4194180603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a4f98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f252a400700 1 -- 172.21.15.112:0/4194180603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a4ff8a0 msgr2=0x561e0a5091f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f252a400700 1 --2- 172.21.15.112:0/4194180603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a4ff8a0 0x561e0a5091f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.341+0000 7f252a400700 1 -- 172.21.15.112:0/4194180603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0a50ae00 con 0x561e0a505490 2024-05-08T05:54:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.342+0000 7f2529a00700 1 --2- 172.21.15.112:0/4194180603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a4ff8a0 0x561e0a5091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.342+0000 7f252a400700 1 --2- 172.21.15.112:0/4194180603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a505490 0x561e0a5142e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f251c0082d0 tx=0x7f251c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826c9f6548f91385 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.342+0000 7f2516000700 1 -- 172.21.15.112:0/4194180603 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c007c10 con 0x561e0a505490 2024-05-08T05:54:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.342+0000 7f2516000700 1 -- 172.21.15.112:0/4194180603 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f251c007db0 con 0x561e0a505490 2024-05-08T05:54:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.342+0000 7f2516000700 1 -- 172.21.15.112:0/4194180603 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c025740 con 0x561e0a505490 2024-05-08T05:54:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.343+0000 7f2536d0ab80 1 -- 172.21.15.112:0/4194180603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a505490 msgr2=0x561e0a5142e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.343+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/4194180603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a505490 0x561e0a5142e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f251c0082d0 tx=0x7f251c00b040 comp rx=0 tx=0).stop 2024-05-08T05:54:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.343+0000 7f2536d0ab80 1 -- 172.21.15.112:0/4194180603 shutdown_connections 2024-05-08T05:54:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.343+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/4194180603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a505490 0x561e0a5142e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.343+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/4194180603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a4ff8a0 0x561e0a5091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.343+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/4194180603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a4f98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.343+0000 7f2536d0ab80 1 -- 172.21.15.112:0/4194180603 >> 172.21.15.112:0/4194180603 conn(0x561e0a477680 msgr2=0x561e0a503fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.343+0000 7f2536d0ab80 1 -- 172.21.15.112:0/4194180603 shutdown_connections 2024-05-08T05:54:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.343+0000 7f2536d0ab80 1 -- 172.21.15.112:0/4194180603 wait complete. 2024-05-08T05:54:45.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.344+0000 7f2536d0ab80 1 -- start start 2024-05-08T05:54:45.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.344+0000 7f2536d0ab80 1 Processor -- start 2024-05-08T05:54:45.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.344+0000 7f2536d0ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a4ff8a0 0x561e0a523470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.344+0000 7f2536d0ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a51f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.344+0000 7f2536d0ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a505490 0x561e0a51fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.344+0000 7f2536d0ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561e0a519b10 con 0x561e0a4ff8a0 2024-05-08T05:54:45.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.344+0000 7f2536d0ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561e0a519440 con 0x561e0a505490 2024-05-08T05:54:45.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.344+0000 7f2536d0ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561e0a51a1e0 con 0x561e0a504b70 2024-05-08T05:54:45.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.344+0000 7f2536d0ab80 10 client.0 _ll_register_callbacks cb 0x561e0a3f8c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:45.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.345+0000 7f2529a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a4ff8a0 0x561e0a523470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.345+0000 7f252a400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a505490 0x561e0a51fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.345+0000 7f2529000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a51f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.345+0000 7f2529a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a4ff8a0 0x561e0a523470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40246/0 (socket says 172.21.15.112:40246) 2024-05-08T05:54:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.345+0000 7f2529a00700 1 -- 172.21.15.112:0/3976178628 learned_addr learned my addr 172.21.15.112:0/3976178628 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.345+0000 7f252a400700 1 -- 172.21.15.112:0/3976178628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 msgr2=0x561e0a51f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.345+0000 7f252a400700 1 --2- 172.21.15.112:0/3976178628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a51f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.345+0000 7f252a400700 1 -- 172.21.15.112:0/3976178628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a4ff8a0 msgr2=0x561e0a523470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.345+0000 7f252a400700 1 --2- 172.21.15.112:0/3976178628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a4ff8a0 0x561e0a523470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.345+0000 7f252a400700 1 -- 172.21.15.112:0/3976178628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e0a520400 con 0x561e0a505490 2024-05-08T05:54:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.345+0000 7f2529000700 1 --2- 172.21.15.112:0/3976178628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a51f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.345+0000 7f252a400700 1 --2- 172.21.15.112:0/3976178628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a505490 0x561e0a51fd90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f251c009610 tx=0x7f251c007f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8565b1c91961937a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.345+0000 7f250b400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:54:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.346+0000 7f250b400700 1 -- 172.21.15.112:0/3976178628 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c007c10 con 0x561e0a505490 2024-05-08T05:54:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.346+0000 7f2536d0ab80 1 -- 172.21.15.112:0/3976178628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f251c007db0 con 0x561e0a505490 2024-05-08T05:54:45.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.346+0000 7f250b400700 1 -- 172.21.15.112:0/3976178628 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f251c01a840 con 0x561e0a505490 2024-05-08T05:54:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.346+0000 7f250b400700 1 -- 172.21.15.112:0/3976178628 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f251c008000 con 0x561e0a505490 2024-05-08T05:54:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.346+0000 7f2536d0ab80 10 client.24998 fetch_fsmap learned FSMap version 24 2024-05-08T05:54:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.346+0000 7f2536d0ab80 1 -- 172.21.15.112:0/3976178628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561e0a584b30 con 0x561e0a505490 2024-05-08T05:54:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.346+0000 7f250b400700 1 -- 172.21.15.112:0/3976178628 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f251c007db0 con 0x561e0a505490 2024-05-08T05:54:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.347+0000 7f250b400700 1 -- 172.21.15.112:0/3976178628 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f251c035070 con 0x561e0a505490 2024-05-08T05:54:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.347+0000 7f2536d0ab80 10 client.24998 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T05:54:45.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.347+0000 7f2536d0ab80 10 client.24998 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:54:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.347+0000 7f2536d0ab80 20 client.24998 populate_metadata read hostname 'smithi112' 2024-05-08T05:54:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.347+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/3976178628 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561e0a589520 0x561e0a588230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.347+0000 7f2536d0ab80 4 client.24998 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:54:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.347+0000 7f2536d0ab80 1 -- 172.21.15.112:0/3976178628 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "status"}) -- 0x561e0a459450 con 0x561e0a589520 2024-05-08T05:54:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.347+0000 7f2529a00700 1 --2- 172.21.15.112:0/3976178628 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561e0a589520 0x561e0a588230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f2529a00700 1 --2- 172.21.15.112:0/3976178628 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561e0a589520 0x561e0a588230 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99cf48e5083aaa82 server_cookie=46c4784dfe5bffbc in_seq=0 out_seq=0 2024-05-08T05:54:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f250b400700 10 client.24998 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T05:54:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f250b400700 1 -- 172.21.15.112:0/3976178628 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561e0a459450 con 0x561e0a589520 2024-05-08T05:54:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f250b400700 10 client.24998 handle_command_reply: tid=0 2024-05-08T05:54:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f2536d0ab80 1 client.24998 shutdown 2024-05-08T05:54:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f2536d0ab80 1 -- 172.21.15.112:0/3976178628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a505490 msgr2=0x561e0a51fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/3976178628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a505490 0x561e0a51fd90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f251c009610 tx=0x7f251c007f60 comp rx=0 tx=0).stop 2024-05-08T05:54:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f2536d0ab80 1 -- 172.21.15.112:0/3976178628 shutdown_connections 2024-05-08T05:54:45.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/3976178628 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561e0a589520 0x561e0a588230 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/3976178628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561e0a505490 0x561e0a51fd90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/3976178628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e0a4ff8a0 0x561e0a523470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f2536d0ab80 1 --2- 172.21.15.112:0/3976178628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561e0a504b70 0x561e0a51f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f250b400700 0 client.24998 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T05:54:45.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f2536d0ab80 1 -- 172.21.15.112:0/3976178628 >> 172.21.15.112:0/3976178628 conn(0x561e0a477680 msgr2=0x561e0a509b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:45.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f2536d0ab80 1 -- 172.21.15.112:0/3976178628 shutdown_connections 2024-05-08T05:54:45.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f2536d0ab80 1 -- 172.21.15.112:0/3976178628 wait complete. 2024-05-08T05:54:45.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.348+0000 7f2536d0ab80 20 client.24998 trim_cache size 0 max 16384 2024-05-08T05:54:45.351 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:54:45.351 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "7a07ff60-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:54:45.351 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-05-08T05:54:45.352 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24518, 2024-05-08T05:54:45.352 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-08T05:54:45.352 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-08T05:54:45.352 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-08T05:54:45.352 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 201.75943459000001, 2024-05-08T05:54:45.352 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 24, 2024-05-08T05:54:45.352 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 84, 2024-05-08T05:54:45.352 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 80, 2024-05-08T05:54:45.352 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 239.38332140899999 2024-05-08T05:54:45.352 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:54:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.349+0000 7f252c400700 1 -- 172.21.15.112:0/2703198517 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f250c0b2900 msgr2=0x7f250c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.349+0000 7f252c400700 1 --2- 172.21.15.112:0/2703198517 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f250c0b2900 0x7f250c0b4dc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x561e0a500260 tx=0x7f251c0099e0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.349+0000 7f252c400700 1 -- 172.21.15.112:0/2703198517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25241023a0 msgr2=0x7f252419bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.349+0000 7f252c400700 1 --2- 172.21.15.112:0/2703198517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25241023a0 0x7f252419bdf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f251800e9c0 tx=0x7f251800ecd0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.349+0000 7f252c400700 1 -- 172.21.15.112:0/2703198517 shutdown_connections 2024-05-08T05:54:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.349+0000 7f252c400700 1 --2- 172.21.15.112:0/2703198517 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f250c0b2900 0x7f250c0b4dc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.349+0000 7f252c400700 1 --2- 172.21.15.112:0/2703198517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2524108410 0x7f25241a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.349+0000 7f252c400700 1 --2- 172.21.15.112:0/2703198517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25241023a0 0x7f252419bdf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.349+0000 7f252c400700 1 --2- 172.21.15.112:0/2703198517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2524102d60 0x7f252419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.349+0000 7f252c400700 1 -- 172.21.15.112:0/2703198517 >> 172.21.15.112:0/2703198517 conn(0x7f2524079350 msgr2=0x7f252406b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.349+0000 7f252c400700 1 -- 172.21.15.112:0/2703198517 shutdown_connections 2024-05-08T05:54:45.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:45.349+0000 7f252c400700 1 -- 172.21.15.112:0/2703198517 wait complete. 2024-05-08T05:54:46.201 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.a status 2024-05-08T05:54:46.648 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:46.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:46 smithi150 ceph-mon[34315]: pgmap v425: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:46.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:46 smithi112 ceph-mon[30003]: pgmap v425: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:47.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:46 smithi162 ceph-mon[34800]: pgmap v425: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:48.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.866+0000 7f90b5600700 1 -- 172.21.15.112:0/1015626491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b010e8b0 msgr2=0x7f90b010ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:48.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.866+0000 7f90b5600700 1 --2- 172.21.15.112:0/1015626491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b010e8b0 0x7f90b010ec90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009990 tx=0x7f90a0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.866+0000 7f90b5600700 1 -- 172.21.15.112:0/1015626491 shutdown_connections 2024-05-08T05:54:48.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.866+0000 7f90b5600700 1 --2- 172.21.15.112:0/1015626491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b010a3a0 0x7f90b0116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.866+0000 7f90b5600700 1 --2- 172.21.15.112:0/1015626491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b010e8b0 0x7f90b010ec90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.866+0000 7f90b5600700 1 --2- 172.21.15.112:0/1015626491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90b01099e0 0x7f90b0109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.866+0000 7f90b5600700 1 -- 172.21.15.112:0/1015626491 >> 172.21.15.112:0/1015626491 conn(0x7f90b0079230 msgr2=0x7f90b01082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:48.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.866+0000 7f90b5600700 1 -- 172.21.15.112:0/1015626491 shutdown_connections 2024-05-08T05:54:48.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.866+0000 7f90b5600700 1 -- 172.21.15.112:0/1015626491 wait complete. 2024-05-08T05:54:48.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.867+0000 7f90b5600700 1 Processor -- start 2024-05-08T05:54:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.867+0000 7f90b5600700 1 -- start start 2024-05-08T05:54:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.868+0000 7f90b5600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b01099e0 0x7f90b01a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.868+0000 7f90b5600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90b010a3a0 0x7f90b01a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.868+0000 7f90b5600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b010e8b0 0x7f90b01a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.868+0000 7f90b5600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90b0119bf0 con 0x7f90b010e8b0 2024-05-08T05:54:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.868+0000 7f90b5600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f90b0119a70 con 0x7f90b01099e0 2024-05-08T05:54:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.868+0000 7f90b5600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f90b0119d70 con 0x7f90b010a3a0 2024-05-08T05:54:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.868+0000 7f90af400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90b010a3a0 0x7f90b01a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.868+0000 7f90b4c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b010e8b0 0x7f90b01a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.868+0000 7f90af400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90b010a3a0 0x7f90b01a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50458/0 (socket says 172.21.15.112:50458) 2024-05-08T05:54:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.868+0000 7f90af400700 1 -- 172.21.15.112:0/2003541917 learned_addr learned my addr 172.21.15.112:0/2003541917 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:48.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.868+0000 7f90afe00700 1 --2- 172.21.15.112:0/2003541917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b01099e0 0x7f90b01a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.869+0000 7f90b4c00700 1 -- 172.21.15.112:0/2003541917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90b010a3a0 msgr2=0x7f90b01a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:48.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.869+0000 7f90b4c00700 1 --2- 172.21.15.112:0/2003541917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90b010a3a0 0x7f90b01a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.869+0000 7f90b4c00700 1 -- 172.21.15.112:0/2003541917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b01099e0 msgr2=0x7f90b01a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:48.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.869+0000 7f90b4c00700 1 --2- 172.21.15.112:0/2003541917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b01099e0 0x7f90b01a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.869+0000 7f90b4c00700 1 -- 172.21.15.112:0/2003541917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b01a50f0 con 0x7f90b010e8b0 2024-05-08T05:54:48.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.869+0000 7f90af400700 1 --2- 172.21.15.112:0/2003541917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90b010a3a0 0x7f90b01a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:48.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.869+0000 7f90b4c00700 1 --2- 172.21.15.112:0/2003541917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b010e8b0 0x7f90b01a49b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f90a800efc0 tx=0x7f90a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:48.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.869+0000 7f90acc00700 1 -- 172.21.15.112:0/2003541917 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a80092e0 con 0x7f90b010e8b0 2024-05-08T05:54:48.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.870+0000 7f90acc00700 1 -- 172.21.15.112:0/2003541917 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a8004bb0 con 0x7f90b010e8b0 2024-05-08T05:54:48.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.870+0000 7f90b5600700 1 -- 172.21.15.112:0/2003541917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b01a5380 con 0x7f90b010e8b0 2024-05-08T05:54:48.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.870+0000 7f90acc00700 1 -- 172.21.15.112:0/2003541917 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8004020 con 0x7f90b010e8b0 2024-05-08T05:54:48.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.870+0000 7f90b5600700 1 -- 172.21.15.112:0/2003541917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b01acc60 con 0x7f90b010e8b0 2024-05-08T05:54:48.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.873+0000 7f90acc00700 1 -- 172.21.15.112:0/2003541917 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f90a801b030 con 0x7f90b010e8b0 2024-05-08T05:54:48.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.874+0000 7f90c14deb80 1 Processor -- start 2024-05-08T05:54:48.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.875+0000 7f90c14deb80 1 -- start start 2024-05-08T05:54:48.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.875+0000 7f90acc00700 1 --2- 172.21.15.112:0/2003541917 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f90940b2880 0x7f90940b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.875+0000 7f90c14deb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba3158990 0x55eba3158d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.875+0000 7f90c14deb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31592b0 0x55eba3152f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.875+0000 7f90c14deb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31535b0 0x55eba3153a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.875+0000 7f90c14deb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eba30a59d0 con 0x55eba31592b0 2024-05-08T05:54:48.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.875+0000 7f90c14deb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55eba3043710 con 0x55eba3158990 2024-05-08T05:54:48.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.875+0000 7f90c14deb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55eba30a1990 con 0x55eba31535b0 2024-05-08T05:54:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.875+0000 7f90acc00700 1 -- 172.21.15.112:0/2003541917 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f90a80e1ca0 con 0x7f90b010e8b0 2024-05-08T05:54:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.875+0000 7f90af400700 1 --2- 172.21.15.112:0/2003541917 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f90940b2880 0x7f90940b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.875+0000 7f90b4c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31535b0 0x55eba3153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.876+0000 7f90b4c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31535b0 0x55eba3153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50472/0 (socket says 172.21.15.112:50472) 2024-05-08T05:54:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.876+0000 7f90b4c00700 1 -- 172.21.15.112:0/518812606 learned_addr learned my addr 172.21.15.112:0/518812606 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.876+0000 7f90af400700 1 --2- 172.21.15.112:0/518812606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31592b0 0x55eba3152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.876+0000 7f90afe00700 1 --2- 172.21.15.112:0/518812606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba3158990 0x55eba3158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.876+0000 7f90b4c00700 1 -- 172.21.15.112:0/518812606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba3158990 msgr2=0x55eba3158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.876+0000 7f90b4c00700 1 --2- 172.21.15.112:0/518812606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba3158990 0x55eba3158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.876+0000 7f90b4c00700 1 -- 172.21.15.112:0/518812606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31592b0 msgr2=0x55eba3152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:48.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.876+0000 7f90b4c00700 1 --2- 172.21.15.112:0/518812606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31592b0 0x55eba3152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.876+0000 7f90b4c00700 1 -- 172.21.15.112:0/518812606 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eba2ff2880 con 0x55eba31535b0 2024-05-08T05:54:48.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.876+0000 7f90b4c00700 1 --2- 172.21.15.112:0/518812606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31535b0 0x55eba3153a30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f90a80a9250 tx=0x7f90a80a8b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89c93f77cfa8d7eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:48.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.876+0000 7f90af400700 1 --2- 172.21.15.112:0/2003541917 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f90940b2880 0x7f90940b4d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f909c0098a0 tx=0x7f909c0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:48.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.877+0000 7f90a5600700 1 -- 172.21.15.112:0/518812606 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a80a99c0 con 0x55eba31535b0 2024-05-08T05:54:48.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.877+0000 7f90a5600700 1 -- 172.21.15.112:0/518812606 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a80a92c0 con 0x55eba31535b0 2024-05-08T05:54:48.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.877+0000 7f90a5600700 1 -- 172.21.15.112:0/518812606 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a80a9520 con 0x55eba31535b0 2024-05-08T05:54:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.877+0000 7f90c14deb80 1 -- 172.21.15.112:0/518812606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31535b0 msgr2=0x55eba3153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.878+0000 7f90c14deb80 1 --2- 172.21.15.112:0/518812606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31535b0 0x55eba3153a30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f90a80a9250 tx=0x7f90a80a8b60 comp rx=0 tx=0).stop 2024-05-08T05:54:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.878+0000 7f90c14deb80 1 -- 172.21.15.112:0/518812606 shutdown_connections 2024-05-08T05:54:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.878+0000 7f90c14deb80 1 --2- 172.21.15.112:0/518812606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31535b0 0x55eba3153a30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.878+0000 7f90c14deb80 1 --2- 172.21.15.112:0/518812606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba3158990 0x55eba3158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.878+0000 7f90c14deb80 1 --2- 172.21.15.112:0/518812606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31592b0 0x55eba3152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.878+0000 7f90c14deb80 1 -- 172.21.15.112:0/518812606 >> 172.21.15.112:0/518812606 conn(0x55eba3081b10 msgr2=0x55eba314d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.878+0000 7f90c14deb80 1 -- 172.21.15.112:0/518812606 shutdown_connections 2024-05-08T05:54:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.878+0000 7f90c14deb80 1 -- 172.21.15.112:0/518812606 wait complete. 2024-05-08T05:54:48.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.878+0000 7f90c14deb80 1 -- start start 2024-05-08T05:54:48.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.879+0000 7f90c14deb80 1 Processor -- start 2024-05-08T05:54:48.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.879+0000 7f90c14deb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba31535b0 0x55eba315ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.879+0000 7f90c14deb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba3158990 0x55eba315c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.879+0000 7f90c14deb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31592b0 0x55eba315c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.879+0000 7f90c14deb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eba315fd60 con 0x55eba3158990 2024-05-08T05:54:48.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.879+0000 7f90c14deb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55eba300bad0 con 0x55eba31535b0 2024-05-08T05:54:48.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.879+0000 7f90c14deb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55eba315fee0 con 0x55eba31592b0 2024-05-08T05:54:48.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.879+0000 7f90c14deb80 10 client.0 _ll_register_callbacks cb 0x55eba304cc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:48.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.880+0000 7f90af400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba3158990 0x55eba315c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.880+0000 7f90b4c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31592b0 0x55eba315c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.880+0000 7f90b4c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31592b0 0x55eba315c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50484/0 (socket says 172.21.15.112:50484) 2024-05-08T05:54:48.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.880+0000 7f90b4c00700 1 -- 172.21.15.112:0/845909177 learned_addr learned my addr 172.21.15.112:0/845909177 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.880+0000 7f90afe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba31535b0 0x55eba315ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.880+0000 7f90b4c00700 1 -- 172.21.15.112:0/845909177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba31535b0 msgr2=0x55eba315ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.880+0000 7f90b4c00700 1 --2- 172.21.15.112:0/845909177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba31535b0 0x55eba315ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.880+0000 7f90b4c00700 1 -- 172.21.15.112:0/845909177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba3158990 msgr2=0x55eba315c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.880+0000 7f90b4c00700 1 --2- 172.21.15.112:0/845909177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba3158990 0x55eba315c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.880+0000 7f90b4c00700 1 -- 172.21.15.112:0/845909177 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eba306b3d0 con 0x55eba31592b0 2024-05-08T05:54:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.880+0000 7f90b4c00700 1 --2- 172.21.15.112:0/845909177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31592b0 0x55eba315c740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f90a800c6f0 tx=0x7f90a80ad6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d455bce540757522 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.880+0000 7f909a000700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:54:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.880+0000 7f909a000700 1 -- 172.21.15.112:0/845909177 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a80a99c0 con 0x55eba31592b0 2024-05-08T05:54:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.880+0000 7f909a000700 1 -- 172.21.15.112:0/845909177 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f90a80dfde0 con 0x55eba31592b0 2024-05-08T05:54:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.880+0000 7f90c14deb80 1 -- 172.21.15.112:0/845909177 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a80a92c0 con 0x55eba31592b0 2024-05-08T05:54:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.881+0000 7f909a000700 1 -- 172.21.15.112:0/845909177 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f90a80b1c00 con 0x55eba31592b0 2024-05-08T05:54:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.881+0000 7f90c14deb80 10 client.25033 fetch_fsmap learned FSMap version 24 2024-05-08T05:54:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.881+0000 7f90c14deb80 1 -- 172.21.15.112:0/845909177 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55eba315fd60 con 0x55eba31592b0 2024-05-08T05:54:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.881+0000 7f909a000700 1 -- 172.21.15.112:0/845909177 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a80a92c0 con 0x55eba31592b0 2024-05-08T05:54:48.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.881+0000 7f909a000700 1 -- 172.21.15.112:0/845909177 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f90a807a080 con 0x55eba31592b0 2024-05-08T05:54:48.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.881+0000 7f90c14deb80 10 client.25033 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T05:54:48.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.881+0000 7f90c14deb80 10 client.25033 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:54:48.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.881+0000 7f90c14deb80 20 client.25033 populate_metadata read hostname 'smithi112' 2024-05-08T05:54:48.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.881+0000 7f90c14deb80 1 --2- 172.21.15.112:0/845909177 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55eba316c690 0x55eba316db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.881+0000 7f90c14deb80 4 client.25033 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:54:48.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.881+0000 7f90c14deb80 1 -- 172.21.15.112:0/845909177 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eba3120050 con 0x55eba316c690 2024-05-08T05:54:48.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.882+0000 7f90afe00700 1 --2- 172.21.15.112:0/845909177 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55eba316c690 0x55eba316db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.882+0000 7f90afe00700 1 --2- 172.21.15.112:0/845909177 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55eba316c690 0x55eba316db00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aba2b884e17822b1 server_cookie=3d06017db680aa51 in_seq=0 out_seq=0 2024-05-08T05:54:48.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.882+0000 7f909a000700 10 client.25033 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:54:48.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.883+0000 7f909a000700 1 -- 172.21.15.112:0/845909177 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eba3120050 con 0x55eba316c690 2024-05-08T05:54:48.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.883+0000 7f909a000700 10 client.25033 handle_command_reply: tid=0 2024-05-08T05:54:48.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.883+0000 7f90c14deb80 1 client.25033 shutdown 2024-05-08T05:54:48.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.883+0000 7f90c14deb80 1 -- 172.21.15.112:0/845909177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31592b0 msgr2=0x55eba315c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.883+0000 7f90c14deb80 1 --2- 172.21.15.112:0/845909177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31592b0 0x55eba315c740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f90a800c6f0 tx=0x7f90a80ad6e0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.883+0000 7f90c14deb80 1 -- 172.21.15.112:0/845909177 shutdown_connections 2024-05-08T05:54:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.883+0000 7f90c14deb80 1 --2- 172.21.15.112:0/845909177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31592b0 0x55eba315c740 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.883+0000 7f90c14deb80 1 --2- 172.21.15.112:0/845909177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba31535b0 0x55eba315ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.883+0000 7f90c14deb80 1 --2- 172.21.15.112:0/845909177 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55eba316c690 0x55eba316db00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.883+0000 7f90c14deb80 1 --2- 172.21.15.112:0/845909177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba3158990 0x55eba315c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.883+0000 7f90c14deb80 1 -- 172.21.15.112:0/845909177 >> 172.21.15.112:0/845909177 conn(0x55eba3081b10 msgr2=0x55eba3148410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.883+0000 7f909a000700 0 client.25033 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:54:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.884+0000 7f90c14deb80 1 -- 172.21.15.112:0/845909177 shutdown_connections 2024-05-08T05:54:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.884+0000 7f90c14deb80 1 -- 172.21.15.112:0/845909177 wait complete. 2024-05-08T05:54:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.884+0000 7f90c14deb80 20 client.25033 trim_cache size 0 max 16384 2024-05-08T05:54:48.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.906+0000 7f90c14deb80 1 Processor -- start 2024-05-08T05:54:48.907 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:48 smithi112 ceph-mon[30003]: pgmap v426: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:48.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.907+0000 7f90c14deb80 1 -- start start 2024-05-08T05:54:48.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.907+0000 7f90c14deb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba31535b0 0x55eba3152f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.907+0000 7f90c14deb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba3158990 0x55eba3176f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.907+0000 7f90c14deb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31592b0 0x55eba31774a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.907+0000 7f90c14deb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eba316be40 con 0x55eba31592b0 2024-05-08T05:54:48.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.907+0000 7f90c14deb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55eba316c510 con 0x55eba31535b0 2024-05-08T05:54:48.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.907+0000 7f90c14deb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55eba316cbe0 con 0x55eba3158990 2024-05-08T05:54:48.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.907+0000 7f90afe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba31535b0 0x55eba3152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.907+0000 7f90b4c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31592b0 0x55eba31774a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.907+0000 7f90afe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba31535b0 0x55eba3152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:42192/0 (socket says 172.21.15.112:42192) 2024-05-08T05:54:48.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.907+0000 7f90afe00700 1 -- 172.21.15.112:0/3032142125 learned_addr learned my addr 172.21.15.112:0/3032142125 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:48.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.908+0000 7f90af400700 1 --2- 172.21.15.112:0/3032142125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba3158990 0x55eba3176f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.908+0000 7f90afe00700 1 -- 172.21.15.112:0/3032142125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba3158990 msgr2=0x55eba3176f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:48.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.908+0000 7f90afe00700 1 --2- 172.21.15.112:0/3032142125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba3158990 0x55eba3176f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.908+0000 7f90afe00700 1 -- 172.21.15.112:0/3032142125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31592b0 msgr2=0x55eba31774a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:48.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.908+0000 7f90afe00700 1 --2- 172.21.15.112:0/3032142125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31592b0 0x55eba31774a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.908+0000 7f90afe00700 1 -- 172.21.15.112:0/3032142125 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eba314eb50 con 0x55eba31535b0 2024-05-08T05:54:48.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.908+0000 7f90afe00700 1 --2- 172.21.15.112:0/3032142125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba31535b0 0x55eba3152f90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009990 tx=0x7f90a000b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d0f9e63649112b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:48.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.909+0000 7f90a4c00700 1 -- 172.21.15.112:0/3032142125 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a000b920 con 0x55eba31535b0 2024-05-08T05:54:48.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.909+0000 7f90a4c00700 1 -- 172.21.15.112:0/3032142125 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a00056a0 con 0x55eba31535b0 2024-05-08T05:54:48.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.909+0000 7f90a4c00700 1 -- 172.21.15.112:0/3032142125 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0028160 con 0x55eba31535b0 2024-05-08T05:54:48.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.910+0000 7f90c14deb80 1 -- 172.21.15.112:0/3032142125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba31535b0 msgr2=0x55eba3152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:48.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.910+0000 7f90c14deb80 1 --2- 172.21.15.112:0/3032142125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba31535b0 0x55eba3152f90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009990 tx=0x7f90a000b260 comp rx=0 tx=0).stop 2024-05-08T05:54:48.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.910+0000 7f90c14deb80 1 -- 172.21.15.112:0/3032142125 shutdown_connections 2024-05-08T05:54:48.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.910+0000 7f90c14deb80 1 --2- 172.21.15.112:0/3032142125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31592b0 0x55eba31774a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.910+0000 7f90c14deb80 1 --2- 172.21.15.112:0/3032142125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba31535b0 0x55eba3152f90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.910+0000 7f90c14deb80 1 --2- 172.21.15.112:0/3032142125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba3158990 0x55eba3176f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.910+0000 7f90c14deb80 1 -- 172.21.15.112:0/3032142125 >> 172.21.15.112:0/3032142125 conn(0x55eba3081b10 msgr2=0x55eba315aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:48.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.910+0000 7f90c14deb80 1 -- 172.21.15.112:0/3032142125 shutdown_connections 2024-05-08T05:54:48.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.910+0000 7f90c14deb80 1 -- 172.21.15.112:0/3032142125 wait complete. 2024-05-08T05:54:48.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.911+0000 7f90c14deb80 1 -- start start 2024-05-08T05:54:48.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.911+0000 7f90c14deb80 1 Processor -- start 2024-05-08T05:54:48.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.912+0000 7f90c14deb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31535b0 0x55eba3176f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.912+0000 7f90c14deb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba3158990 0x55eba31774c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.912+0000 7f90c14deb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31592b0 0x55eba3171f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.912+0000 7f90c14deb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55eba316d980 con 0x55eba31535b0 2024-05-08T05:54:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.912+0000 7f90c14deb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55eba316d2b0 con 0x55eba3158990 2024-05-08T05:54:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.912+0000 7f90c14deb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55eba316e050 con 0x55eba31592b0 2024-05-08T05:54:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.912+0000 7f90c14deb80 10 client.0 _ll_register_callbacks cb 0x55eba304cc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.912+0000 7f90af400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba3158990 0x55eba31774c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.912+0000 7f90b4c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31592b0 0x55eba3171f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.912+0000 7f90afe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31535b0 0x55eba3176f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.912+0000 7f90af400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba3158990 0x55eba31774c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:42196/0 (socket says 172.21.15.112:42196) 2024-05-08T05:54:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.912+0000 7f90af400700 1 -- 172.21.15.112:0/551721056 learned_addr learned my addr 172.21.15.112:0/551721056 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.912+0000 7f90af400700 1 -- 172.21.15.112:0/551721056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31592b0 msgr2=0x55eba3171f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.912+0000 7f90af400700 1 --2- 172.21.15.112:0/551721056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31592b0 0x55eba3171f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.912+0000 7f90af400700 1 -- 172.21.15.112:0/551721056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31535b0 msgr2=0x55eba3176f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.913+0000 7f90af400700 1 --2- 172.21.15.112:0/551721056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31535b0 0x55eba3176f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.913+0000 7f90af400700 1 -- 172.21.15.112:0/551721056 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eba3172470 con 0x55eba3158990 2024-05-08T05:54:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.913+0000 7f90afe00700 1 --2- 172.21.15.112:0/551721056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31535b0 0x55eba3176f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:48.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.913+0000 7f90af400700 1 --2- 172.21.15.112:0/551721056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba3158990 0x55eba31774c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f909c0096d0 tx=0x7f909c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d786d5d24c7dc01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:48.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.913+0000 7f909a000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:54:48.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.913+0000 7f909a000700 1 -- 172.21.15.112:0/551721056 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c01fda0 con 0x55eba3158990 2024-05-08T05:54:48.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.913+0000 7f909a000700 1 -- 172.21.15.112:0/551721056 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f909c02a400 con 0x55eba3158990 2024-05-08T05:54:48.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.913+0000 7f90c14deb80 1 -- 172.21.15.112:0/551721056 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a00056a0 con 0x55eba3158990 2024-05-08T05:54:48.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.914+0000 7f909a000700 1 -- 172.21.15.112:0/551721056 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f909c02a600 con 0x55eba3158990 2024-05-08T05:54:48.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.914+0000 7f90c14deb80 10 client.25016 fetch_fsmap learned FSMap version 24 2024-05-08T05:54:48.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.914+0000 7f90c14deb80 1 -- 172.21.15.112:0/551721056 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55eba31d8330 con 0x55eba3158990 2024-05-08T05:54:48.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.914+0000 7f909a000700 1 -- 172.21.15.112:0/551721056 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a00056a0 con 0x55eba3158990 2024-05-08T05:54:48.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.914+0000 7f909a000700 1 -- 172.21.15.112:0/551721056 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f909c01c040 con 0x55eba3158990 2024-05-08T05:54:48.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.914+0000 7f90c14deb80 10 client.25016 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T05:54:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.914+0000 7f90c14deb80 10 client.25016 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:54:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.914+0000 7f90c14deb80 20 client.25016 populate_metadata read hostname 'smithi112' 2024-05-08T05:54:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.914+0000 7f90c14deb80 1 --2- 172.21.15.112:0/551721056 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55eba31dd000 0x55eba31dbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.914+0000 7f90c14deb80 4 client.25016 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:54:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.914+0000 7f90c14deb80 1 -- 172.21.15.112:0/551721056 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "status"}) -- 0x55eba30b33b0 con 0x55eba31dd000 2024-05-08T05:54:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.915+0000 7f90afe00700 1 --2- 172.21.15.112:0/551721056 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55eba31dd000 0x55eba31dbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.915+0000 7f90afe00700 1 --2- 172.21.15.112:0/551721056 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55eba31dd000 0x55eba31dbd10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adc9df28098eb904 server_cookie=22fdaac36bb5cc8e in_seq=0 out_seq=0 2024-05-08T05:54:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.915+0000 7f909a000700 10 client.25016 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:54:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.916+0000 7f909a000700 1 -- 172.21.15.112:0/551721056 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55eba30b33b0 con 0x55eba31dd000 2024-05-08T05:54:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.916+0000 7f909a000700 10 client.25016 handle_command_reply: tid=0 2024-05-08T05:54:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.916+0000 7f90c14deb80 1 client.25016 shutdown 2024-05-08T05:54:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.916+0000 7f90c14deb80 1 -- 172.21.15.112:0/551721056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba3158990 msgr2=0x55eba31774c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.916+0000 7f90c14deb80 1 --2- 172.21.15.112:0/551721056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba3158990 0x55eba31774c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f909c0096d0 tx=0x7f909c004750 comp rx=0 tx=0).stop 2024-05-08T05:54:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.916+0000 7f90c14deb80 1 -- 172.21.15.112:0/551721056 shutdown_connections 2024-05-08T05:54:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.916+0000 7f90c14deb80 1 --2- 172.21.15.112:0/551721056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba31592b0 0x55eba3171f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.916+0000 7f90c14deb80 1 --2- 172.21.15.112:0/551721056 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55eba31dd000 0x55eba31dbd10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.916+0000 7f90c14deb80 1 --2- 172.21.15.112:0/551721056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55eba31535b0 0x55eba3176f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.916+0000 7f90c14deb80 1 --2- 172.21.15.112:0/551721056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eba3158990 0x55eba31774c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.916+0000 7f90c14deb80 1 -- 172.21.15.112:0/551721056 >> 172.21.15.112:0/551721056 conn(0x55eba3081b10 msgr2=0x55eba3159690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.916+0000 7f909a000700 0 client.25016 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:54:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.917+0000 7f90c14deb80 1 -- 172.21.15.112:0/551721056 shutdown_connections 2024-05-08T05:54:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.917+0000 7f90c14deb80 1 -- 172.21.15.112:0/551721056 wait complete. 2024-05-08T05:54:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.917+0000 7f90c14deb80 20 client.25016 trim_cache size 0 max 16384 2024-05-08T05:54:48.919 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:54:48.919 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "7a07ff60-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:54:48.919 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-05-08T05:54:48.920 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24529, 2024-05-08T05:54:48.920 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-05-08T05:54:48.920 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-05-08T05:54:48.920 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-05-08T05:54:48.920 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 221.59266630499999, 2024-05-08T05:54:48.920 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 24, 2024-05-08T05:54:48.920 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 84, 2024-05-08T05:54:48.920 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 80, 2024-05-08T05:54:48.920 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 245.63896924100001 2024-05-08T05:54:48.920 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:54:48.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.918+0000 7f90b5600700 1 -- 172.21.15.112:0/2003541917 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f90940b2880 msgr2=0x7f90940b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:48.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.918+0000 7f90b5600700 1 --2- 172.21.15.112:0/2003541917 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f90940b2880 0x7f90940b4d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f909c0098a0 tx=0x7f909c0095c0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.918+0000 7f90b5600700 1 -- 172.21.15.112:0/2003541917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b010e8b0 msgr2=0x7f90b01a49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:48.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.918+0000 7f90b5600700 1 --2- 172.21.15.112:0/2003541917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b010e8b0 0x7f90b01a49b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f90a800efc0 tx=0x7f90a800c3a0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.918+0000 7f90b5600700 1 -- 172.21.15.112:0/2003541917 shutdown_connections 2024-05-08T05:54:48.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.918+0000 7f90b5600700 1 --2- 172.21.15.112:0/2003541917 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f90940b2880 0x7f90940b4d40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.918+0000 7f90b5600700 1 --2- 172.21.15.112:0/2003541917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90b010e8b0 0x7f90b01a49b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.918+0000 7f90b5600700 1 --2- 172.21.15.112:0/2003541917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b01099e0 0x7f90b01a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.918+0000 7f90b5600700 1 --2- 172.21.15.112:0/2003541917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90b010a3a0 0x7f90b01a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:48.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.918+0000 7f90b5600700 1 -- 172.21.15.112:0/2003541917 >> 172.21.15.112:0/2003541917 conn(0x7f90b0079230 msgr2=0x7f90b0079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:48.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.918+0000 7f90b5600700 1 -- 172.21.15.112:0/2003541917 shutdown_connections 2024-05-08T05:54:48.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:48.918+0000 7f90b5600700 1 -- 172.21.15.112:0/2003541917 wait complete. 2024-05-08T05:54:48.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:48 smithi150 ceph-mon[34315]: pgmap v426: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:49.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:48 smithi162 ceph-mon[34800]: pgmap v426: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:49.781 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T05:54:49.792 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T05:54:49.792 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T05:54:49.793 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T05:54:49.793 INFO:teuthology.run_tasks:Running task kclient... 2024-05-08T05:54:49.823 INFO:tasks.kclient:Mounting kernel clients... 2024-05-08T05:54:49.823 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-08T05:54:49.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:54:49.823 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-05-08T05:54:49.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:54:49.882 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-05-08T05:54:49.973 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-05-08T05:54:49.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:54:49.977 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-08T05:54:49.978 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T05:54:49.978 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:54:49.978 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T05:54:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:50.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:50 smithi112 ceph-mon[30003]: pgmap v427: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:50.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:50 smithi150 ceph-mon[34315]: pgmap v427: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:51.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:50 smithi162 ceph-mon[34800]: pgmap v427: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:52.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.611+0000 7f2f24600700 1 -- 172.21.15.112:0/1858730221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f1c10fb60 msgr2=0x7f2f1c10ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:52.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.611+0000 7f2f24600700 1 --2- 172.21.15.112:0/1858730221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f1c10fb60 0x7f2f1c10ff40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009990 tx=0x7f2f10009ca0 comp rx=0 tx=0).stop 2024-05-08T05:54:52.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.611+0000 7f2f24600700 1 -- 172.21.15.112:0/1858730221 shutdown_connections 2024-05-08T05:54:52.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.611+0000 7f2f24600700 1 --2- 172.21.15.112:0/1858730221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c10a490 0x7f2f1c1169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:52.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.611+0000 7f2f24600700 1 --2- 172.21.15.112:0/1858730221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f1c10fb60 0x7f2f1c10ff40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:52.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.611+0000 7f2f24600700 1 --2- 172.21.15.112:0/1858730221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f1c109ad0 0x7f2f1c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:52.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.611+0000 7f2f24600700 1 -- 172.21.15.112:0/1858730221 >> 172.21.15.112:0/1858730221 conn(0x7f2f1c076c70 msgr2=0x7f2f1c1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:52.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.611+0000 7f2f24600700 1 -- 172.21.15.112:0/1858730221 shutdown_connections 2024-05-08T05:54:52.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.611+0000 7f2f24600700 1 -- 172.21.15.112:0/1858730221 wait complete. 2024-05-08T05:54:52.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.612+0000 7f2f24600700 1 Processor -- start 2024-05-08T05:54:52.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.613+0000 7f2f24600700 1 -- start start 2024-05-08T05:54:52.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.613+0000 7f2f24600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f1c109ad0 0x7f2f1c1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:52.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.613+0000 7f2f24600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c10a490 0x7f2f1c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:52.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.613+0000 7f2f24600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f1c10fb60 0x7f2f1c1a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:52.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.613+0000 7f2f24600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c119cd0 con 0x7f2f1c109ad0 2024-05-08T05:54:52.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.613+0000 7f2f24600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c119b50 con 0x7f2f1c10fb60 2024-05-08T05:54:52.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.613+0000 7f2f24600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c119e50 con 0x7f2f1c10a490 2024-05-08T05:54:52.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.613+0000 7f2f21c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f1c109ad0 0x7f2f1c1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:52.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.613+0000 7f2f21c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f1c109ad0 0x7f2f1c1a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40336/0 (socket says 172.21.15.112:40336) 2024-05-08T05:54:52.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.613+0000 7f2f21c00700 1 -- 172.21.15.112:0/2674764649 learned_addr learned my addr 172.21.15.112:0/2674764649 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:52.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.613+0000 7f2f21200700 1 --2- 172.21.15.112:0/2674764649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c10a490 0x7f2f1c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:52.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.613+0000 7f2f22600700 1 --2- 172.21.15.112:0/2674764649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f1c10fb60 0x7f2f1c1a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:52.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.614+0000 7f2f21200700 1 -- 172.21.15.112:0/2674764649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f1c10fb60 msgr2=0x7f2f1c1a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:52.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.614+0000 7f2f21200700 1 --2- 172.21.15.112:0/2674764649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f1c10fb60 0x7f2f1c1a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:52.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.614+0000 7f2f21200700 1 -- 172.21.15.112:0/2674764649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f1c109ad0 msgr2=0x7f2f1c1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:52.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.614+0000 7f2f21200700 1 --2- 172.21.15.112:0/2674764649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f1c109ad0 0x7f2f1c1a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:52.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.614+0000 7f2f21200700 1 -- 172.21.15.112:0/2674764649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1c1a5160 con 0x7f2f1c10a490 2024-05-08T05:54:52.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.614+0000 7f2f21c00700 1 --2- 172.21.15.112:0/2674764649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f1c109ad0 0x7f2f1c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:52.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.614+0000 7f2f21200700 1 --2- 172.21.15.112:0/2674764649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c10a490 0x7f2f1c1a06e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800e9f0 tx=0x7f2f0800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:52.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.614+0000 7f2f1a000700 1 -- 172.21.15.112:0/2674764649 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0800c9c0 con 0x7f2f1c10a490 2024-05-08T05:54:52.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.614+0000 7f2f24600700 1 -- 172.21.15.112:0/2674764649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f1c1a5450 con 0x7f2f1c10a490 2024-05-08T05:54:52.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.615+0000 7f2f1a000700 1 -- 172.21.15.112:0/2674764649 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f0800cb60 con 0x7f2f1c10a490 2024-05-08T05:54:52.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.615+0000 7f2f1a000700 1 -- 172.21.15.112:0/2674764649 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08018680 con 0x7f2f1c10a490 2024-05-08T05:54:52.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.615+0000 7f2f24600700 1 -- 172.21.15.112:0/2674764649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f1c1accd0 con 0x7f2f1c10a490 2024-05-08T05:54:52.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.616+0000 7f2f24600700 1 -- 172.21.15.112:0/2674764649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f1c06a190 con 0x7f2f1c10a490 2024-05-08T05:54:52.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.618+0000 7f2f1a000700 1 -- 172.21.15.112:0/2674764649 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2f08010430 con 0x7f2f1c10a490 2024-05-08T05:54:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.620+0000 7f2f1a000700 1 --2- 172.21.15.112:0/2674764649 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2f040b2900 0x7f2f040b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.620+0000 7f2f1a000700 1 -- 172.21.15.112:0/2674764649 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f2f080e26f0 con 0x7f2f1c10a490 2024-05-08T05:54:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.620+0000 7f2f21c00700 1 --2- 172.21.15.112:0/2674764649 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2f040b2900 0x7f2f040b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:52.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.621+0000 7f2f21c00700 1 --2- 172.21.15.112:0/2674764649 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2f040b2900 0x7f2f040b4dc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009750 tx=0x7f2f1001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:52.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.622+0000 7f2f1a000700 1 -- 172.21.15.112:0/2674764649 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f080aa380 con 0x7f2f1c10a490 2024-05-08T05:54:52.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:52 smithi112 ceph-mon[30003]: pgmap v428: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:52.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.904+0000 7f2f24600700 1 -- 172.21.15.112:0/2674764649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2f1c02e510 con 0x7f2f1c10a490 2024-05-08T05:54:52.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.905+0000 7f2f1a000700 1 -- 172.21.15.112:0/2674764649 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f2f080b2400 con 0x7f2f1c10a490 2024-05-08T05:54:52.905 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:54:52.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.908+0000 7f2f24600700 1 -- 172.21.15.112:0/2674764649 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2f040b2900 msgr2=0x7f2f040b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:52.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.908+0000 7f2f24600700 1 --2- 172.21.15.112:0/2674764649 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2f040b2900 0x7f2f040b4dc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009750 tx=0x7f2f1001a040 comp rx=0 tx=0).stop 2024-05-08T05:54:52.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.908+0000 7f2f24600700 1 -- 172.21.15.112:0/2674764649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c10a490 msgr2=0x7f2f1c1a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:52.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.908+0000 7f2f24600700 1 --2- 172.21.15.112:0/2674764649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c10a490 0x7f2f1c1a06e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800e9f0 tx=0x7f2f0800ed00 comp rx=0 tx=0).stop 2024-05-08T05:54:52.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.908+0000 7f2f24600700 1 -- 172.21.15.112:0/2674764649 shutdown_connections 2024-05-08T05:54:52.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.908+0000 7f2f24600700 1 --2- 172.21.15.112:0/2674764649 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2f040b2900 0x7f2f040b4dc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:52.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.908+0000 7f2f24600700 1 --2- 172.21.15.112:0/2674764649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f1c10fb60 0x7f2f1c1a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:52.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.908+0000 7f2f24600700 1 --2- 172.21.15.112:0/2674764649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f1c109ad0 0x7f2f1c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:52.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.908+0000 7f2f24600700 1 --2- 172.21.15.112:0/2674764649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f1c10a490 0x7f2f1c1a06e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:52.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.908+0000 7f2f24600700 1 -- 172.21.15.112:0/2674764649 >> 172.21.15.112:0/2674764649 conn(0x7f2f1c076c70 msgr2=0x7f2f1c077270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:52.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.908+0000 7f2f24600700 1 -- 172.21.15.112:0/2674764649 shutdown_connections 2024-05-08T05:54:52.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:52.908+0000 7f2f24600700 1 -- 172.21.15.112:0/2674764649 wait complete. 2024-05-08T05:54:52.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:52 smithi150 ceph-mon[34315]: pgmap v428: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:53.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:52 smithi162 ceph-mon[34800]: pgmap v428: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:53.805 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T05:54:53.805 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:54:53.805 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-05-08T05:54:53.805 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:54:53.805 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:54:53.805 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T05:54:53.805 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T05:54:53.805 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:54:53.805 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:54:53.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:54:53.806 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-05-08T05:54:53.862 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:54:53.862 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:54:53.862 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:54:53.862 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:54:53.862 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-05-08T05:54:53.862 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:54:53.862 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:54:53.862 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:54:53.863 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-05-08T05:54:53.863 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:54:53.863 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-05-08T05:54:53.863 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-05-08T05:54:53.863 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:54:53.863 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:54:53.863 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:54:53.863 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:54:53.863 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T05:54:53.863 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 41780sec preferred_lft 41780sec 2024-05-08T05:54:53.863 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2024-05-08T05:54:53.863 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:54:53.864 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T05:54:53.866 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:54:53.866 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-05-08T05:54:53.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-05-08T05:54:53.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-05-08T05:54:53.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-05-08T05:54:53.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T05:54:53.866 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-05-08T05:54:53.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:53 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/2674764649' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T05:54:53.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:54:53.999 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:53 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2674764649' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T05:54:54.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:53 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/2674764649' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T05:54:54.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:54:54.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:54:54.367 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:54:54.435 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-05-08T05:54:54.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:54:54.439 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-05-08T05:54:54.513 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-05-08T05:54:54.513 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:54:54.513 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T05:54:54.513 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T05:54:54.513 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:54:54.514 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:54:54.514 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-05-08T05:54:54.515 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T05:54:54.515 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T05:54:54.515 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T05:54:54.515 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-05-08T05:54:54.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:54:54.895 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:54 smithi112 ceph-mon[30003]: pgmap v429: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:54.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:54 smithi150 ceph-mon[34315]: pgmap v429: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:55.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:54 smithi162 ceph-mon[34800]: pgmap v429: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:54:55.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:54:55.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:54:55.265 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-05-08T05:54:55.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:54:55.320 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-05-08T05:54:55.374 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:54:55.374 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-05-08T05:54:55.374 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:54:55.374 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T05:54:55.374 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-05-08T05:54:55.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:54:55.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:54:55.561 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T05:54:55.562 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:54:55.562 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-05-08T05:54:55.562 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T05:54:55.562 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T05:54:55.562 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T05:54:55.562 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T05:54:55.562 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T05:54:55.562 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-05-08T05:54:55.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:54:55.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:54:55.850 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:54:55.850 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-05-08T05:54:55.850 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-05-08T05:54:55.850 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T05:54:55.850 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-05-08T05:54:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:54:56.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:54:56.045 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:54:56.046 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:54:56.092 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:54:56.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:54:56.092 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:54:56.185 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-08T05:54:56.185 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-08T05:54:56.185 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/f10a0d2e-83c8-40e2-b411-c0dc32989241 2024-05-08T05:54:56.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:54:56.185 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/f10a0d2e-83c8-40e2-b411-c0dc32989241 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-08T05:54:56.330 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-08T05:54:56.330 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-05-08T05:54:56.772 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:56 smithi112 ceph-mon[30003]: pgmap v430: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:56.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:56 smithi150 ceph-mon[34315]: pgmap v430: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:57.056 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-08T05:54:57.057 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-05-08T05:54:57.057 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.150:3300,172.21.15.162:3300" 2024-05-08T05:54:57.057 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with new device syntax: 0@7a07ff60-0cfd-11ef-bc97-c7b262605968.cephfs=/volumes/qa/sv_1/f10a0d2e-83c8-40e2-b411-c0dc32989241 2024-05-08T05:54:57.057 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.112:3300/172.21.15.150:3300/172.21.15.162:3300" will pass to kernel 2024-05-08T05:54:57.066 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:54:57.066 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:54:57.066 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:54:57.066 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:54:57.066 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:54:57.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:56 smithi162 ceph-mon[34800]: pgmap v430: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:54:57.071 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:54:57.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:54:57.071 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:54:57.173 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:54:57.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:54:57.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:54:57.295 INFO:teuthology.orchestra.run.smithi112.stdout:client25025 2024-05-08T05:54:57.295 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:54:57.295 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:54:57.345 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:54:57.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:54:57.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:54:57.509 INFO:teuthology.orchestra.run.smithi112.stdout:client25025 2024-05-08T05:54:57.510 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:54:57.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:54:57.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:54:57.628 INFO:teuthology.orchestra.run.smithi112.stdout:client25025 2024-05-08T05:54:57.628 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968.client25025/status') 2024-05-08T05:54:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:54:57.761 INFO:teuthology.orchestra.run.smithi112.stdout:instance: client.25025 (3)172.21.15.112:0/164743798 2024-05-08T05:54:57.761 INFO:teuthology.orchestra.run.smithi112.stdout:blocklisted: false 2024-05-08T05:54:57.761 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records in 2024-05-08T05:54:57.762 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records out 2024-05-08T05:54:57.762 INFO:teuthology.orchestra.run.smithi112.stderr:71 bytes copied, 9.6254e-05 s, 738 kB/s 2024-05-08T05:54:57.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:54:57.774 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T05:54:57.784 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T05:54:57.784 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-05-08T05:54:57.785 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-08T05:54:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.982+0000 7f211de00640 1 -- 172.21.15.112:0/2005785788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2118108510 msgr2=0x7f211810a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.982+0000 7f211de00640 1 --2- 172.21.15.112:0/2005785788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2118108510 0x7f211810a960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2110009ca0 tx=0x7f211002a8c0 comp rx=0 tx=0).stop 2024-05-08T05:54:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.982+0000 7f211de00640 1 -- 172.21.15.112:0/2005785788 shutdown_connections 2024-05-08T05:54:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.982+0000 7f211de00640 1 --2- 172.21.15.112:0/2005785788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2118108510 0x7f211810a960 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.982+0000 7f211de00640 1 --2- 172.21.15.112:0/2005785788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2118105be0 0x7f2118107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.982+0000 7f211de00640 1 --2- 172.21.15.112:0/2005785788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21181032b0 0x7f21181056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.982+0000 7f211de00640 1 -- 172.21.15.112:0/2005785788 >> 172.21.15.112:0/2005785788 conn(0x7f21180fcca0 msgr2=0x7f21180ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.982+0000 7f211de00640 1 -- 172.21.15.112:0/2005785788 shutdown_connections 2024-05-08T05:54:57.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.983+0000 7f211de00640 1 -- 172.21.15.112:0/2005785788 wait complete. 2024-05-08T05:54:57.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.983+0000 7f211de00640 1 Processor -- start 2024-05-08T05:54:57.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.984+0000 7f211de00640 1 -- start start 2024-05-08T05:54:57.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.984+0000 7f211de00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21181032b0 0x7f2118102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:57.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.984+0000 7f211de00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2118105be0 0x7f2118101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:57.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.984+0000 7f211de00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2118108510 0x7f21181017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:57.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.984+0000 7f211de00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f211810e380 con 0x7f2118108510 2024-05-08T05:54:57.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.984+0000 7f211de00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f211810e200 con 0x7f21181032b0 2024-05-08T05:54:57.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.984+0000 7f211de00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f211810e500 con 0x7f2118105be0 2024-05-08T05:54:57.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.984+0000 7f211d400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2118108510 0x7f21181017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:57.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.984+0000 7f211d400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2118108510 0x7f21181017a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43554/0 (socket says 172.21.15.112:43554) 2024-05-08T05:54:57.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.984+0000 7f211d400640 1 -- 172.21.15.112:0/261654489 learned_addr learned my addr 172.21.15.112:0/261654489 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:57.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.984+0000 7f2117e00640 1 --2- 172.21.15.112:0/261654489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2118105be0 0x7f2118101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:57.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.984+0000 7f211ca00640 1 --2- 172.21.15.112:0/261654489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21181032b0 0x7f2118102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:57.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.985+0000 7f211d400640 1 -- 172.21.15.112:0/261654489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2118105be0 msgr2=0x7f2118101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:57.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.985+0000 7f211d400640 1 --2- 172.21.15.112:0/261654489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2118105be0 0x7f2118101260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:57.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.985+0000 7f211d400640 1 -- 172.21.15.112:0/261654489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21181032b0 msgr2=0x7f2118102c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:57.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.985+0000 7f211d400640 1 --2- 172.21.15.112:0/261654489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21181032b0 0x7f2118102c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:57.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.985+0000 7f211d400640 1 -- 172.21.15.112:0/261654489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2118102030 con 0x7f2118108510 2024-05-08T05:54:57.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.985+0000 7f211d400640 1 --2- 172.21.15.112:0/261654489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2118108510 0x7f21181017a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f21100097b0 tx=0x7f2110031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:57.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.986+0000 7f2115600640 1 -- 172.21.15.112:0/261654489 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110003b70 con 0x7f2118108510 2024-05-08T05:54:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.986+0000 7f2115600640 1 -- 172.21.15.112:0/261654489 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2110003d10 con 0x7f2118108510 2024-05-08T05:54:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.986+0000 7f211de00640 1 -- 172.21.15.112:0/261654489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21181a2b40 con 0x7f2118108510 2024-05-08T05:54:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.986+0000 7f2115600640 1 -- 172.21.15.112:0/261654489 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211002fc10 con 0x7f2118108510 2024-05-08T05:54:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.986+0000 7f211de00640 1 -- 172.21.15.112:0/261654489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21181a3090 con 0x7f2118108510 2024-05-08T05:54:57.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.987+0000 7f211de00640 1 -- 172.21.15.112:0/261654489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21180fe790 con 0x7f2118108510 2024-05-08T05:54:57.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.989+0000 7f2115600640 1 -- 172.21.15.112:0/261654489 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2110004040 con 0x7f2118108510 2024-05-08T05:54:57.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.991+0000 7f2115600640 1 --2- 172.21.15.112:0/261654489 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f21000b27e0 0x7f21000b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:57.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.991+0000 7f2115600640 1 -- 172.21.15.112:0/261654489 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f2110103800 con 0x7f2118108510 2024-05-08T05:54:57.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.991+0000 7f211ca00640 1 --2- 172.21.15.112:0/261654489 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f21000b27e0 0x7f21000b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:57.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.992+0000 7f211ca00640 1 --2- 172.21.15.112:0/261654489 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f21000b27e0 0x7f21000b4ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f210c009bd0 tx=0x7f210c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:57.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:57.996+0000 7f2115600640 1 -- 172.21.15.112:0/261654489 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f21100cb490 con 0x7f2118108510 2024-05-08T05:54:58.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.246+0000 7f211de00640 1 -- 172.21.15.112:0/261654489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f21180635b0 con 0x7f2118108510 2024-05-08T05:54:58.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.666+0000 7f2115600640 1 -- 172.21.15.112:0/261654489 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f21100cad90 con 0x7f2118108510 2024-05-08T05:54:58.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.671+0000 7f211de00640 1 -- 172.21.15.112:0/261654489 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f21000b27e0 msgr2=0x7f21000b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:58.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.671+0000 7f211de00640 1 --2- 172.21.15.112:0/261654489 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f21000b27e0 0x7f21000b4ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f210c009bd0 tx=0x7f210c005f90 comp rx=0 tx=0).stop 2024-05-08T05:54:58.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.671+0000 7f211de00640 1 -- 172.21.15.112:0/261654489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2118108510 msgr2=0x7f21181017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:58.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.671+0000 7f211de00640 1 --2- 172.21.15.112:0/261654489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2118108510 0x7f21181017a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f21100097b0 tx=0x7f2110031770 comp rx=0 tx=0).stop 2024-05-08T05:54:58.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.672+0000 7f211de00640 1 -- 172.21.15.112:0/261654489 shutdown_connections 2024-05-08T05:54:58.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.672+0000 7f211de00640 1 --2- 172.21.15.112:0/261654489 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f21000b27e0 0x7f21000b4ca0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:58.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.672+0000 7f211de00640 1 --2- 172.21.15.112:0/261654489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2118108510 0x7f21181017a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:58.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.672+0000 7f211de00640 1 --2- 172.21.15.112:0/261654489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2118105be0 0x7f2118101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:58.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.672+0000 7f211de00640 1 --2- 172.21.15.112:0/261654489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21181032b0 0x7f2118102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:58.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.672+0000 7f211de00640 1 -- 172.21.15.112:0/261654489 >> 172.21.15.112:0/261654489 conn(0x7f21180fcca0 msgr2=0x7f2118103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:58.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.672+0000 7f211de00640 1 -- 172.21.15.112:0/261654489 shutdown_connections 2024-05-08T05:54:58.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.672+0000 7f211de00640 1 -- 172.21.15.112:0/261654489 wait complete. 2024-05-08T05:54:58.695 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-08T05:54:58.705 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-08T05:54:58.722 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:54:58.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.925+0000 7f6bc9c00640 1 -- 172.21.15.112:0/189684013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc4105c20 msgr2=0x7f6bc41060a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:58.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.925+0000 7f6bc9c00640 1 --2- 172.21.15.112:0/189684013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc4105c20 0x7f6bc41060a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009900 tx=0x7f6bac02a410 comp rx=0 tx=0).stop 2024-05-08T05:54:58.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.925+0000 7f6bc9c00640 1 -- 172.21.15.112:0/189684013 shutdown_connections 2024-05-08T05:54:58.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.925+0000 7f6bc9c00640 1 --2- 172.21.15.112:0/189684013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bc41065e0 0x7f6bc410ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:58.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.925+0000 7f6bc9c00640 1 --2- 172.21.15.112:0/189684013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc4105c20 0x7f6bc41060a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:58.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.925+0000 7f6bc9c00640 1 --2- 172.21.15.112:0/189684013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bc4104a30 0x7f6bc4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:58.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.925+0000 7f6bc9c00640 1 -- 172.21.15.112:0/189684013 >> 172.21.15.112:0/189684013 conn(0x7f6bc40fffc0 msgr2=0x7f6bc41023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:58.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.926+0000 7f6bc9c00640 1 -- 172.21.15.112:0/189684013 shutdown_connections 2024-05-08T05:54:58.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.926+0000 7f6bc9c00640 1 -- 172.21.15.112:0/189684013 wait complete. 2024-05-08T05:54:58.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.926+0000 7f6bc9c00640 1 Processor -- start 2024-05-08T05:54:58.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.927+0000 7f6bc9c00640 1 -- start start 2024-05-08T05:54:58.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.927+0000 7f6bc9c00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc4104a30 0x7f6bc419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:58.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.927+0000 7f6bc9c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bc4105c20 0x7f6bc419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:58.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.927+0000 7f6bc9c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bc41065e0 0x7f6bc41a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:58.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.927+0000 7f6bc9c00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41105b0 con 0x7f6bc4104a30 2024-05-08T05:54:58.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.927+0000 7f6bc9c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4110430 con 0x7f6bc4105c20 2024-05-08T05:54:58.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.927+0000 7f6bc9c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4110730 con 0x7f6bc41065e0 2024-05-08T05:54:58.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.927+0000 7f6bc3e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc4104a30 0x7f6bc419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:58.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.927+0000 7f6bc3e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc4104a30 0x7f6bc419ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43582/0 (socket says 172.21.15.112:43582) 2024-05-08T05:54:58.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.927+0000 7f6bc3e00640 1 -- 172.21.15.112:0/433917646 learned_addr learned my addr 172.21.15.112:0/433917646 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:58.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.927+0000 7f6bc9200640 1 --2- 172.21.15.112:0/433917646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bc41065e0 0x7f6bc41a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:58.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.927+0000 7f6bbb400640 1 --2- 172.21.15.112:0/433917646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bc4105c20 0x7f6bc419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:58.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.928+0000 7f6bc3e00640 1 -- 172.21.15.112:0/433917646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bc41065e0 msgr2=0x7f6bc41a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:58.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.928+0000 7f6bc3e00640 1 --2- 172.21.15.112:0/433917646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bc41065e0 0x7f6bc41a21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:58.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.928+0000 7f6bc3e00640 1 -- 172.21.15.112:0/433917646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bc4105c20 msgr2=0x7f6bc419b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:58.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.928+0000 7f6bc3e00640 1 --2- 172.21.15.112:0/433917646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bc4105c20 0x7f6bc419b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:58.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.928+0000 7f6bc3e00640 1 -- 172.21.15.112:0/433917646 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc41a2850 con 0x7f6bc4104a30 2024-05-08T05:54:58.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.928+0000 7f6bc3e00640 1 --2- 172.21.15.112:0/433917646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc4104a30 0x7f6bc419ac40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400e9c0 tx=0x7f6bb400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:58.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.928+0000 7f6bc1600640 1 -- 172.21.15.112:0/433917646 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb400cd80 con 0x7f6bc4104a30 2024-05-08T05:54:58.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.928+0000 7f6bc9c00640 1 -- 172.21.15.112:0/433917646 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc41a2b40 con 0x7f6bc4104a30 2024-05-08T05:54:58.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.928+0000 7f6bc1600640 1 -- 172.21.15.112:0/433917646 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb40035f0 con 0x7f6bc4104a30 2024-05-08T05:54:58.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.928+0000 7f6bc1600640 1 -- 172.21.15.112:0/433917646 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4018630 con 0x7f6bc4104a30 2024-05-08T05:54:58.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.929+0000 7f6bc9c00640 1 -- 172.21.15.112:0/433917646 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc41a30e0 con 0x7f6bc4104a30 2024-05-08T05:54:58.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.930+0000 7f6bc9c00640 1 -- 172.21.15.112:0/433917646 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc410bdd0 con 0x7f6bc4104a30 2024-05-08T05:54:58.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.932+0000 7f6bc1600640 1 -- 172.21.15.112:0/433917646 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6bb40037a0 con 0x7f6bc4104a30 2024-05-08T05:54:58.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:58 smithi150 ceph-mon[34315]: pgmap v431: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:58.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:58 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/261654489' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T05:54:58.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.934+0000 7f6bc1600640 1 --2- 172.21.15.112:0/433917646 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6ba80b27f0 0x7f6ba80b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:58.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.935+0000 7f6bc1600640 1 -- 172.21.15.112:0/433917646 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f6bb40e25e0 con 0x7f6bc4104a30 2024-05-08T05:54:58.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.935+0000 7f6bbb400640 1 --2- 172.21.15.112:0/433917646 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6ba80b27f0 0x7f6ba80b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:58.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.935+0000 7f6bbb400640 1 --2- 172.21.15.112:0/433917646 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6ba80b27f0 0x7f6ba80b4cb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6bac031500 tx=0x7f6bac005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:58.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:58.939+0000 7f6bc1600640 1 -- 172.21.15.112:0/433917646 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6bb40aa2d0 con 0x7f6bc4104a30 2024-05-08T05:54:59.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:58 smithi112 ceph-mon[30003]: pgmap v431: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:59.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:58 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/261654489' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T05:54:59.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:58 smithi162 ceph-mon[34800]: pgmap v431: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:54:59.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:58 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/261654489' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T05:54:59.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.152+0000 7f6bc9c00640 1 -- 172.21.15.112:0/433917646 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6bc402db50 con 0x7f6bc4104a30 2024-05-08T05:54:59.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.154+0000 7f6bc1600640 1 -- 172.21.15.112:0/433917646 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22829 (secure 0 0 0) 0x7f6bb40a9bd0 con 0x7f6bc4104a30 2024-05-08T05:54:59.155 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:54:59.155 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":84,"fsid":"7a07ff60-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:48.498871+0000","modified":"2024-05-08T05:53:05.251932+0000","last_up_change":"2024-05-08T05:49:54.039380+0000","last_in_change":"2024-05-08T05:49:34.443850+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:47.485105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:50:58.918909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.175022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b378542-1bbe-4614-8805-8a6b875ada55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6803","nonce":3419091820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6805","nonce":3419091820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6809","nonce":3419091820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6807","nonce":3419091820}]},"public_addr":"172.21.15.112:6803/3419091820","cluster_addr":"172.21.15.112:6805/3419091820","heartbeat_back_addr":"172.21.15.112:6809/3419091820","heartbeat_front_addr":"172.21.15.112:6807/3419091820","state":["exists","up"]},{"osd":1,"uuid":"826ef036-2db7-470e-9956-d9a58b9181e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6801","nonce":2728419797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6803","nonce":2728419797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6807","nonce":2728419797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6805","nonce":2728419797}]},"public_addr":"172.21.15.150:6801/2728419797","cluster_addr":"172.21.15.150:6803/2728419797","heartbeat_back_addr":"172.21.15.150:6807/2728419797","heartbeat_front_addr":"172.21.15.150:6805/2728419797","state":["exists","up"]},{"osd":2,"uuid":"32e710e7-2a87-4863-bc7a-906d31aa4ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6801","nonce":1556203215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6803","nonce":1556203215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6807","nonce":1556203215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6805","nonce":1556203215}]},"public_addr":"172.21.15.162:6801/1556203215","cluster_addr":"172.21.15.162:6803/1556203215","heartbeat_back_addr":"172.21.15.162:6807/1556203215","heartbeat_front_addr":"172.21.15.162:6805/1556203215","state":["exists","up"]},{"osd":3,"uuid":"ba1d4a5f-6a95-4b9e-8a48-8524a899b004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6811","nonce":3325718755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6813","nonce":3325718755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6817","nonce":3325718755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6815","nonce":3325718755}]},"public_addr":"172.21.15.112:6811/3325718755","cluster_addr":"172.21.15.112:6813/3325718755","heartbeat_back_addr":"172.21.15.112:6817/3325718755","heartbeat_front_addr":"172.21.15.112:6815/3325718755","state":["exists","up"]},{"osd":4,"uuid":"cb4f1a42-8de8-4b50-b3bc-602de9f4803a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6809","nonce":1548914683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6811","nonce":1548914683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6815","nonce":1548914683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6813","nonce":1548914683}]},"public_addr":"172.21.15.150:6809/1548914683","cluster_addr":"172.21.15.150:6811/1548914683","heartbeat_back_addr":"172.21.15.150:6815/1548914683","heartbeat_front_addr":"172.21.15.150:6813/1548914683","state":["exists","up"]},{"osd":5,"uuid":"a26c03b8-050c-4a2c-8f23-5801fffb7c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6809","nonce":409836750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6811","nonce":409836750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6815","nonce":409836750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6813","nonce":409836750}]},"public_addr":"172.21.15.162:6809/409836750","cluster_addr":"172.21.15.162:6811/409836750","heartbeat_back_addr":"172.21.15.162:6815/409836750","heartbeat_front_addr":"172.21.15.162:6813/409836750","state":["exists","up"]},{"osd":6,"uuid":"8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6819","nonce":1098984186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6821","nonce":1098984186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6825","nonce":1098984186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6823","nonce":1098984186}]},"public_addr":"172.21.15.112:6819/1098984186","cluster_addr":"172.21.15.112:6821/1098984186","heartbeat_back_addr":"172.21.15.112:6825/1098984186","heartbeat_front_addr":"172.21.15.112:6823/1098984186","state":["exists","up"]},{"osd":7,"uuid":"af3629b4-9bed-4669-97a6-48deba503a96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6817","nonce":4009836564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6819","nonce":4009836564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6823","nonce":4009836564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6821","nonce":4009836564}]},"public_addr":"172.21.15.150:6817/4009836564","cluster_addr":"172.21.15.150:6819/4009836564","heartbeat_back_addr":"172.21.15.150:6823/4009836564","heartbeat_front_addr":"172.21.15.150:6821/4009836564","state":["exists","up"]},{"osd":8,"uuid":"9333e87d-dd78-4467-bbde-c2b44f4ea07f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6817","nonce":852791606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6819","nonce":852791606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6823","nonce":852791606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6821","nonce":852791606}]},"public_addr":"172.21.15.162:6817/852791606","cluster_addr":"172.21.15.162:6819/852791606","heartbeat_back_addr":"172.21.15.162:6823/852791606","heartbeat_front_addr":"172.21.15.162:6821/852791606","state":["exists","up"]},{"osd":9,"uuid":"09723581-1995-42e0-8db7-776ea98a918c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6827","nonce":2186764674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6829","nonce":2186764674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6833","nonce":2186764674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6831","nonce":2186764674}]},"public_addr":"172.21.15.112:6827/2186764674","cluster_addr":"172.21.15.112:6829/2186764674","heartbeat_back_addr":"172.21.15.112:6833/2186764674","heartbeat_front_addr":"172.21.15.112:6831/2186764674","state":["exists","up"]},{"osd":10,"uuid":"34e08e3e-1231-4899-a5b3-76d449105bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6825","nonce":2838995474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6827","nonce":2838995474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6831","nonce":2838995474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6829","nonce":2838995474}]},"public_addr":"172.21.15.150:6825/2838995474","cluster_addr":"172.21.15.150:6827/2838995474","heartbeat_back_addr":"172.21.15.150:6831/2838995474","heartbeat_front_addr":"172.21.15.150:6829/2838995474","state":["exists","up"]},{"osd":11,"uuid":"6fc34ec0-78ca-403c-acb7-b29d83ff64e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6825","nonce":3702614331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6827","nonce":3702614331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6831","nonce":3702614331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6829","nonce":3702614331}]},"public_addr":"172.21.15.162:6825/3702614331","cluster_addr":"172.21.15.162:6827/3702614331","heartbeat_back_addr":"172.21.15.162:6831/3702614331","heartbeat_front_addr":"172.21.15.162:6829/3702614331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:54.518702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:19.939543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:44.425586+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:16.126471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:41.696071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:05.517405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:38.549318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:03.799649+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:28.007282+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:01.395459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:27.963637+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:52.181212+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3667148497":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6800/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3425327158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1404611205":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3765334043":"2024-05-09T05:42:29.819307+0000","172.21.15.112:6800/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/2814303790":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1091751583":"2024-05-09T05:42:29.819307+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:54:59.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.157+0000 7f6bc9c00640 1 -- 172.21.15.112:0/433917646 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6ba80b27f0 msgr2=0x7f6ba80b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:59.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.157+0000 7f6bc9c00640 1 --2- 172.21.15.112:0/433917646 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6ba80b27f0 0x7f6ba80b4cb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6bac031500 tx=0x7f6bac005a20 comp rx=0 tx=0).stop 2024-05-08T05:54:59.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.157+0000 7f6bc9c00640 1 -- 172.21.15.112:0/433917646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc4104a30 msgr2=0x7f6bc419ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:59.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.157+0000 7f6bc9c00640 1 --2- 172.21.15.112:0/433917646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc4104a30 0x7f6bc419ac40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400e9c0 tx=0x7f6bb400ee80 comp rx=0 tx=0).stop 2024-05-08T05:54:59.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.157+0000 7f6bc9c00640 1 -- 172.21.15.112:0/433917646 shutdown_connections 2024-05-08T05:54:59.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.157+0000 7f6bc9c00640 1 --2- 172.21.15.112:0/433917646 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6ba80b27f0 0x7f6ba80b4cb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.157+0000 7f6bc9c00640 1 --2- 172.21.15.112:0/433917646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bc41065e0 0x7f6bc41a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.157+0000 7f6bc9c00640 1 --2- 172.21.15.112:0/433917646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bc4105c20 0x7f6bc419b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.157+0000 7f6bc9c00640 1 --2- 172.21.15.112:0/433917646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc4104a30 0x7f6bc419ac40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.157+0000 7f6bc9c00640 1 -- 172.21.15.112:0/433917646 >> 172.21.15.112:0/433917646 conn(0x7f6bc40fffc0 msgr2=0x7f6bc4108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:59.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.157+0000 7f6bc9c00640 1 -- 172.21.15.112:0/433917646 shutdown_connections 2024-05-08T05:54:59.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.157+0000 7f6bc9c00640 1 -- 172.21.15.112:0/433917646 wait complete. 2024-05-08T05:54:59.178 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T05:45:47.485105+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T05:50:58.918909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T05:51:03.175022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T05:54:59.179 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T05:54:59.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.386+0000 7f74abe00640 1 -- 172.21.15.112:0/667346373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74ac103ab0 msgr2=0x7f74ac105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:59.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.386+0000 7f74abe00640 1 --2- 172.21.15.112:0/667346373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74ac103ab0 0x7f74ac105f20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7498009900 tx=0x7f749802a3f0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.386+0000 7f74abe00640 1 -- 172.21.15.112:0/667346373 shutdown_connections 2024-05-08T05:54:59.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.386+0000 7f74abe00640 1 --2- 172.21.15.112:0/667346373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74ac106460 0x7f74ac1068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.386+0000 7f74abe00640 1 --2- 172.21.15.112:0/667346373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74ac103ab0 0x7f74ac105f20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.386+0000 7f74abe00640 1 --2- 172.21.15.112:0/667346373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ac101140 0x7f74ac103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.386+0000 7f74abe00640 1 -- 172.21.15.112:0/667346373 >> 172.21.15.112:0/667346373 conn(0x7f74ac0fcd60 msgr2=0x7f74ac0ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:59.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.386+0000 7f74abe00640 1 -- 172.21.15.112:0/667346373 shutdown_connections 2024-05-08T05:54:59.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.386+0000 7f74abe00640 1 -- 172.21.15.112:0/667346373 wait complete. 2024-05-08T05:54:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.387+0000 7f74abe00640 1 Processor -- start 2024-05-08T05:54:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.387+0000 7f74abe00640 1 -- start start 2024-05-08T05:54:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.387+0000 7f74abe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74ac101140 0x7f74ac19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.387+0000 7f74abe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74ac103ab0 0x7f74ac19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.387+0000 7f74abe00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ac106460 0x7f74ac1a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.387+0000 7f74abe00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac114a30 con 0x7f74ac106460 2024-05-08T05:54:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.387+0000 7f74abe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1148b0 con 0x7f74ac101140 2024-05-08T05:54:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.387+0000 7f74abe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac114bb0 con 0x7f74ac103ab0 2024-05-08T05:54:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.388+0000 7f74ab400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ac106460 0x7f74ac1a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.388+0000 7f74aa000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74ac103ab0 0x7f74ac19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.388+0000 7f74aaa00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74ac101140 0x7f74ac19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.388+0000 7f74ab400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ac106460 0x7f74ac1a66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43592/0 (socket says 172.21.15.112:43592) 2024-05-08T05:54:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.388+0000 7f74ab400640 1 -- 172.21.15.112:0/3549188938 learned_addr learned my addr 172.21.15.112:0/3549188938 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:59.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.388+0000 7f74aaa00640 1 -- 172.21.15.112:0/3549188938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74ac103ab0 msgr2=0x7f74ac19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:59.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.388+0000 7f74aaa00640 1 --2- 172.21.15.112:0/3549188938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74ac103ab0 0x7f74ac19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.388+0000 7f74aaa00640 1 -- 172.21.15.112:0/3549188938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ac106460 msgr2=0x7f74ac1a66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:59.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.388+0000 7f74aaa00640 1 --2- 172.21.15.112:0/3549188938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ac106460 0x7f74ac1a66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.388+0000 7f74aaa00640 1 -- 172.21.15.112:0/3549188938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ac1a6d20 con 0x7f74ac101140 2024-05-08T05:54:59.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.388+0000 7f74aa000640 1 --2- 172.21.15.112:0/3549188938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74ac103ab0 0x7f74ac19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:59.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.389+0000 7f74aaa00640 1 --2- 172.21.15.112:0/3549188938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74ac101140 0x7f74ac19f100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f749c005fd0 tx=0x7f749c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.389+0000 7f74a3400640 1 -- 172.21.15.112:0/3549188938 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c009c30 con 0x7f74ac101140 2024-05-08T05:54:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.389+0000 7f74a3400640 1 -- 172.21.15.112:0/3549188938 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f749c009dd0 con 0x7f74ac101140 2024-05-08T05:54:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.389+0000 7f74abe00640 1 -- 172.21.15.112:0/3549188938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ac1a6fb0 con 0x7f74ac101140 2024-05-08T05:54:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.389+0000 7f74a3400640 1 -- 172.21.15.112:0/3549188938 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c003e20 con 0x7f74ac101140 2024-05-08T05:54:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.389+0000 7f74abe00640 1 -- 172.21.15.112:0/3549188938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ac1a7470 con 0x7f74ac101140 2024-05-08T05:54:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.390+0000 7f74abe00640 1 -- 172.21.15.112:0/3549188938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74ac0fe850 con 0x7f74ac101140 2024-05-08T05:54:59.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.392+0000 7f74a3400640 1 -- 172.21.15.112:0/3549188938 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f749c003fe0 con 0x7f74ac101140 2024-05-08T05:54:59.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.394+0000 7f74a3400640 1 --2- 172.21.15.112:0/3549188938 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f74900b27f0 0x7f74900b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:59.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.394+0000 7f74a3400640 1 -- 172.21.15.112:0/3549188938 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f749c0e1dc0 con 0x7f74ac101140 2024-05-08T05:54:59.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.394+0000 7f74aa000640 1 --2- 172.21.15.112:0/3549188938 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f74900b27f0 0x7f74900b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:59.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.395+0000 7f74aa000640 1 --2- 172.21.15.112:0/3549188938 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f74900b27f0 0x7f74900b4cb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7498031500 tx=0x7f7498005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:59.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.396+0000 7f74a3400640 1 -- 172.21.15.112:0/3549188938 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f749c017020 con 0x7f74ac101140 2024-05-08T05:54:59.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.603+0000 7f74abe00640 1 -- 172.21.15.112:0/3549188938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f74ac02dd00 con 0x7f74ac101140 2024-05-08T05:54:59.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.604+0000 7f74a3400640 1 -- 172.21.15.112:0/3549188938 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f749c0a9400 con 0x7f74ac101140 2024-05-08T05:54:59.604 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-05-08T05:54:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.608+0000 7f74abe00640 1 -- 172.21.15.112:0/3549188938 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f74900b27f0 msgr2=0x7f74900b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.608+0000 7f74abe00640 1 --2- 172.21.15.112:0/3549188938 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f74900b27f0 0x7f74900b4cb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7498031500 tx=0x7f7498005a20 comp rx=0 tx=0).stop 2024-05-08T05:54:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.608+0000 7f74abe00640 1 -- 172.21.15.112:0/3549188938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74ac101140 msgr2=0x7f74ac19f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.608+0000 7f74abe00640 1 --2- 172.21.15.112:0/3549188938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74ac101140 0x7f74ac19f100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f749c005fd0 tx=0x7f749c00dde0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.608+0000 7f74abe00640 1 -- 172.21.15.112:0/3549188938 shutdown_connections 2024-05-08T05:54:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.608+0000 7f74abe00640 1 --2- 172.21.15.112:0/3549188938 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f74900b27f0 0x7f74900b4cb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.608+0000 7f74abe00640 1 --2- 172.21.15.112:0/3549188938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ac106460 0x7f74ac1a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.608+0000 7f74abe00640 1 --2- 172.21.15.112:0/3549188938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74ac103ab0 0x7f74ac19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.608+0000 7f74abe00640 1 --2- 172.21.15.112:0/3549188938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74ac101140 0x7f74ac19f100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.608+0000 7f74abe00640 1 -- 172.21.15.112:0/3549188938 >> 172.21.15.112:0/3549188938 conn(0x7f74ac0fcd60 msgr2=0x7f74ac101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.608+0000 7f74abe00640 1 -- 172.21.15.112:0/3549188938 shutdown_connections 2024-05-08T05:54:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.608+0000 7f74abe00640 1 -- 172.21.15.112:0/3549188938 wait complete. 2024-05-08T05:54:59.627 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T05:54:59.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.821+0000 7efcc2e00640 1 -- 172.21.15.112:0/1788011006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcbc104af0 msgr2=0x7efcbc104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:59.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.821+0000 7efcc2e00640 1 --2- 172.21.15.112:0/1788011006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcbc104af0 0x7efcbc104f10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009900 tx=0x7efcb002a3f0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.821+0000 7efcc2e00640 1 -- 172.21.15.112:0/1788011006 shutdown_connections 2024-05-08T05:54:59.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.821+0000 7efcc2e00640 1 --2- 172.21.15.112:0/1788011006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcbc1066a0 0x7efcbc10cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.821+0000 7efcc2e00640 1 --2- 172.21.15.112:0/1788011006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcbc105ce0 0x7efcbc106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.821+0000 7efcc2e00640 1 --2- 172.21.15.112:0/1788011006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcbc104af0 0x7efcbc104f10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.821+0000 7efcc2e00640 1 -- 172.21.15.112:0/1788011006 >> 172.21.15.112:0/1788011006 conn(0x7efcbc100060 msgr2=0x7efcbc1024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:59.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.821+0000 7efcc2e00640 1 -- 172.21.15.112:0/1788011006 shutdown_connections 2024-05-08T05:54:59.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.821+0000 7efcc2e00640 1 -- 172.21.15.112:0/1788011006 wait complete. 2024-05-08T05:54:59.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.822+0000 7efcc2e00640 1 Processor -- start 2024-05-08T05:54:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.822+0000 7efcc2e00640 1 -- start start 2024-05-08T05:54:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.822+0000 7efcc2e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcbc104af0 0x7efcbc19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.822+0000 7efcc2e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcbc105ce0 0x7efcbc19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.822+0000 7efcc2e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcbc1066a0 0x7efcbc1a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.822+0000 7efcc2e00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc110660 con 0x7efcbc104af0 2024-05-08T05:54:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.822+0000 7efcc2e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc1104e0 con 0x7efcbc105ce0 2024-05-08T05:54:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.822+0000 7efcc2e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc1107e0 con 0x7efcbc1066a0 2024-05-08T05:54:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.823+0000 7efcc1000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcbc105ce0 0x7efcbc19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.823+0000 7efcc1a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcbc104af0 0x7efcbc19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.823+0000 7efcc2400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcbc1066a0 0x7efcbc1a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.823+0000 7efcc1000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcbc105ce0 0x7efcbc19b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:41236/0 (socket says 172.21.15.112:41236) 2024-05-08T05:54:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.823+0000 7efcc1000640 1 -- 172.21.15.112:0/1596146337 learned_addr learned my addr 172.21.15.112:0/1596146337 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:54:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.823+0000 7efcc1000640 1 -- 172.21.15.112:0/1596146337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcbc1066a0 msgr2=0x7efcbc1a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.823+0000 7efcc1000640 1 --2- 172.21.15.112:0/1596146337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcbc1066a0 0x7efcbc1a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.823+0000 7efcc1000640 1 -- 172.21.15.112:0/1596146337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcbc104af0 msgr2=0x7efcbc19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:59.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.823+0000 7efcc1000640 1 --2- 172.21.15.112:0/1596146337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcbc104af0 0x7efcbc19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.823+0000 7efcc1000640 1 -- 172.21.15.112:0/1596146337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcbc1a2960 con 0x7efcbc105ce0 2024-05-08T05:54:59.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.823+0000 7efcc2400640 1 --2- 172.21.15.112:0/1596146337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcbc1066a0 0x7efcbc1a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:59.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.823+0000 7efcc1000640 1 --2- 172.21.15.112:0/1596146337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcbc105ce0 0x7efcbc19b290 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efca800e9f0 tx=0x7efca800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:59.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.824+0000 7efcba000640 1 -- 172.21.15.112:0/1596146337 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca800cd80 con 0x7efcbc105ce0 2024-05-08T05:54:59.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.824+0000 7efcba000640 1 -- 172.21.15.112:0/1596146337 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efca80054d0 con 0x7efcbc105ce0 2024-05-08T05:54:59.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.824+0000 7efcba000640 1 -- 172.21.15.112:0/1596146337 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8010690 con 0x7efcbc105ce0 2024-05-08T05:54:59.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.824+0000 7efcc2e00640 1 -- 172.21.15.112:0/1596146337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcbc1a2c50 con 0x7efcbc105ce0 2024-05-08T05:54:59.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.824+0000 7efcc2e00640 1 -- 172.21.15.112:0/1596146337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcbc1a3120 con 0x7efcbc105ce0 2024-05-08T05:54:59.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.825+0000 7efcc2e00640 1 -- 172.21.15.112:0/1596146337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcbc10beb0 con 0x7efcbc105ce0 2024-05-08T05:54:59.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.827+0000 7efcba000640 1 -- 172.21.15.112:0/1596146337 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7efca8004e90 con 0x7efcbc105ce0 2024-05-08T05:54:59.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.828+0000 7efcba000640 1 --2- 172.21.15.112:0/1596146337 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efca40b2870 0x7efca40b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:59.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.829+0000 7efcba000640 1 -- 172.21.15.112:0/1596146337 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7efca80e26a0 con 0x7efcbc105ce0 2024-05-08T05:54:59.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.829+0000 7efcc1a00640 1 --2- 172.21.15.112:0/1596146337 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efca40b2870 0x7efca40b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:59.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.829+0000 7efcc1a00640 1 --2- 172.21.15.112:0/1596146337 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efca40b2870 0x7efca40b4d30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efcb0005ab0 tx=0x7efcb0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:59.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:54:59.833+0000 7efcba000640 1 -- 172.21.15.112:0/1596146337 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efca80aa330 con 0x7efcbc105ce0 2024-05-08T05:54:59.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:59 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/261654489' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T05:54:59.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:59 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:54:59.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:59 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/433917646' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:54:59.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:54:59 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3549188938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T05:55:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.039+0000 7efcc2e00640 1 -- 172.21.15.112:0/1596146337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efcbc0635b0 con 0x7efcbc105ce0 2024-05-08T05:55:00.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.039+0000 7efcba000640 1 -- 172.21.15.112:0/1596146337 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7efca80a9c30 con 0x7efcbc105ce0 2024-05-08T05:55:00.040 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-05-08T05:55:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.042+0000 7efcc2e00640 1 -- 172.21.15.112:0/1596146337 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efca40b2870 msgr2=0x7efca40b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.042+0000 7efcc2e00640 1 --2- 172.21.15.112:0/1596146337 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efca40b2870 0x7efca40b4d30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efcb0005ab0 tx=0x7efcb0005a20 comp rx=0 tx=0).stop 2024-05-08T05:55:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.042+0000 7efcc2e00640 1 -- 172.21.15.112:0/1596146337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcbc105ce0 msgr2=0x7efcbc19b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.042+0000 7efcc2e00640 1 --2- 172.21.15.112:0/1596146337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcbc105ce0 0x7efcbc19b290 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efca800e9f0 tx=0x7efca800edf0 comp rx=0 tx=0).stop 2024-05-08T05:55:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.042+0000 7efcc2e00640 1 -- 172.21.15.112:0/1596146337 shutdown_connections 2024-05-08T05:55:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.042+0000 7efcc2e00640 1 --2- 172.21.15.112:0/1596146337 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7efca40b2870 0x7efca40b4d30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.042+0000 7efcc2e00640 1 --2- 172.21.15.112:0/1596146337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcbc1066a0 0x7efcbc1a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.042+0000 7efcc2e00640 1 --2- 172.21.15.112:0/1596146337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcbc105ce0 0x7efcbc19b290 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.042+0000 7efcc2e00640 1 --2- 172.21.15.112:0/1596146337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcbc104af0 0x7efcbc19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.042+0000 7efcc2e00640 1 -- 172.21.15.112:0/1596146337 >> 172.21.15.112:0/1596146337 conn(0x7efcbc100060 msgr2=0x7efcbc109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.042+0000 7efcc2e00640 1 -- 172.21.15.112:0/1596146337 shutdown_connections 2024-05-08T05:55:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.042+0000 7efcc2e00640 1 -- 172.21.15.112:0/1596146337 wait complete. 2024-05-08T05:55:00.058 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:59 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/261654489' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T05:55:00.058 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:59 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:55:00.058 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:59 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/433917646' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:55:00.058 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:54:59 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3549188938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T05:55:00.061 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T05:55:00.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:59 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/261654489' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T05:55:00.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:59 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:55:00.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:59 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/433917646' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:55:00.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:54:59 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3549188938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T05:55:00.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.259+0000 7fca75a00640 1 -- 172.21.15.112:0/338513432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca70075410 msgr2=0x7fca70075890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.259+0000 7fca75a00640 1 --2- 172.21.15.112:0/338513432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca70075410 0x7fca70075890 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fca58009890 tx=0x7fca5802a360 comp rx=0 tx=0).stop 2024-05-08T05:55:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.259+0000 7fca75a00640 1 -- 172.21.15.112:0/338513432 shutdown_connections 2024-05-08T05:55:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.259+0000 7fca75a00640 1 --2- 172.21.15.112:0/338513432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca7010efd0 0x7fca701113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.259+0000 7fca75a00640 1 --2- 172.21.15.112:0/338513432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca70075410 0x7fca70075890 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.259+0000 7fca75a00640 1 --2- 172.21.15.112:0/338513432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca700769f0 0x7fca70074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.259+0000 7fca75a00640 1 -- 172.21.15.112:0/338513432 >> 172.21.15.112:0/338513432 conn(0x7fca700fdf90 msgr2=0x7fca701003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.259+0000 7fca75a00640 1 -- 172.21.15.112:0/338513432 shutdown_connections 2024-05-08T05:55:00.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.259+0000 7fca75a00640 1 -- 172.21.15.112:0/338513432 wait complete. 2024-05-08T05:55:00.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.260+0000 7fca75a00640 1 Processor -- start 2024-05-08T05:55:00.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.260+0000 7fca75a00640 1 -- start start 2024-05-08T05:55:00.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.260+0000 7fca75a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca70075410 0x7fca7019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:00.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.260+0000 7fca75a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca700769f0 0x7fca7019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:00.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.261+0000 7fca75a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca7010efd0 0x7fca701a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:00.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.261+0000 7fca75a00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fca70114b00 con 0x7fca7010efd0 2024-05-08T05:55:00.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.261+0000 7fca75a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fca70114980 con 0x7fca700769f0 2024-05-08T05:55:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.261+0000 7fca75a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fca70114c80 con 0x7fca70075410 2024-05-08T05:55:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.261+0000 7fca6f400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca7010efd0 0x7fca701a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.261+0000 7fca6f400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca7010efd0 0x7fca701a6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43624/0 (socket says 172.21.15.112:43624) 2024-05-08T05:55:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.261+0000 7fca6f400640 1 -- 172.21.15.112:0/591056136 learned_addr learned my addr 172.21.15.112:0/591056136 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.261+0000 7fca6e000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca700769f0 0x7fca7019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.261+0000 7fca6ea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca70075410 0x7fca7019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.261+0000 7fca6f400640 1 -- 172.21.15.112:0/591056136 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca70075410 msgr2=0x7fca7019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.261+0000 7fca6f400640 1 --2- 172.21.15.112:0/591056136 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca70075410 0x7fca7019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.261+0000 7fca6f400640 1 -- 172.21.15.112:0/591056136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca700769f0 msgr2=0x7fca7019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.261+0000 7fca6f400640 1 --2- 172.21.15.112:0/591056136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca700769f0 0x7fca7019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.261+0000 7fca6f400640 1 -- 172.21.15.112:0/591056136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca701a6e80 con 0x7fca7010efd0 2024-05-08T05:55:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.262+0000 7fca6f400640 1 --2- 172.21.15.112:0/591056136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca7010efd0 0x7fca701a6810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fca68009fd0 tx=0x7fca6800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.262+0000 7fca67400640 1 -- 172.21.15.112:0/591056136 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68004b20 con 0x7fca7010efd0 2024-05-08T05:55:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.262+0000 7fca67400640 1 -- 172.21.15.112:0/591056136 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca68003e20 con 0x7fca7010efd0 2024-05-08T05:55:00.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.262+0000 7fca67400640 1 -- 172.21.15.112:0/591056136 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68004820 con 0x7fca7010efd0 2024-05-08T05:55:00.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.262+0000 7fca75a00640 1 -- 172.21.15.112:0/591056136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca701a7170 con 0x7fca7010efd0 2024-05-08T05:55:00.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.262+0000 7fca75a00640 1 -- 172.21.15.112:0/591056136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca701a7640 con 0x7fca7010efd0 2024-05-08T05:55:00.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.263+0000 7fca75a00640 1 -- 172.21.15.112:0/591056136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca701051a0 con 0x7fca7010efd0 2024-05-08T05:55:00.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.266+0000 7fca67400640 1 -- 172.21.15.112:0/591056136 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fca6801d020 con 0x7fca7010efd0 2024-05-08T05:55:00.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.267+0000 7fca67400640 1 --2- 172.21.15.112:0/591056136 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fca540b27f0 0x7fca540b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:00.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.267+0000 7fca67400640 1 -- 172.21.15.112:0/591056136 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fca680e2cb0 con 0x7fca7010efd0 2024-05-08T05:55:00.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.268+0000 7fca6ea00640 1 --2- 172.21.15.112:0/591056136 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fca540b27f0 0x7fca540b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:00.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.268+0000 7fca6ea00640 1 --2- 172.21.15.112:0/591056136 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fca540b27f0 0x7fca540b4cb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca60009bd0 tx=0x7fca60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:00.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.270+0000 7fca67400640 1 -- 172.21.15.112:0/591056136 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca68024030 con 0x7fca7010efd0 2024-05-08T05:55:00.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.477+0000 7fca75a00640 1 -- 172.21.15.112:0/591056136 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fca700635b0 con 0x7fca7010efd0 2024-05-08T05:55:00.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.478+0000 7fca67400640 1 -- 172.21.15.112:0/591056136 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7fca680aa940 con 0x7fca7010efd0 2024-05-08T05:55:00.478 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-05-08T05:55:00.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.480+0000 7fca75a00640 1 -- 172.21.15.112:0/591056136 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fca540b27f0 msgr2=0x7fca540b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:00.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.480+0000 7fca75a00640 1 --2- 172.21.15.112:0/591056136 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fca540b27f0 0x7fca540b4cb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca60009bd0 tx=0x7fca60005f90 comp rx=0 tx=0).stop 2024-05-08T05:55:00.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.480+0000 7fca75a00640 1 -- 172.21.15.112:0/591056136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca7010efd0 msgr2=0x7fca701a6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:00.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.480+0000 7fca75a00640 1 --2- 172.21.15.112:0/591056136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca7010efd0 0x7fca701a6810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fca68009fd0 tx=0x7fca6800c800 comp rx=0 tx=0).stop 2024-05-08T05:55:00.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.480+0000 7fca75a00640 1 -- 172.21.15.112:0/591056136 shutdown_connections 2024-05-08T05:55:00.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.480+0000 7fca75a00640 1 --2- 172.21.15.112:0/591056136 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fca540b27f0 0x7fca540b4cb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:00.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.480+0000 7fca75a00640 1 --2- 172.21.15.112:0/591056136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca7010efd0 0x7fca701a6810 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:00.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.480+0000 7fca75a00640 1 --2- 172.21.15.112:0/591056136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca700769f0 0x7fca7019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:00.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.480+0000 7fca75a00640 1 --2- 172.21.15.112:0/591056136 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca70075410 0x7fca7019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:00.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.480+0000 7fca75a00640 1 -- 172.21.15.112:0/591056136 >> 172.21.15.112:0/591056136 conn(0x7fca700fdf90 msgr2=0x7fca70102370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:00.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.481+0000 7fca75a00640 1 -- 172.21.15.112:0/591056136 shutdown_connections 2024-05-08T05:55:00.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:00.481+0000 7fca75a00640 1 -- 172.21.15.112:0/591056136 wait complete. 2024-05-08T05:55:00.501 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-08T05:55:00.501 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:55:00.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:00 smithi112 ceph-mon[30003]: pgmap v432: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-08T05:55:00.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:00 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1596146337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T05:55:00.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:00 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/591056136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T05:55:00.946 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:55:01.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:00 smithi162 ceph-mon[34800]: pgmap v432: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-08T05:55:01.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:00 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1596146337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T05:55:01.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:00 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/591056136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T05:55:01.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:00 smithi150 ceph-mon[34315]: pgmap v432: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-08T05:55:01.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:00 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1596146337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T05:55:01.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:00 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/591056136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T05:55:03.025 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:02 smithi112 ceph-mon[30003]: pgmap v433: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 6 op/s 2024-05-08T05:55:03.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:02 smithi162 ceph-mon[34800]: pgmap v433: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 6 op/s 2024-05-08T05:55:03.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:02 smithi150 ceph-mon[34315]: pgmap v433: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 6 op/s 2024-05-08T05:55:03.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.348+0000 7f0c15a00700 1 -- 172.21.15.112:0/918203505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1010ff50 msgr2=0x7f0c10112340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:03.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.348+0000 7f0c15a00700 1 --2- 172.21.15.112:0/918203505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1010ff50 0x7f0c10112340 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800b0a0 tx=0x7f0c0800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:55:03.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.348+0000 7f0c15a00700 1 -- 172.21.15.112:0/918203505 shutdown_connections 2024-05-08T05:55:03.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.348+0000 7f0c15a00700 1 --2- 172.21.15.112:0/918203505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1010ff50 0x7f0c10112340 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:03.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.348+0000 7f0c15a00700 1 --2- 172.21.15.112:0/918203505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c1006b1b0 0x7f0c1006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:03.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.348+0000 7f0c15a00700 1 --2- 172.21.15.112:0/918203505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1006bad0 0x7f0c1010fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:03.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.348+0000 7f0c15a00700 1 -- 172.21.15.112:0/918203505 >> 172.21.15.112:0/918203505 conn(0x7f0c10079350 msgr2=0x7f0c10079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:03.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.348+0000 7f0c15a00700 1 -- 172.21.15.112:0/918203505 shutdown_connections 2024-05-08T05:55:03.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.348+0000 7f0c15a00700 1 -- 172.21.15.112:0/918203505 wait complete. 2024-05-08T05:55:03.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.349+0000 7f0c15a00700 1 Processor -- start 2024-05-08T05:55:03.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.349+0000 7f0c15a00700 1 -- start start 2024-05-08T05:55:03.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.350+0000 7f0c15a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1006b1b0 0x7f0c1019bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:03.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.350+0000 7f0c15a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c1006bad0 0x7f0c1019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:03.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.350+0000 7f0c15a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1010ff50 0x7f0c101a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:03.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.350+0000 7f0c15a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c101158c0 con 0x7f0c1010ff50 2024-05-08T05:55:03.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.350+0000 7f0c15a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10115740 con 0x7f0c1006bad0 2024-05-08T05:55:03.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.350+0000 7f0c15a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10115a40 con 0x7f0c1006b1b0 2024-05-08T05:55:03.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.350+0000 7f0c0f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1010ff50 0x7f0c101a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:03.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.350+0000 7f0c0f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1010ff50 0x7f0c101a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43642/0 (socket says 172.21.15.112:43642) 2024-05-08T05:55:03.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.350+0000 7f0c0f400700 1 -- 172.21.15.112:0/3131261356 learned_addr learned my addr 172.21.15.112:0/3131261356 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:03.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.350+0000 7f0c0e000700 1 --2- 172.21.15.112:0/3131261356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c1006bad0 0x7f0c1019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.350+0000 7f0c0ea00700 1 --2- 172.21.15.112:0/3131261356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1006b1b0 0x7f0c1019bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.351+0000 7f0c0f400700 1 -- 172.21.15.112:0/3131261356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1006b1b0 msgr2=0x7f0c1019bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.351+0000 7f0c0f400700 1 --2- 172.21.15.112:0/3131261356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1006b1b0 0x7f0c1019bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.351+0000 7f0c0f400700 1 -- 172.21.15.112:0/3131261356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c1006bad0 msgr2=0x7f0c1019c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.351+0000 7f0c0f400700 1 --2- 172.21.15.112:0/3131261356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c1006bad0 0x7f0c1019c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.351+0000 7f0c0f400700 1 -- 172.21.15.112:0/3131261356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c101a0d90 con 0x7f0c1010ff50 2024-05-08T05:55:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.351+0000 7f0c0f400700 1 --2- 172.21.15.112:0/3131261356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1010ff50 0x7f0c101a0650 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800b070 tx=0x7f0c08004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.351+0000 7f0c07400700 1 -- 172.21.15.112:0/3131261356 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08021070 con 0x7f0c1010ff50 2024-05-08T05:55:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.351+0000 7f0c07400700 1 -- 172.21.15.112:0/3131261356 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c08007e30 con 0x7f0c1010ff50 2024-05-08T05:55:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.352+0000 7f0c15a00700 1 -- 172.21.15.112:0/3131261356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c101a1020 con 0x7f0c1010ff50 2024-05-08T05:55:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.352+0000 7f0c07400700 1 -- 172.21.15.112:0/3131261356 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c080045e0 con 0x7f0c1010ff50 2024-05-08T05:55:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.352+0000 7f0c15a00700 1 -- 172.21.15.112:0/3131261356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c101a8900 con 0x7f0c1010ff50 2024-05-08T05:55:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.353+0000 7f0c15a00700 1 -- 172.21.15.112:0/3131261356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c1006a190 con 0x7f0c1010ff50 2024-05-08T05:55:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.355+0000 7f0c07400700 1 -- 172.21.15.112:0/3131261356 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0c08003c90 con 0x7f0c1010ff50 2024-05-08T05:55:03.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.358+0000 7f0c07400700 1 --2- 172.21.15.112:0/3131261356 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0bf40b2900 0x7f0bf40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.358+0000 7f0c07400700 1 -- 172.21.15.112:0/3131261356 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f0c080e30d0 con 0x7f0c1010ff50 2024-05-08T05:55:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.358+0000 7f0c0ea00700 1 --2- 172.21.15.112:0/3131261356 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0bf40b2900 0x7f0bf40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.359+0000 7f0c0ea00700 1 --2- 172.21.15.112:0/3131261356 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0bf40b2900 0x7f0bf40b4dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009c60 tx=0x7f0c00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:03.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.361+0000 7f0c07400700 1 -- 172.21.15.112:0/3131261356 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0c080aadf0 con 0x7f0c1010ff50 2024-05-08T05:55:03.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.648+0000 7f0c15a00700 1 -- 172.21.15.112:0/3131261356 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c1002e510 con 0x7f0c1010ff50 2024-05-08T05:55:03.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.650+0000 7f0c07400700 1 -- 172.21.15.112:0/3131261356 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11891 (secure 0 0 0) 0x7f0c080aa6f0 con 0x7f0c1010ff50 2024-05-08T05:55:03.650 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:55:03.651 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T05:54:58.248303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24518,24511]},"id":1}]} 2024-05-08T05:55:03.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.654+0000 7f0c15a00700 1 -- 172.21.15.112:0/3131261356 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0bf40b2900 msgr2=0x7f0bf40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:03.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.654+0000 7f0c15a00700 1 --2- 172.21.15.112:0/3131261356 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0bf40b2900 0x7f0bf40b4dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009c60 tx=0x7f0c00009260 comp rx=0 tx=0).stop 2024-05-08T05:55:03.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.654+0000 7f0c15a00700 1 -- 172.21.15.112:0/3131261356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1010ff50 msgr2=0x7f0c101a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:03.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.654+0000 7f0c15a00700 1 --2- 172.21.15.112:0/3131261356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1010ff50 0x7f0c101a0650 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800b070 tx=0x7f0c08004c60 comp rx=0 tx=0).stop 2024-05-08T05:55:03.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.654+0000 7f0c15a00700 1 -- 172.21.15.112:0/3131261356 shutdown_connections 2024-05-08T05:55:03.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.654+0000 7f0c15a00700 1 --2- 172.21.15.112:0/3131261356 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0bf40b2900 0x7f0bf40b4dc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:03.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.654+0000 7f0c15a00700 1 --2- 172.21.15.112:0/3131261356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c1010ff50 0x7f0c101a0650 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:03.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.654+0000 7f0c15a00700 1 --2- 172.21.15.112:0/3131261356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1006b1b0 0x7f0c1019bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:03.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.654+0000 7f0c15a00700 1 --2- 172.21.15.112:0/3131261356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c1006bad0 0x7f0c1019c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:03.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.654+0000 7f0c15a00700 1 -- 172.21.15.112:0/3131261356 >> 172.21.15.112:0/3131261356 conn(0x7f0c10079350 msgr2=0x7f0c10111440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:03.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.655+0000 7f0c15a00700 1 -- 172.21.15.112:0/3131261356 shutdown_connections 2024-05-08T05:55:03.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:03.655+0000 7f0c15a00700 1 -- 172.21.15.112:0/3131261356 wait complete. 2024-05-08T05:55:03.656 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-05-08T05:55:04.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:03 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3131261356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:04.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:03 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3131261356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:04.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:03 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3131261356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:04.476 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-08T05:55:04.476 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-08T05:55:04.476 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:55:04.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.668+0000 7fd46d400640 1 -- 172.21.15.112:0/2320415030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd468105db0 msgr2=0x7fd4681081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:04.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.668+0000 7fd46d400640 1 --2- 172.21.15.112:0/2320415030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd468105db0 0x7fd4681081a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd450009900 tx=0x7fd45002a3f0 comp rx=0 tx=0).stop 2024-05-08T05:55:04.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.669+0000 7fd46d400640 1 -- 172.21.15.112:0/2320415030 shutdown_connections 2024-05-08T05:55:04.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.669+0000 7fd46d400640 1 --2- 172.21.15.112:0/2320415030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4681086e0 0x7fd46810ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:04.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.669+0000 7fd46d400640 1 --2- 172.21.15.112:0/2320415030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd468105db0 0x7fd4681081a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:04.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.669+0000 7fd46d400640 1 --2- 172.21.15.112:0/2320415030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd468103480 0x7fd468105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:04.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.669+0000 7fd46d400640 1 -- 172.21.15.112:0/2320415030 >> 172.21.15.112:0/2320415030 conn(0x7fd4680fce90 msgr2=0x7fd4680ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:04.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.669+0000 7fd46d400640 1 -- 172.21.15.112:0/2320415030 shutdown_connections 2024-05-08T05:55:04.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.669+0000 7fd46d400640 1 -- 172.21.15.112:0/2320415030 wait complete. 2024-05-08T05:55:04.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.670+0000 7fd46d400640 1 Processor -- start 2024-05-08T05:55:04.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.670+0000 7fd46d400640 1 -- start start 2024-05-08T05:55:04.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.670+0000 7fd46d400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd468103480 0x7fd468102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:04.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.670+0000 7fd46d400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd468105db0 0x7fd468101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:04.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.670+0000 7fd46d400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4681086e0 0x7fd468101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:04.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.670+0000 7fd46d400640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd46810e570 con 0x7fd468103480 2024-05-08T05:55:04.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.670+0000 7fd46d400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd46810e3f0 con 0x7fd468105db0 2024-05-08T05:55:04.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.670+0000 7fd46d400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd46810e6f0 con 0x7fd4681086e0 2024-05-08T05:55:04.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.670+0000 7fd466800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd468103480 0x7fd468102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:04.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.671+0000 7fd466800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd468103480 0x7fd468102df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33054/0 (socket says 172.21.15.112:33054) 2024-05-08T05:55:04.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.671+0000 7fd466800640 1 -- 172.21.15.112:0/1698387237 learned_addr learned my addr 172.21.15.112:0/1698387237 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:04.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.671+0000 7fd45fe00640 1 --2- 172.21.15.112:0/1698387237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd468105db0 0x7fd468101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:04.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.671+0000 7fd467200640 1 --2- 172.21.15.112:0/1698387237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4681086e0 0x7fd468101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:04.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.671+0000 7fd466800640 1 -- 172.21.15.112:0/1698387237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4681086e0 msgr2=0x7fd468101970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:04.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.671+0000 7fd466800640 1 --2- 172.21.15.112:0/1698387237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4681086e0 0x7fd468101970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:04.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.671+0000 7fd466800640 1 -- 172.21.15.112:0/1698387237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd468105db0 msgr2=0x7fd468101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:04.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.671+0000 7fd466800640 1 --2- 172.21.15.112:0/1698387237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd468105db0 0x7fd468101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:04.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.671+0000 7fd466800640 1 -- 172.21.15.112:0/1698387237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd468102200 con 0x7fd468103480 2024-05-08T05:55:04.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.671+0000 7fd45fe00640 1 --2- 172.21.15.112:0/1698387237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd468105db0 0x7fd468101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:55:04.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.671+0000 7fd466800640 1 --2- 172.21.15.112:0/1698387237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd468103480 0x7fd468102df0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd45800ea60 tx=0x7fd45800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:04.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.671+0000 7fd45f400640 1 -- 172.21.15.112:0/1698387237 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45800cde0 con 0x7fd468103480 2024-05-08T05:55:04.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.672+0000 7fd46d400640 1 -- 172.21.15.112:0/1698387237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4681a2d60 con 0x7fd468103480 2024-05-08T05:55:04.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.672+0000 7fd45f400640 1 -- 172.21.15.112:0/1698387237 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4580035f0 con 0x7fd468103480 2024-05-08T05:55:04.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.672+0000 7fd45f400640 1 -- 172.21.15.112:0/1698387237 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458010690 con 0x7fd468103480 2024-05-08T05:55:04.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.672+0000 7fd46d400640 1 -- 172.21.15.112:0/1698387237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4681a31c0 con 0x7fd468103480 2024-05-08T05:55:04.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.673+0000 7fd46d400640 1 -- 172.21.15.112:0/1698387237 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4680fe980 con 0x7fd468103480 2024-05-08T05:55:04.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.675+0000 7fd45f400640 1 -- 172.21.15.112:0/1698387237 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd4580037a0 con 0x7fd468103480 2024-05-08T05:55:04.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.677+0000 7fd45f400640 1 --2- 172.21.15.112:0/1698387237 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd44c0b27f0 0x7fd44c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:04.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.677+0000 7fd45f400640 1 -- 172.21.15.112:0/1698387237 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd4580e25e0 con 0x7fd468103480 2024-05-08T05:55:04.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.677+0000 7fd45fe00640 1 --2- 172.21.15.112:0/1698387237 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd44c0b27f0 0x7fd44c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:04.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.678+0000 7fd45fe00640 1 --2- 172.21.15.112:0/1698387237 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd44c0b27f0 0x7fd44c0b4cb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd450006010 tx=0x7fd4500314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:04.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.680+0000 7fd45f400640 1 -- 172.21.15.112:0/1698387237 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd4580aa2d0 con 0x7fd468103480 2024-05-08T05:55:04.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.886+0000 7fd46d400640 1 -- 172.21.15.112:0/1698387237 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd468062df0 con 0x7fd44c0b27f0 2024-05-08T05:55:04.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.898+0000 7fd45f400640 1 -- 172.21.15.112:0/1698387237 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258834 (secure 0 0 0) 0x7fd468062df0 con 0x7fd44c0b27f0 2024-05-08T05:55:04.904 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:55:04.906 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-05-08T05:55:04.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.904+0000 7fd46d400640 1 -- 172.21.15.112:0/1698387237 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd44c0b27f0 msgr2=0x7fd44c0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.904+0000 7fd46d400640 1 --2- 172.21.15.112:0/1698387237 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd44c0b27f0 0x7fd44c0b4cb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd450006010 tx=0x7fd4500314a0 comp rx=0 tx=0).stop 2024-05-08T05:55:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.905+0000 7fd46d400640 1 -- 172.21.15.112:0/1698387237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd468103480 msgr2=0x7fd468102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.905+0000 7fd46d400640 1 --2- 172.21.15.112:0/1698387237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd468103480 0x7fd468102df0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd45800ea60 tx=0x7fd45800ef20 comp rx=0 tx=0).stop 2024-05-08T05:55:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.905+0000 7fd46d400640 1 -- 172.21.15.112:0/1698387237 shutdown_connections 2024-05-08T05:55:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.905+0000 7fd46d400640 1 --2- 172.21.15.112:0/1698387237 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd44c0b27f0 0x7fd44c0b4cb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.905+0000 7fd46d400640 1 --2- 172.21.15.112:0/1698387237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4681086e0 0x7fd468101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.905+0000 7fd46d400640 1 --2- 172.21.15.112:0/1698387237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd468105db0 0x7fd468101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.905+0000 7fd46d400640 1 --2- 172.21.15.112:0/1698387237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd468103480 0x7fd468102df0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.905+0000 7fd46d400640 1 -- 172.21.15.112:0/1698387237 >> 172.21.15.112:0/1698387237 conn(0x7fd4680fce90 msgr2=0x7fd468103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.905+0000 7fd46d400640 1 -- 172.21.15.112:0/1698387237 shutdown_connections 2024-05-08T05:55:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:04.905+0000 7fd46d400640 1 -- 172.21.15.112:0/1698387237 wait complete. 2024-05-08T05:55:04.925 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-05-08T05:55:03.567795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661677,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1381,"num_read_kb":793,"num_write":306,"num_write_kb":2280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951974,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":747,"ondisk_log_size":747,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366380,"kb_used_data":13044,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575444,"statfs":{"total":1151940427776,"available":1151565254656,"internally_reserved":0,"allocated":13357056,"data_stored":7408239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4934,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":37,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004939"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.580018+0000","last_change":"2024-05-08T05:51:04.320319+0000","last_active":"2024-05-08T05:53:07.580018+0000","last_peered":"2024-05-08T05:53:07.580018+0000","last_clean":"2024-05-08T05:53:07.580018+0000","last_became_active":"2024-05-08T05:51:04.319961+0000","last_became_peered":"2024-05-08T05:51:04.319961+0000","last_unstale":"2024-05-08T05:53:07.580018+0000","last_undegraded":"2024-05-08T05:53:07.580018+0000","last_fullsized":"2024-05-08T05:53:07.580018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:32.463261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032466+0000","last_change":"2024-05-08T05:51:04.315531+0000","last_active":"2024-05-08T05:53:07.032466+0000","last_peered":"2024-05-08T05:53:07.032466+0000","last_clean":"2024-05-08T05:53:07.032466+0000","last_became_active":"2024-05-08T05:51:04.314116+0000","last_became_peered":"2024-05-08T05:51:04.314116+0000","last_unstale":"2024-05-08T05:53:07.032466+0000","last_undegraded":"2024-05-08T05:53:07.032466+0000","last_fullsized":"2024-05-08T05:53:07.032466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:12:03.505888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032236+0000","last_change":"2024-05-08T05:51:04.315826+0000","last_active":"2024-05-08T05:53:07.032236+0000","last_peered":"2024-05-08T05:53:07.032236+0000","last_clean":"2024-05-08T05:53:07.032236+0000","last_became_active":"2024-05-08T05:51:04.315460+0000","last_became_peered":"2024-05-08T05:51:04.315460+0000","last_unstale":"2024-05-08T05:53:07.032236+0000","last_undegraded":"2024-05-08T05:53:07.032236+0000","last_fullsized":"2024-05-08T05:53:07.032236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:22:20.412368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715439+0000","last_change":"2024-05-08T05:51:04.297401+0000","last_active":"2024-05-08T05:53:05.715439+0000","last_peered":"2024-05-08T05:53:05.715439+0000","last_clean":"2024-05-08T05:53:05.715439+0000","last_became_active":"2024-05-08T05:51:04.297061+0000","last_became_peered":"2024-05-08T05:51:04.297061+0000","last_unstale":"2024-05-08T05:53:05.715439+0000","last_undegraded":"2024-05-08T05:53:05.715439+0000","last_fullsized":"2024-05-08T05:53:05.715439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:27.967517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268821+0000","last_change":"2024-05-08T05:51:04.315069+0000","last_active":"2024-05-08T05:53:05.268821+0000","last_peered":"2024-05-08T05:53:05.268821+0000","last_clean":"2024-05-08T05:53:05.268821+0000","last_became_active":"2024-05-08T05:51:04.314513+0000","last_became_peered":"2024-05-08T05:51:04.314513+0000","last_unstale":"2024-05-08T05:53:05.268821+0000","last_undegraded":"2024-05-08T05:53:05.268821+0000","last_fullsized":"2024-05-08T05:53:05.268821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:02.520130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265219+0000","last_change":"2024-05-08T05:51:04.302851+0000","last_active":"2024-05-08T05:53:05.265219+0000","last_peered":"2024-05-08T05:53:05.265219+0000","last_clean":"2024-05-08T05:53:05.265219+0000","last_became_active":"2024-05-08T05:51:04.302256+0000","last_became_peered":"2024-05-08T05:51:04.302256+0000","last_unstale":"2024-05-08T05:53:05.265219+0000","last_undegraded":"2024-05-08T05:53:05.265219+0000","last_fullsized":"2024-05-08T05:53:05.265219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:45.587319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139305+0000","last_change":"2024-05-08T05:51:04.308674+0000","last_active":"2024-05-08T05:53:07.139305+0000","last_peered":"2024-05-08T05:53:07.139305+0000","last_clean":"2024-05-08T05:53:07.139305+0000","last_became_active":"2024-05-08T05:51:04.308295+0000","last_became_peered":"2024-05-08T05:51:04.308295+0000","last_unstale":"2024-05-08T05:53:07.139305+0000","last_undegraded":"2024-05-08T05:53:07.139305+0000","last_fullsized":"2024-05-08T05:53:07.139305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:59:17.561655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571645+0000","last_change":"2024-05-08T05:51:04.314511+0000","last_active":"2024-05-08T05:53:05.571645+0000","last_peered":"2024-05-08T05:53:05.571645+0000","last_clean":"2024-05-08T05:53:05.571645+0000","last_became_active":"2024-05-08T05:51:04.314082+0000","last_became_peered":"2024-05-08T05:51:04.314082+0000","last_unstale":"2024-05-08T05:53:05.571645+0000","last_undegraded":"2024-05-08T05:53:05.571645+0000","last_fullsized":"2024-05-08T05:53:05.571645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:24.182391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269793+0000","last_change":"2024-05-08T05:51:04.293745+0000","last_active":"2024-05-08T05:53:05.269793+0000","last_peered":"2024-05-08T05:53:05.269793+0000","last_clean":"2024-05-08T05:53:05.269793+0000","last_became_active":"2024-05-08T05:51:04.293334+0000","last_became_peered":"2024-05-08T05:51:04.293334+0000","last_unstale":"2024-05-08T05:53:05.269793+0000","last_undegraded":"2024-05-08T05:53:05.269793+0000","last_fullsized":"2024-05-08T05:53:05.269793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:16.576967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265481+0000","last_change":"2024-05-08T05:51:04.297800+0000","last_active":"2024-05-08T05:53:05.265481+0000","last_peered":"2024-05-08T05:53:05.265481+0000","last_clean":"2024-05-08T05:53:05.265481+0000","last_became_active":"2024-05-08T05:51:04.297349+0000","last_became_peered":"2024-05-08T05:51:04.297349+0000","last_unstale":"2024-05-08T05:53:05.265481+0000","last_undegraded":"2024-05-08T05:53:05.265481+0000","last_fullsized":"2024-05-08T05:53:05.265481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:55.784512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139645+0000","last_change":"2024-05-08T05:51:04.297997+0000","last_active":"2024-05-08T05:53:07.139645+0000","last_peered":"2024-05-08T05:53:07.139645+0000","last_clean":"2024-05-08T05:53:07.139645+0000","last_became_active":"2024-05-08T05:51:04.297516+0000","last_became_peered":"2024-05-08T05:51:04.297516+0000","last_unstale":"2024-05-08T05:53:07.139645+0000","last_undegraded":"2024-05-08T05:53:07.139645+0000","last_fullsized":"2024-05-08T05:53:07.139645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:25.991555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715483+0000","last_change":"2024-05-08T05:51:04.295610+0000","last_active":"2024-05-08T05:53:05.715483+0000","last_peered":"2024-05-08T05:53:05.715483+0000","last_clean":"2024-05-08T05:53:05.715483+0000","last_became_active":"2024-05-08T05:51:04.294553+0000","last_became_peered":"2024-05-08T05:51:04.294553+0000","last_unstale":"2024-05-08T05:53:05.715483+0000","last_undegraded":"2024-05-08T05:53:05.715483+0000","last_fullsized":"2024-05-08T05:53:05.715483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:40:14.492057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820666+0000","last_change":"2024-05-08T05:51:04.303879+0000","last_active":"2024-05-08T05:53:05.820666+0000","last_peered":"2024-05-08T05:53:05.820666+0000","last_clean":"2024-05-08T05:53:05.820666+0000","last_became_active":"2024-05-08T05:51:04.303423+0000","last_became_peered":"2024-05-08T05:51:04.303423+0000","last_unstale":"2024-05-08T05:53:05.820666+0000","last_undegraded":"2024-05-08T05:53:05.820666+0000","last_fullsized":"2024-05-08T05:53:05.820666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:50:55.905858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.031822+0000","last_change":"2024-05-08T05:51:04.309818+0000","last_active":"2024-05-08T05:53:07.031822+0000","last_peered":"2024-05-08T05:53:07.031822+0000","last_clean":"2024-05-08T05:53:07.031822+0000","last_became_active":"2024-05-08T05:51:04.309434+0000","last_became_peered":"2024-05-08T05:51:04.309434+0000","last_unstale":"2024-05-08T05:53:07.031822+0000","last_undegraded":"2024-05-08T05:53:07.031822+0000","last_fullsized":"2024-05-08T05:53:07.031822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:55.832239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.031573+0000","last_change":"2024-05-08T05:51:04.316708+0000","last_active":"2024-05-08T05:53:07.031573+0000","last_peered":"2024-05-08T05:53:07.031573+0000","last_clean":"2024-05-08T05:53:07.031573+0000","last_became_active":"2024-05-08T05:51:04.316525+0000","last_became_peered":"2024-05-08T05:51:04.316525+0000","last_unstale":"2024-05-08T05:53:07.031573+0000","last_undegraded":"2024-05-08T05:53:07.031573+0000","last_fullsized":"2024-05-08T05:53:07.031573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:33:38.246611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269307+0000","last_change":"2024-05-08T05:51:04.314780+0000","last_active":"2024-05-08T05:53:05.269307+0000","last_peered":"2024-05-08T05:53:05.269307+0000","last_clean":"2024-05-08T05:53:05.269307+0000","last_became_active":"2024-05-08T05:51:04.314406+0000","last_became_peered":"2024-05-08T05:51:04.314406+0000","last_unstale":"2024-05-08T05:53:05.269307+0000","last_undegraded":"2024-05-08T05:53:05.269307+0000","last_fullsized":"2024-05-08T05:53:05.269307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:39:26.926238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579947+0000","last_change":"2024-05-08T05:51:04.319425+0000","last_active":"2024-05-08T05:53:07.579947+0000","last_peered":"2024-05-08T05:53:07.579947+0000","last_clean":"2024-05-08T05:53:07.579947+0000","last_became_active":"2024-05-08T05:51:04.319106+0000","last_became_peered":"2024-05-08T05:51:04.319106+0000","last_unstale":"2024-05-08T05:53:07.579947+0000","last_undegraded":"2024-05-08T05:53:07.579947+0000","last_fullsized":"2024-05-08T05:53:07.579947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:59:54.023162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267374+0000","last_change":"2024-05-08T05:51:04.303547+0000","last_active":"2024-05-08T05:53:05.267374+0000","last_peered":"2024-05-08T05:53:05.267374+0000","last_clean":"2024-05-08T05:53:05.267374+0000","last_became_active":"2024-05-08T05:51:04.303182+0000","last_became_peered":"2024-05-08T05:51:04.303182+0000","last_unstale":"2024-05-08T05:53:05.267374+0000","last_undegraded":"2024-05-08T05:53:05.267374+0000","last_fullsized":"2024-05-08T05:53:05.267374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:11:59.624329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505589+0000","last_change":"2024-05-08T05:51:04.315674+0000","last_active":"2024-05-08T05:53:05.505589+0000","last_peered":"2024-05-08T05:53:05.505589+0000","last_clean":"2024-05-08T05:53:05.505589+0000","last_became_active":"2024-05-08T05:51:04.314911+0000","last_became_peered":"2024-05-08T05:51:04.314911+0000","last_unstale":"2024-05-08T05:53:05.505589+0000","last_undegraded":"2024-05-08T05:53:05.505589+0000","last_fullsized":"2024-05-08T05:53:05.505589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:49.501508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267963+0000","last_change":"2024-05-08T05:51:00.273554+0000","last_active":"2024-05-08T05:53:05.267963+0000","last_peered":"2024-05-08T05:53:05.267963+0000","last_clean":"2024-05-08T05:53:05.267963+0000","last_became_active":"2024-05-08T05:51:00.272858+0000","last_became_peered":"2024-05-08T05:51:00.272858+0000","last_unstale":"2024-05-08T05:53:05.267963+0000","last_undegraded":"2024-05-08T05:53:05.267963+0000","last_fullsized":"2024-05-08T05:53:05.267963+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:47:37.418887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572441+0000","last_change":"2024-05-08T05:51:04.306392+0000","last_active":"2024-05-08T05:53:05.572441+0000","last_peered":"2024-05-08T05:53:05.572441+0000","last_clean":"2024-05-08T05:53:05.572441+0000","last_became_active":"2024-05-08T05:51:04.304248+0000","last_became_peered":"2024-05-08T05:51:04.304248+0000","last_unstale":"2024-05-08T05:53:05.572441+0000","last_undegraded":"2024-05-08T05:53:05.572441+0000","last_fullsized":"2024-05-08T05:53:05.572441+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:49:26.034883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572774+0000","last_change":"2024-05-08T05:51:01.187292+0000","last_active":"2024-05-08T05:53:05.572774+0000","last_peered":"2024-05-08T05:53:05.572774+0000","last_clean":"2024-05-08T05:53:05.572774+0000","last_became_active":"2024-05-08T05:51:01.185948+0000","last_became_peered":"2024-05-08T05:51:01.185948+0000","last_unstale":"2024-05-08T05:53:05.572774+0000","last_undegraded":"2024-05-08T05:53:05.572774+0000","last_fullsized":"2024-05-08T05:53:05.572774+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:02.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032756+0000","last_change":"2024-05-08T05:51:04.318439+0000","last_active":"2024-05-08T05:53:07.032756+0000","last_peered":"2024-05-08T05:53:07.032756+0000","last_clean":"2024-05-08T05:53:07.032756+0000","last_became_active":"2024-05-08T05:51:04.317722+0000","last_became_peered":"2024-05-08T05:51:04.317722+0000","last_unstale":"2024-05-08T05:53:07.032756+0000","last_undegraded":"2024-05-08T05:53:07.032756+0000","last_fullsized":"2024-05-08T05:53:07.032756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:36:43.648643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269296+0000","last_change":"2024-05-08T05:51:00.269684+0000","last_active":"2024-05-08T05:53:05.269296+0000","last_peered":"2024-05-08T05:53:05.269296+0000","last_clean":"2024-05-08T05:53:05.269296+0000","last_became_active":"2024-05-08T05:51:00.269329+0000","last_became_peered":"2024-05-08T05:51:00.269329+0000","last_unstale":"2024-05-08T05:53:05.269296+0000","last_undegraded":"2024-05-08T05:53:05.269296+0000","last_fullsized":"2024-05-08T05:53:05.269296+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:27:10.422994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269410+0000","last_change":"2024-05-08T05:51:04.308048+0000","last_active":"2024-05-08T05:53:05.269410+0000","last_peered":"2024-05-08T05:53:05.269410+0000","last_clean":"2024-05-08T05:53:05.269410+0000","last_became_active":"2024-05-08T05:51:04.307550+0000","last_became_peered":"2024-05-08T05:51:04.307550+0000","last_unstale":"2024-05-08T05:53:05.269410+0000","last_undegraded":"2024-05-08T05:53:05.269410+0000","last_fullsized":"2024-05-08T05:53:05.269410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:13.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269033+0000","last_change":"2024-05-08T05:51:00.268142+0000","last_active":"2024-05-08T05:53:05.269033+0000","last_peered":"2024-05-08T05:53:05.269033+0000","last_clean":"2024-05-08T05:53:05.269033+0000","last_became_active":"2024-05-08T05:51:00.267756+0000","last_became_peered":"2024-05-08T05:51:00.267756+0000","last_unstale":"2024-05-08T05:53:05.269033+0000","last_undegraded":"2024-05-08T05:53:05.269033+0000","last_fullsized":"2024-05-08T05:53:05.269033+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:01.439680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270314+0000","last_change":"2024-05-08T05:51:04.292341+0000","last_active":"2024-05-08T05:53:05.270314+0000","last_peered":"2024-05-08T05:53:05.270314+0000","last_clean":"2024-05-08T05:53:05.270314+0000","last_became_active":"2024-05-08T05:51:04.291086+0000","last_became_peered":"2024-05-08T05:51:04.291086+0000","last_unstale":"2024-05-08T05:53:05.270314+0000","last_undegraded":"2024-05-08T05:53:05.270314+0000","last_fullsized":"2024-05-08T05:53:05.270314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:47.108100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":252,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:54:57.975810+0000","last_change":"2024-05-08T05:51:00.268100+0000","last_active":"2024-05-08T05:54:57.975810+0000","last_peered":"2024-05-08T05:54:57.975810+0000","last_clean":"2024-05-08T05:54:57.975810+0000","last_became_active":"2024-05-08T05:51:00.267678+0000","last_became_peered":"2024-05-08T05:51:00.267678+0000","last_unstale":"2024-05-08T05:54:57.975810+0000","last_undegraded":"2024-05-08T05:54:57.975810+0000","last_fullsized":"2024-05-08T05:54:57.975810+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:11:31.731548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":215,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268766+0000","last_change":"2024-05-08T05:51:04.303872+0000","last_active":"2024-05-08T05:53:05.268766+0000","last_peered":"2024-05-08T05:53:05.268766+0000","last_clean":"2024-05-08T05:53:05.268766+0000","last_became_active":"2024-05-08T05:51:04.303456+0000","last_became_peered":"2024-05-08T05:51:04.303456+0000","last_unstale":"2024-05-08T05:53:05.268766+0000","last_undegraded":"2024-05-08T05:53:05.268766+0000","last_fullsized":"2024-05-08T05:53:05.268766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:42:05.098488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265391+0000","last_change":"2024-05-08T05:51:00.275657+0000","last_active":"2024-05-08T05:53:05.265391+0000","last_peered":"2024-05-08T05:53:05.265391+0000","last_clean":"2024-05-08T05:53:05.265391+0000","last_became_active":"2024-05-08T05:51:00.275254+0000","last_became_peered":"2024-05-08T05:51:00.275254+0000","last_unstale":"2024-05-08T05:53:05.265391+0000","last_undegraded":"2024-05-08T05:53:05.265391+0000","last_fullsized":"2024-05-08T05:53:05.265391+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:28.555698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270118+0000","last_change":"2024-05-08T05:51:04.293791+0000","last_active":"2024-05-08T05:53:05.270118+0000","last_peered":"2024-05-08T05:53:05.270118+0000","last_clean":"2024-05-08T05:53:05.270118+0000","last_became_active":"2024-05-08T05:51:04.293405+0000","last_became_peered":"2024-05-08T05:51:04.293405+0000","last_unstale":"2024-05-08T05:53:05.270118+0000","last_undegraded":"2024-05-08T05:53:05.270118+0000","last_fullsized":"2024-05-08T05:53:05.270118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:28.064452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820147+0000","last_change":"2024-05-08T05:51:00.280295+0000","last_active":"2024-05-08T05:53:05.820147+0000","last_peered":"2024-05-08T05:53:05.820147+0000","last_clean":"2024-05-08T05:53:05.820147+0000","last_became_active":"2024-05-08T05:51:00.279721+0000","last_became_peered":"2024-05-08T05:51:00.279721+0000","last_unstale":"2024-05-08T05:53:05.820147+0000","last_undegraded":"2024-05-08T05:53:05.820147+0000","last_fullsized":"2024-05-08T05:53:05.820147+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:06:38.064872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571814+0000","last_change":"2024-05-08T05:51:04.307546+0000","last_active":"2024-05-08T05:53:05.571814+0000","last_peered":"2024-05-08T05:53:05.571814+0000","last_clean":"2024-05-08T05:53:05.571814+0000","last_became_active":"2024-05-08T05:51:04.307164+0000","last_became_peered":"2024-05-08T05:51:04.307164+0000","last_unstale":"2024-05-08T05:53:05.571814+0000","last_undegraded":"2024-05-08T05:53:05.571814+0000","last_fullsized":"2024-05-08T05:53:05.571814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:36:02.269040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269777+0000","last_change":"2024-05-08T05:51:00.267987+0000","last_active":"2024-05-08T05:53:05.269777+0000","last_peered":"2024-05-08T05:53:05.269777+0000","last_clean":"2024-05-08T05:53:05.269777+0000","last_became_active":"2024-05-08T05:51:00.267633+0000","last_became_peered":"2024-05-08T05:51:00.267633+0000","last_unstale":"2024-05-08T05:53:05.269777+0000","last_undegraded":"2024-05-08T05:53:05.269777+0000","last_fullsized":"2024-05-08T05:53:05.269777+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:13.034300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.266492+0000","last_change":"2024-05-08T05:51:04.311422+0000","last_active":"2024-05-08T05:53:05.266492+0000","last_peered":"2024-05-08T05:53:05.266492+0000","last_clean":"2024-05-08T05:53:05.266492+0000","last_became_active":"2024-05-08T05:51:04.311052+0000","last_became_peered":"2024-05-08T05:51:04.311052+0000","last_unstale":"2024-05-08T05:53:05.266492+0000","last_undegraded":"2024-05-08T05:53:05.266492+0000","last_fullsized":"2024-05-08T05:53:05.266492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:47.790583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":246,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:54:57.974211+0000","last_change":"2024-05-08T05:51:00.274252+0000","last_active":"2024-05-08T05:54:57.974211+0000","last_peered":"2024-05-08T05:54:57.974211+0000","last_clean":"2024-05-08T05:54:57.974211+0000","last_became_active":"2024-05-08T05:51:00.273942+0000","last_became_peered":"2024-05-08T05:51:00.273942+0000","last_unstale":"2024-05-08T05:54:57.974211+0000","last_undegraded":"2024-05-08T05:54:57.974211+0000","last_fullsized":"2024-05-08T05:54:57.974211+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:36.778032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":212,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269700+0000","last_change":"2024-05-08T05:51:04.296414+0000","last_active":"2024-05-08T05:53:05.269700+0000","last_peered":"2024-05-08T05:53:05.269700+0000","last_clean":"2024-05-08T05:53:05.269700+0000","last_became_active":"2024-05-08T05:51:04.296015+0000","last_became_peered":"2024-05-08T05:51:04.296015+0000","last_unstale":"2024-05-08T05:53:05.269700+0000","last_undegraded":"2024-05-08T05:53:05.269700+0000","last_fullsized":"2024-05-08T05:53:05.269700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:48:15.609385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139516+0000","last_change":"2024-05-08T05:51:00.268234+0000","last_active":"2024-05-08T05:53:07.139516+0000","last_peered":"2024-05-08T05:53:07.139516+0000","last_clean":"2024-05-08T05:53:07.139516+0000","last_became_active":"2024-05-08T05:51:00.267846+0000","last_became_peered":"2024-05-08T05:51:00.267846+0000","last_unstale":"2024-05-08T05:53:07.139516+0000","last_undegraded":"2024-05-08T05:53:07.139516+0000","last_fullsized":"2024-05-08T05:53:07.139516+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:26.919004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032653+0000","last_change":"2024-05-08T05:51:04.318215+0000","last_active":"2024-05-08T05:53:07.032653+0000","last_peered":"2024-05-08T05:53:07.032653+0000","last_clean":"2024-05-08T05:53:07.032653+0000","last_became_active":"2024-05-08T05:51:04.317309+0000","last_became_peered":"2024-05-08T05:51:04.317309+0000","last_unstale":"2024-05-08T05:53:07.032653+0000","last_undegraded":"2024-05-08T05:53:07.032653+0000","last_fullsized":"2024-05-08T05:53:07.032653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:11.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270484+0000","last_change":"2024-05-08T05:51:01.186438+0000","last_active":"2024-05-08T05:53:05.270484+0000","last_peered":"2024-05-08T05:53:05.270484+0000","last_clean":"2024-05-08T05:53:05.270484+0000","last_became_active":"2024-05-08T05:51:01.186039+0000","last_became_peered":"2024-05-08T05:51:01.186039+0000","last_unstale":"2024-05-08T05:53:05.270484+0000","last_undegraded":"2024-05-08T05:53:05.270484+0000","last_fullsized":"2024-05-08T05:53:05.270484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:34.652184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269557+0000","last_change":"2024-05-08T05:52:05.575531+0000","last_active":"2024-05-08T05:53:05.269557+0000","last_peered":"2024-05-08T05:53:05.269557+0000","last_clean":"2024-05-08T05:53:05.269557+0000","last_became_active":"2024-05-08T05:52:05.575042+0000","last_became_peered":"2024-05-08T05:52:05.575042+0000","last_unstale":"2024-05-08T05:53:05.269557+0000","last_undegraded":"2024-05-08T05:53:05.269557+0000","last_fullsized":"2024-05-08T05:53:05.269557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:56.096772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'36","reported_seq":502,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:54:58.977550+0000","last_change":"2024-05-08T05:51:00.263385+0000","last_active":"2024-05-08T05:54:58.977550+0000","last_peered":"2024-05-08T05:54:58.977550+0000","last_clean":"2024-05-08T05:54:58.977550+0000","last_became_active":"2024-05-08T05:51:00.262997+0000","last_became_peered":"2024-05-08T05:51:00.262997+0000","last_unstale":"2024-05-08T05:54:58.977550+0000","last_undegraded":"2024-05-08T05:54:58.977550+0000","last_fullsized":"2024-05-08T05:54:58.977550+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:22:09.727481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66623,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":436,"num_read_kb":283,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269140+0000","last_change":"2024-05-08T05:51:04.294790+0000","last_active":"2024-05-08T05:53:05.269140+0000","last_peered":"2024-05-08T05:53:05.269140+0000","last_clean":"2024-05-08T05:53:05.269140+0000","last_became_active":"2024-05-08T05:51:04.294418+0000","last_became_peered":"2024-05-08T05:51:04.294418+0000","last_unstale":"2024-05-08T05:53:05.269140+0000","last_undegraded":"2024-05-08T05:53:05.269140+0000","last_fullsized":"2024-05-08T05:53:05.269140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:11:10.624434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505362+0000","last_change":"2024-05-08T05:51:00.258507+0000","last_active":"2024-05-08T05:53:05.505362+0000","last_peered":"2024-05-08T05:53:05.505362+0000","last_clean":"2024-05-08T05:53:05.505362+0000","last_became_active":"2024-05-08T05:51:00.258179+0000","last_became_peered":"2024-05-08T05:51:00.258179+0000","last_unstale":"2024-05-08T05:53:05.505362+0000","last_undegraded":"2024-05-08T05:53:05.505362+0000","last_fullsized":"2024-05-08T05:53:05.505362+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:22.376499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820379+0000","last_change":"2024-05-08T05:51:04.310947+0000","last_active":"2024-05-08T05:53:05.820379+0000","last_peered":"2024-05-08T05:53:05.820379+0000","last_clean":"2024-05-08T05:53:05.820379+0000","last_became_active":"2024-05-08T05:51:04.310378+0000","last_became_peered":"2024-05-08T05:51:04.310378+0000","last_unstale":"2024-05-08T05:53:05.820379+0000","last_undegraded":"2024-05-08T05:53:05.820379+0000","last_fullsized":"2024-05-08T05:53:05.820379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:19.367660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270609+0000","last_change":"2024-05-08T05:51:00.271104+0000","last_active":"2024-05-08T05:53:05.270609+0000","last_peered":"2024-05-08T05:53:05.270609+0000","last_clean":"2024-05-08T05:53:05.270609+0000","last_became_active":"2024-05-08T05:51:00.270497+0000","last_became_peered":"2024-05-08T05:51:00.270497+0000","last_unstale":"2024-05-08T05:53:05.270609+0000","last_undegraded":"2024-05-08T05:53:05.270609+0000","last_fullsized":"2024-05-08T05:53:05.270609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:40.959423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139512+0000","last_change":"2024-05-08T05:51:04.310243+0000","last_active":"2024-05-08T05:53:07.139512+0000","last_peered":"2024-05-08T05:53:07.139512+0000","last_clean":"2024-05-08T05:53:07.139512+0000","last_became_active":"2024-05-08T05:51:04.309871+0000","last_became_peered":"2024-05-08T05:51:04.309871+0000","last_unstale":"2024-05-08T05:53:07.139512+0000","last_undegraded":"2024-05-08T05:53:07.139512+0000","last_fullsized":"2024-05-08T05:53:07.139512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:18:21.045621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":251,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:54:57.082440+0000","last_change":"2024-05-08T05:51:00.266096+0000","last_active":"2024-05-08T05:54:57.082440+0000","last_peered":"2024-05-08T05:54:57.082440+0000","last_clean":"2024-05-08T05:54:57.082440+0000","last_became_active":"2024-05-08T05:51:00.265706+0000","last_became_peered":"2024-05-08T05:51:00.265706+0000","last_unstale":"2024-05-08T05:54:57.082440+0000","last_undegraded":"2024-05-08T05:54:57.082440+0000","last_fullsized":"2024-05-08T05:54:57.082440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:47.388732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":216,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139267+0000","last_change":"2024-05-08T05:51:04.293141+0000","last_active":"2024-05-08T05:53:07.139267+0000","last_peered":"2024-05-08T05:53:07.139267+0000","last_clean":"2024-05-08T05:53:07.139267+0000","last_became_active":"2024-05-08T05:51:04.292772+0000","last_became_peered":"2024-05-08T05:51:04.292772+0000","last_unstale":"2024-05-08T05:53:07.139267+0000","last_undegraded":"2024-05-08T05:53:07.139267+0000","last_fullsized":"2024-05-08T05:53:07.139267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:25:32.801209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268837+0000","last_change":"2024-05-08T05:52:05.042102+0000","last_active":"2024-05-08T05:53:05.268837+0000","last_peered":"2024-05-08T05:53:05.268837+0000","last_clean":"2024-05-08T05:53:05.268837+0000","last_became_active":"2024-05-08T05:52:05.027787+0000","last_became_peered":"2024-05-08T05:52:05.027787+0000","last_unstale":"2024-05-08T05:53:05.268837+0000","last_undegraded":"2024-05-08T05:53:05.268837+0000","last_fullsized":"2024-05-08T05:53:05.268837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:05.580309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269940+0000","last_change":"2024-05-08T05:51:04.290096+0000","last_active":"2024-05-08T05:53:05.269940+0000","last_peered":"2024-05-08T05:53:05.269940+0000","last_clean":"2024-05-08T05:53:05.269940+0000","last_became_active":"2024-05-08T05:51:04.289684+0000","last_became_peered":"2024-05-08T05:51:04.289684+0000","last_unstale":"2024-05-08T05:53:05.269940+0000","last_undegraded":"2024-05-08T05:53:05.269940+0000","last_fullsized":"2024-05-08T05:53:05.269940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:02.400348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579795+0000","last_change":"2024-05-08T05:52:05.034004+0000","last_active":"2024-05-08T05:53:07.579795+0000","last_peered":"2024-05-08T05:53:07.579795+0000","last_clean":"2024-05-08T05:53:07.579795+0000","last_became_active":"2024-05-08T05:52:05.023497+0000","last_became_peered":"2024-05-08T05:52:05.023497+0000","last_unstale":"2024-05-08T05:53:07.579795+0000","last_undegraded":"2024-05-08T05:53:07.579795+0000","last_fullsized":"2024-05-08T05:53:07.579795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:06.462940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579850+0000","last_change":"2024-05-08T05:52:05.027648+0000","last_active":"2024-05-08T05:53:07.579850+0000","last_peered":"2024-05-08T05:53:07.579850+0000","last_clean":"2024-05-08T05:53:07.579850+0000","last_became_active":"2024-05-08T05:52:05.027262+0000","last_became_peered":"2024-05-08T05:52:05.027262+0000","last_unstale":"2024-05-08T05:53:07.579850+0000","last_undegraded":"2024-05-08T05:53:07.579850+0000","last_fullsized":"2024-05-08T05:53:07.579850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:36:18.588967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267116+0000","last_change":"2024-05-08T05:51:01.189324+0000","last_active":"2024-05-08T05:53:05.267116+0000","last_peered":"2024-05-08T05:53:05.267116+0000","last_clean":"2024-05-08T05:53:05.267116+0000","last_became_active":"2024-05-08T05:51:01.188651+0000","last_became_peered":"2024-05-08T05:51:01.188651+0000","last_unstale":"2024-05-08T05:53:05.267116+0000","last_undegraded":"2024-05-08T05:53:05.267116+0000","last_fullsized":"2024-05-08T05:53:05.267116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:45.149213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265294+0000","last_change":"2024-05-08T05:51:04.306811+0000","last_active":"2024-05-08T05:53:05.265294+0000","last_peered":"2024-05-08T05:53:05.265294+0000","last_clean":"2024-05-08T05:53:05.265294+0000","last_became_active":"2024-05-08T05:51:04.306423+0000","last_became_peered":"2024-05-08T05:51:04.306423+0000","last_unstale":"2024-05-08T05:53:05.265294+0000","last_undegraded":"2024-05-08T05:53:05.265294+0000","last_fullsized":"2024-05-08T05:53:05.265294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:08.830682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139525+0000","last_change":"2024-05-08T05:51:00.276543+0000","last_active":"2024-05-08T05:53:07.139525+0000","last_peered":"2024-05-08T05:53:07.139525+0000","last_clean":"2024-05-08T05:53:07.139525+0000","last_became_active":"2024-05-08T05:51:00.276233+0000","last_became_peered":"2024-05-08T05:51:00.276233+0000","last_unstale":"2024-05-08T05:53:07.139525+0000","last_undegraded":"2024-05-08T05:53:07.139525+0000","last_fullsized":"2024-05-08T05:53:07.139525+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:11.229942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265173+0000","last_change":"2024-05-08T05:51:04.322033+0000","last_active":"2024-05-08T05:53:05.265173+0000","last_peered":"2024-05-08T05:53:05.265173+0000","last_clean":"2024-05-08T05:53:05.265173+0000","last_became_active":"2024-05-08T05:51:04.321555+0000","last_became_peered":"2024-05-08T05:51:04.321555+0000","last_unstale":"2024-05-08T05:53:05.265173+0000","last_undegraded":"2024-05-08T05:53:05.265173+0000","last_fullsized":"2024-05-08T05:53:05.265173+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:21:50.820709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.266462+0000","last_change":"2024-05-08T05:52:05.027723+0000","last_active":"2024-05-08T05:53:05.266462+0000","last_peered":"2024-05-08T05:53:05.266462+0000","last_clean":"2024-05-08T05:53:05.266462+0000","last_became_active":"2024-05-08T05:52:05.026508+0000","last_became_peered":"2024-05-08T05:52:05.026508+0000","last_unstale":"2024-05-08T05:53:05.266462+0000","last_undegraded":"2024-05-08T05:53:05.266462+0000","last_fullsized":"2024-05-08T05:53:05.266462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:14:32.399433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":250,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:55:00.087804+0000","last_change":"2024-05-08T05:51:01.187556+0000","last_active":"2024-05-08T05:55:00.087804+0000","last_peered":"2024-05-08T05:55:00.087804+0000","last_clean":"2024-05-08T05:55:00.087804+0000","last_became_active":"2024-05-08T05:51:01.186584+0000","last_became_peered":"2024-05-08T05:51:01.186584+0000","last_unstale":"2024-05-08T05:55:00.087804+0000","last_undegraded":"2024-05-08T05:55:00.087804+0000","last_fullsized":"2024-05-08T05:55:00.087804+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:29.036633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.274091+0000","last_change":"2024-05-08T05:53:05.272875+0000","last_active":"2024-05-08T05:53:05.274091+0000","last_peered":"2024-05-08T05:53:05.274091+0000","last_clean":"2024-05-08T05:53:05.274091+0000","last_became_active":"2024-05-08T05:53:05.271672+0000","last_became_peered":"2024-05-08T05:53:05.271672+0000","last_unstale":"2024-05-08T05:53:05.274091+0000","last_undegraded":"2024-05-08T05:53:05.274091+0000","last_fullsized":"2024-05-08T05:53:05.274091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:08:28.776982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715396+0000","last_change":"2024-05-08T05:51:00.272501+0000","last_active":"2024-05-08T05:53:05.715396+0000","last_peered":"2024-05-08T05:53:05.715396+0000","last_clean":"2024-05-08T05:53:05.715396+0000","last_became_active":"2024-05-08T05:51:00.272166+0000","last_became_peered":"2024-05-08T05:51:00.272166+0000","last_unstale":"2024-05-08T05:53:05.715396+0000","last_undegraded":"2024-05-08T05:53:05.715396+0000","last_fullsized":"2024-05-08T05:53:05.715396+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:30.980047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":104,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139425+0000","last_change":"2024-05-08T05:49:31.820799+0000","last_active":"2024-05-08T05:53:07.139425+0000","last_peered":"2024-05-08T05:53:07.139425+0000","last_clean":"2024-05-08T05:53:07.139425+0000","last_became_active":"2024-05-08T05:49:31.815568+0000","last_became_peered":"2024-05-08T05:49:31.815568+0000","last_unstale":"2024-05-08T05:53:07.139425+0000","last_undegraded":"2024-05-08T05:53:07.139425+0000","last_fullsized":"2024-05-08T05:53:07.139425+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:48.170242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:48.170242+0000","last_clean_scrub_stamp":"2024-05-08T05:45:48.170242+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:39.527036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820012+0000","last_change":"2024-05-08T05:51:04.301375+0000","last_active":"2024-05-08T05:53:05.820012+0000","last_peered":"2024-05-08T05:53:05.820012+0000","last_clean":"2024-05-08T05:53:05.820012+0000","last_became_active":"2024-05-08T05:51:04.301027+0000","last_became_peered":"2024-05-08T05:51:04.301027+0000","last_unstale":"2024-05-08T05:53:05.820012+0000","last_undegraded":"2024-05-08T05:53:05.820012+0000","last_fullsized":"2024-05-08T05:53:05.820012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:21.393242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579619+0000","last_change":"2024-05-08T05:52:05.037427+0000","last_active":"2024-05-08T05:53:07.579619+0000","last_peered":"2024-05-08T05:53:07.579619+0000","last_clean":"2024-05-08T05:53:07.579619+0000","last_became_active":"2024-05-08T05:52:05.023644+0000","last_became_peered":"2024-05-08T05:52:05.023644+0000","last_unstale":"2024-05-08T05:53:07.579619+0000","last_undegraded":"2024-05-08T05:53:07.579619+0000","last_fullsized":"2024-05-08T05:53:07.579619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:49.193375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269166+0000","last_change":"2024-05-08T05:51:01.187171+0000","last_active":"2024-05-08T05:53:05.269166+0000","last_peered":"2024-05-08T05:53:05.269166+0000","last_clean":"2024-05-08T05:53:05.269166+0000","last_became_active":"2024-05-08T05:51:01.186142+0000","last_became_peered":"2024-05-08T05:51:01.186142+0000","last_unstale":"2024-05-08T05:53:05.269166+0000","last_undegraded":"2024-05-08T05:53:05.269166+0000","last_fullsized":"2024-05-08T05:53:05.269166+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:37:17.021905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820912+0000","last_change":"2024-05-08T05:51:04.311538+0000","last_active":"2024-05-08T05:53:05.820912+0000","last_peered":"2024-05-08T05:53:05.820912+0000","last_clean":"2024-05-08T05:53:05.820912+0000","last_became_active":"2024-05-08T05:51:04.310535+0000","last_became_peered":"2024-05-08T05:51:04.310535+0000","last_unstale":"2024-05-08T05:53:05.820912+0000","last_undegraded":"2024-05-08T05:53:05.820912+0000","last_fullsized":"2024-05-08T05:53:05.820912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:01:04.497986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'13","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:13.203771+0000","last_change":"2024-05-08T05:51:00.277923+0000","last_active":"2024-05-08T05:53:13.203771+0000","last_peered":"2024-05-08T05:53:13.203771+0000","last_clean":"2024-05-08T05:53:13.203771+0000","last_became_active":"2024-05-08T05:51:00.274896+0000","last_became_peered":"2024-05-08T05:51:00.274896+0000","last_unstale":"2024-05-08T05:53:13.203771+0000","last_undegraded":"2024-05-08T05:53:13.203771+0000","last_fullsized":"2024-05-08T05:53:13.203771+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:04:32.543451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.573064+0000","last_change":"2024-05-08T05:51:04.306185+0000","last_active":"2024-05-08T05:53:05.573064+0000","last_peered":"2024-05-08T05:53:05.573064+0000","last_clean":"2024-05-08T05:53:05.573064+0000","last_became_active":"2024-05-08T05:51:04.305229+0000","last_became_peered":"2024-05-08T05:51:04.305229+0000","last_unstale":"2024-05-08T05:53:05.573064+0000","last_undegraded":"2024-05-08T05:53:05.573064+0000","last_fullsized":"2024-05-08T05:53:05.573064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:48.859978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715588+0000","last_change":"2024-05-08T05:51:01.140178+0000","last_active":"2024-05-08T05:53:05.715588+0000","last_peered":"2024-05-08T05:53:05.715588+0000","last_clean":"2024-05-08T05:53:05.715588+0000","last_became_active":"2024-05-08T05:51:01.139772+0000","last_became_peered":"2024-05-08T05:51:01.139772+0000","last_unstale":"2024-05-08T05:53:05.715588+0000","last_undegraded":"2024-05-08T05:53:05.715588+0000","last_fullsized":"2024-05-08T05:53:05.715588+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:54:40.132683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032654+0000","last_change":"2024-05-08T05:51:04.319015+0000","last_active":"2024-05-08T05:53:07.032654+0000","last_peered":"2024-05-08T05:53:07.032654+0000","last_clean":"2024-05-08T05:53:07.032654+0000","last_became_active":"2024-05-08T05:51:04.318465+0000","last_became_peered":"2024-05-08T05:51:04.318465+0000","last_unstale":"2024-05-08T05:53:07.032654+0000","last_undegraded":"2024-05-08T05:53:07.032654+0000","last_fullsized":"2024-05-08T05:53:07.032654+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:07:19.336656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269322+0000","last_change":"2024-05-08T05:51:01.185619+0000","last_active":"2024-05-08T05:53:05.269322+0000","last_peered":"2024-05-08T05:53:05.269322+0000","last_clean":"2024-05-08T05:53:05.269322+0000","last_became_active":"2024-05-08T05:51:01.185325+0000","last_became_peered":"2024-05-08T05:51:01.185325+0000","last_unstale":"2024-05-08T05:53:05.269322+0000","last_undegraded":"2024-05-08T05:53:05.269322+0000","last_fullsized":"2024-05-08T05:53:05.269322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:41.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268756+0000","last_change":"2024-05-08T05:51:04.295018+0000","last_active":"2024-05-08T05:53:05.268756+0000","last_peered":"2024-05-08T05:53:05.268756+0000","last_clean":"2024-05-08T05:53:05.268756+0000","last_became_active":"2024-05-08T05:51:04.294432+0000","last_became_peered":"2024-05-08T05:51:04.294432+0000","last_unstale":"2024-05-08T05:53:05.268756+0000","last_undegraded":"2024-05-08T05:53:05.268756+0000","last_fullsized":"2024-05-08T05:53:05.268756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:01.925456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571832+0000","last_change":"2024-05-08T05:51:00.280527+0000","last_active":"2024-05-08T05:53:05.571832+0000","last_peered":"2024-05-08T05:53:05.571832+0000","last_clean":"2024-05-08T05:53:05.571832+0000","last_became_active":"2024-05-08T05:51:00.278718+0000","last_became_peered":"2024-05-08T05:51:00.278718+0000","last_unstale":"2024-05-08T05:53:05.571832+0000","last_undegraded":"2024-05-08T05:53:05.571832+0000","last_fullsized":"2024-05-08T05:53:05.571832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:48:27.149713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579777+0000","last_change":"2024-05-08T05:51:04.314543+0000","last_active":"2024-05-08T05:53:07.579777+0000","last_peered":"2024-05-08T05:53:07.579777+0000","last_clean":"2024-05-08T05:53:07.579777+0000","last_became_active":"2024-05-08T05:51:04.314168+0000","last_became_peered":"2024-05-08T05:51:04.314168+0000","last_unstale":"2024-05-08T05:53:07.579777+0000","last_undegraded":"2024-05-08T05:53:07.579777+0000","last_fullsized":"2024-05-08T05:53:07.579777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:10:23.215322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:11.750983+0000","last_change":"2024-05-08T05:51:00.268081+0000","last_active":"2024-05-08T05:53:11.750983+0000","last_peered":"2024-05-08T05:53:11.750983+0000","last_clean":"2024-05-08T05:53:11.750983+0000","last_became_active":"2024-05-08T05:51:00.267470+0000","last_became_peered":"2024-05-08T05:51:00.267470+0000","last_unstale":"2024-05-08T05:53:11.750983+0000","last_undegraded":"2024-05-08T05:53:11.750983+0000","last_fullsized":"2024-05-08T05:53:11.750983+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:45.796019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572395+0000","last_change":"2024-05-08T05:51:04.322010+0000","last_active":"2024-05-08T05:53:05.572395+0000","last_peered":"2024-05-08T05:53:05.572395+0000","last_clean":"2024-05-08T05:53:05.572395+0000","last_became_active":"2024-05-08T05:51:04.320867+0000","last_became_peered":"2024-05-08T05:51:04.320867+0000","last_unstale":"2024-05-08T05:53:05.572395+0000","last_undegraded":"2024-05-08T05:53:05.572395+0000","last_fullsized":"2024-05-08T05:53:05.572395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:49.853518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268990+0000","last_change":"2024-05-08T05:52:05.045290+0000","last_active":"2024-05-08T05:53:05.268990+0000","last_peered":"2024-05-08T05:53:05.268990+0000","last_clean":"2024-05-08T05:53:05.268990+0000","last_became_active":"2024-05-08T05:52:05.029284+0000","last_became_peered":"2024-05-08T05:52:05.029284+0000","last_unstale":"2024-05-08T05:53:05.268990+0000","last_undegraded":"2024-05-08T05:53:05.268990+0000","last_fullsized":"2024-05-08T05:53:05.268990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:20.491862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139647+0000","last_change":"2024-05-08T05:51:04.302813+0000","last_active":"2024-05-08T05:53:07.139647+0000","last_peered":"2024-05-08T05:53:07.139647+0000","last_clean":"2024-05-08T05:53:07.139647+0000","last_became_active":"2024-05-08T05:51:04.302424+0000","last_became_peered":"2024-05-08T05:51:04.302424+0000","last_unstale":"2024-05-08T05:53:07.139647+0000","last_undegraded":"2024-05-08T05:53:07.139647+0000","last_fullsized":"2024-05-08T05:53:07.139647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:52:17.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505505+0000","last_change":"2024-05-08T05:51:01.187513+0000","last_active":"2024-05-08T05:53:05.505505+0000","last_peered":"2024-05-08T05:53:05.505505+0000","last_clean":"2024-05-08T05:53:05.505505+0000","last_became_active":"2024-05-08T05:51:01.186780+0000","last_became_peered":"2024-05-08T05:51:01.186780+0000","last_unstale":"2024-05-08T05:53:05.505505+0000","last_undegraded":"2024-05-08T05:53:05.505505+0000","last_fullsized":"2024-05-08T05:53:05.505505+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:42:26.206351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139833+0000","last_change":"2024-05-08T05:51:04.295338+0000","last_active":"2024-05-08T05:53:07.139833+0000","last_peered":"2024-05-08T05:53:07.139833+0000","last_clean":"2024-05-08T05:53:07.139833+0000","last_became_active":"2024-05-08T05:51:04.294990+0000","last_became_peered":"2024-05-08T05:51:04.294990+0000","last_unstale":"2024-05-08T05:53:07.139833+0000","last_undegraded":"2024-05-08T05:53:07.139833+0000","last_fullsized":"2024-05-08T05:53:07.139833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:29.411662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270051+0000","last_change":"2024-05-08T05:51:00.273052+0000","last_active":"2024-05-08T05:53:05.270051+0000","last_peered":"2024-05-08T05:53:05.270051+0000","last_clean":"2024-05-08T05:53:05.270051+0000","last_became_active":"2024-05-08T05:51:00.272718+0000","last_became_peered":"2024-05-08T05:51:00.272718+0000","last_unstale":"2024-05-08T05:53:05.270051+0000","last_undegraded":"2024-05-08T05:53:05.270051+0000","last_fullsized":"2024-05-08T05:53:05.270051+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:44.494378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139455+0000","last_change":"2024-05-08T05:51:04.306209+0000","last_active":"2024-05-08T05:53:07.139455+0000","last_peered":"2024-05-08T05:53:07.139455+0000","last_clean":"2024-05-08T05:53:07.139455+0000","last_became_active":"2024-05-08T05:51:04.305752+0000","last_became_peered":"2024-05-08T05:51:04.305752+0000","last_unstale":"2024-05-08T05:53:07.139455+0000","last_undegraded":"2024-05-08T05:53:07.139455+0000","last_fullsized":"2024-05-08T05:53:07.139455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:24.586111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820689+0000","last_change":"2024-05-08T05:51:00.278426+0000","last_active":"2024-05-08T05:53:05.820689+0000","last_peered":"2024-05-08T05:53:05.820689+0000","last_clean":"2024-05-08T05:53:05.820689+0000","last_became_active":"2024-05-08T05:51:00.276764+0000","last_became_peered":"2024-05-08T05:51:00.276764+0000","last_unstale":"2024-05-08T05:53:05.820689+0000","last_undegraded":"2024-05-08T05:53:05.820689+0000","last_fullsized":"2024-05-08T05:53:05.820689+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:34.158628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139683+0000","last_change":"2024-05-08T05:51:04.294511+0000","last_active":"2024-05-08T05:53:07.139683+0000","last_peered":"2024-05-08T05:53:07.139683+0000","last_clean":"2024-05-08T05:53:07.139683+0000","last_became_active":"2024-05-08T05:51:04.294145+0000","last_became_peered":"2024-05-08T05:51:04.294145+0000","last_unstale":"2024-05-08T05:53:07.139683+0000","last_undegraded":"2024-05-08T05:53:07.139683+0000","last_fullsized":"2024-05-08T05:53:07.139683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:10.034060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579833+0000","last_change":"2024-05-08T05:51:04.298905+0000","last_active":"2024-05-08T05:53:07.579833+0000","last_peered":"2024-05-08T05:53:07.579833+0000","last_clean":"2024-05-08T05:53:07.579833+0000","last_became_active":"2024-05-08T05:51:04.298586+0000","last_became_peered":"2024-05-08T05:51:04.298586+0000","last_unstale":"2024-05-08T05:53:07.579833+0000","last_undegraded":"2024-05-08T05:53:07.579833+0000","last_fullsized":"2024-05-08T05:53:07.579833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:43:24.899015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505323+0000","last_change":"2024-05-08T05:51:04.300503+0000","last_active":"2024-05-08T05:53:05.505323+0000","last_peered":"2024-05-08T05:53:05.505323+0000","last_clean":"2024-05-08T05:53:05.505323+0000","last_became_active":"2024-05-08T05:51:04.299771+0000","last_became_peered":"2024-05-08T05:51:04.299771+0000","last_unstale":"2024-05-08T05:53:05.505323+0000","last_undegraded":"2024-05-08T05:53:05.505323+0000","last_fullsized":"2024-05-08T05:53:05.505323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:19.790501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505267+0000","last_change":"2024-05-08T05:52:05.027724+0000","last_active":"2024-05-08T05:53:05.505267+0000","last_peered":"2024-05-08T05:53:05.505267+0000","last_clean":"2024-05-08T05:53:05.505267+0000","last_became_active":"2024-05-08T05:52:05.027338+0000","last_became_peered":"2024-05-08T05:52:05.027338+0000","last_unstale":"2024-05-08T05:53:05.505267+0000","last_undegraded":"2024-05-08T05:53:05.505267+0000","last_fullsized":"2024-05-08T05:53:05.505267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:03.324348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032152+0000","last_change":"2024-05-08T05:52:05.574645+0000","last_active":"2024-05-08T05:53:07.032152+0000","last_peered":"2024-05-08T05:53:07.032152+0000","last_clean":"2024-05-08T05:53:07.032152+0000","last_became_active":"2024-05-08T05:52:05.573751+0000","last_became_peered":"2024-05-08T05:52:05.573751+0000","last_unstale":"2024-05-08T05:53:07.032152+0000","last_undegraded":"2024-05-08T05:53:07.032152+0000","last_fullsized":"2024-05-08T05:53:07.032152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:57:40.656001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820990+0000","last_change":"2024-05-08T05:51:04.315769+0000","last_active":"2024-05-08T05:53:05.820990+0000","last_peered":"2024-05-08T05:53:05.820990+0000","last_clean":"2024-05-08T05:53:05.820990+0000","last_became_active":"2024-05-08T05:51:04.315322+0000","last_became_peered":"2024-05-08T05:51:04.315322+0000","last_unstale":"2024-05-08T05:53:05.820990+0000","last_undegraded":"2024-05-08T05:53:05.820990+0000","last_fullsized":"2024-05-08T05:53:05.820990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:52.096556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579878+0000","last_change":"2024-05-08T05:51:04.298128+0000","last_active":"2024-05-08T05:53:07.579878+0000","last_peered":"2024-05-08T05:53:07.579878+0000","last_clean":"2024-05-08T05:53:07.579878+0000","last_became_active":"2024-05-08T05:51:04.297743+0000","last_became_peered":"2024-05-08T05:51:04.297743+0000","last_unstale":"2024-05-08T05:53:07.579878+0000","last_undegraded":"2024-05-08T05:53:07.579878+0000","last_fullsized":"2024-05-08T05:53:07.579878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:18.334026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505656+0000","last_change":"2024-05-08T05:51:04.301056+0000","last_active":"2024-05-08T05:53:05.505656+0000","last_peered":"2024-05-08T05:53:05.505656+0000","last_clean":"2024-05-08T05:53:05.505656+0000","last_became_active":"2024-05-08T05:51:04.300733+0000","last_became_peered":"2024-05-08T05:51:04.300733+0000","last_unstale":"2024-05-08T05:53:05.505656+0000","last_undegraded":"2024-05-08T05:53:05.505656+0000","last_fullsized":"2024-05-08T05:53:05.505656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:21.616867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572880+0000","last_change":"2024-05-08T05:51:04.299079+0000","last_active":"2024-05-08T05:53:05.572880+0000","last_peered":"2024-05-08T05:53:05.572880+0000","last_clean":"2024-05-08T05:53:05.572880+0000","last_became_active":"2024-05-08T05:51:04.298628+0000","last_became_peered":"2024-05-08T05:51:04.298628+0000","last_unstale":"2024-05-08T05:53:05.572880+0000","last_undegraded":"2024-05-08T05:53:05.572880+0000","last_fullsized":"2024-05-08T05:53:05.572880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:45.031695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270220+0000","last_change":"2024-05-08T05:51:04.296140+0000","last_active":"2024-05-08T05:53:05.270220+0000","last_peered":"2024-05-08T05:53:05.270220+0000","last_clean":"2024-05-08T05:53:05.270220+0000","last_became_active":"2024-05-08T05:51:04.294940+0000","last_became_peered":"2024-05-08T05:51:04.294940+0000","last_unstale":"2024-05-08T05:53:05.270220+0000","last_undegraded":"2024-05-08T05:53:05.270220+0000","last_fullsized":"2024-05-08T05:53:05.270220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:37.192639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571649+0000","last_change":"2024-05-08T05:51:04.305803+0000","last_active":"2024-05-08T05:53:05.571649+0000","last_peered":"2024-05-08T05:53:05.571649+0000","last_clean":"2024-05-08T05:53:05.571649+0000","last_became_active":"2024-05-08T05:51:04.305482+0000","last_became_peered":"2024-05-08T05:51:04.305482+0000","last_unstale":"2024-05-08T05:53:05.571649+0000","last_undegraded":"2024-05-08T05:53:05.571649+0000","last_fullsized":"2024-05-08T05:53:05.571649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:02.050153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269746+0000","last_change":"2024-05-08T05:51:04.293787+0000","last_active":"2024-05-08T05:53:05.269746+0000","last_peered":"2024-05-08T05:53:05.269746+0000","last_clean":"2024-05-08T05:53:05.269746+0000","last_became_active":"2024-05-08T05:51:04.293455+0000","last_became_peered":"2024-05-08T05:51:04.293455+0000","last_unstale":"2024-05-08T05:53:05.269746+0000","last_undegraded":"2024-05-08T05:53:05.269746+0000","last_fullsized":"2024-05-08T05:53:05.269746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:09:53.211411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820895+0000","last_change":"2024-05-08T05:51:04.321442+0000","last_active":"2024-05-08T05:53:05.820895+0000","last_peered":"2024-05-08T05:53:05.820895+0000","last_clean":"2024-05-08T05:53:05.820895+0000","last_became_active":"2024-05-08T05:51:04.320982+0000","last_became_peered":"2024-05-08T05:51:04.320982+0000","last_unstale":"2024-05-08T05:53:05.820895+0000","last_undegraded":"2024-05-08T05:53:05.820895+0000","last_fullsized":"2024-05-08T05:53:05.820895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:45.089609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579671+0000","last_change":"2024-05-08T05:51:04.294866+0000","last_active":"2024-05-08T05:53:07.579671+0000","last_peered":"2024-05-08T05:53:07.579671+0000","last_clean":"2024-05-08T05:53:07.579671+0000","last_became_active":"2024-05-08T05:51:04.294504+0000","last_became_peered":"2024-05-08T05:51:04.294504+0000","last_unstale":"2024-05-08T05:53:07.579671+0000","last_undegraded":"2024-05-08T05:53:07.579671+0000","last_fullsized":"2024-05-08T05:53:07.579671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:33.139864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71075,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1305,"num_read_kb":729,"num_write":181,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":134,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":213225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645376,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":452538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841604,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1111152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005131,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":452752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201360,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":452650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430293,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1540096,"data_stored":1045053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593819,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":452141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":518669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":453412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":520105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182507,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":452452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986278,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":453110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790051,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36852,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708300,"statfs":{"total":95995035648,"available":95957299200,"internally_reserved":0,"allocated":1552384,"data_stored":1044205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:55:04.929 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:55:05.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:04 smithi112 ceph-mon[30003]: pgmap v434: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-08T05:55:05.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:04 smithi162 ceph-mon[34800]: pgmap v434: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-08T05:55:05.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.172+0000 7f06f3e00640 1 -- 172.21.15.112:0/2060377929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f4104400 msgr2=0x7f06f410ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:05.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.172+0000 7f06f3e00640 1 --2- 172.21.15.112:0/2060377929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f4104400 0x7f06f410ac90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009ca0 tx=0x7f06ec02a8c0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.172+0000 7f06f3e00640 1 -- 172.21.15.112:0/2060377929 shutdown_connections 2024-05-08T05:55:05.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.172+0000 7f06f3e00640 1 --2- 172.21.15.112:0/2060377929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f4104400 0x7f06f410ac90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.172+0000 7f06f3e00640 1 --2- 172.21.15.112:0/2060377929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06f4103a40 0x7f06f4103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.172+0000 7f06f3e00640 1 --2- 172.21.15.112:0/2060377929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06f4102850 0x7f06f4102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.172+0000 7f06f3e00640 1 -- 172.21.15.112:0/2060377929 >> 172.21.15.112:0/2060377929 conn(0x7f06f40fddc0 msgr2=0x7f06f4100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:05.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.172+0000 7f06f3e00640 1 -- 172.21.15.112:0/2060377929 shutdown_connections 2024-05-08T05:55:05.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.172+0000 7f06f3e00640 1 -- 172.21.15.112:0/2060377929 wait complete. 2024-05-08T05:55:05.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.173+0000 7f06f3e00640 1 Processor -- start 2024-05-08T05:55:05.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.173+0000 7f06f3e00640 1 -- start start 2024-05-08T05:55:05.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.173+0000 7f06f3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06f4102850 0x7f06f4076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:05.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.173+0000 7f06f3e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f4103a40 0x7f06f4074da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:05.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.173+0000 7f06f3e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06f4104400 0x7f06f40752e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:05.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.173+0000 7f06f3e00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06f410e360 con 0x7f06f4103a40 2024-05-08T05:55:05.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.173+0000 7f06f3e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f06f410e1e0 con 0x7f06f4104400 2024-05-08T05:55:05.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.174+0000 7f06f3e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f06f410e4e0 con 0x7f06f4102850 2024-05-08T05:55:05.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.174+0000 7f06f2a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06f4102850 0x7f06f4076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:05.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.174+0000 7f06ea000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f4103a40 0x7f06f4074da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:05.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.174+0000 7f06f2a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06f4102850 0x7f06f4076760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:57274/0 (socket says 172.21.15.112:57274) 2024-05-08T05:55:05.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.174+0000 7f06f2a00640 1 -- 172.21.15.112:0/573008726 learned_addr learned my addr 172.21.15.112:0/573008726 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:05.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.174+0000 7f06f3400640 1 --2- 172.21.15.112:0/573008726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06f4104400 0x7f06f40752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:05.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.174+0000 7f06f2a00640 1 -- 172.21.15.112:0/573008726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06f4104400 msgr2=0x7f06f40752e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:05.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.174+0000 7f06f2a00640 1 --2- 172.21.15.112:0/573008726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06f4104400 0x7f06f40752e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.174+0000 7f06f2a00640 1 -- 172.21.15.112:0/573008726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f4103a40 msgr2=0x7f06f4074da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:05.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.174+0000 7f06f2a00640 1 --2- 172.21.15.112:0/573008726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f4103a40 0x7f06f4074da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.174+0000 7f06f2a00640 1 -- 172.21.15.112:0/573008726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f4075b70 con 0x7f06f4102850 2024-05-08T05:55:05.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.174+0000 7f06f2a00640 1 --2- 172.21.15.112:0/573008726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06f4102850 0x7f06f4076760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f06e400e9c0 tx=0x7f06e400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:05.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.175+0000 7f06ebe00640 1 -- 172.21.15.112:0/573008726 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e400cd80 con 0x7f06f4102850 2024-05-08T05:55:05.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.175+0000 7f06f3e00640 1 -- 172.21.15.112:0/573008726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f41a2b00 con 0x7f06f4102850 2024-05-08T05:55:05.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.175+0000 7f06ebe00640 1 -- 172.21.15.112:0/573008726 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06e40035f0 con 0x7f06f4102850 2024-05-08T05:55:05.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.175+0000 7f06ebe00640 1 -- 172.21.15.112:0/573008726 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4010690 con 0x7f06f4102850 2024-05-08T05:55:05.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.175+0000 7f06f3e00640 1 -- 172.21.15.112:0/573008726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f41a3050 con 0x7f06f4102850 2024-05-08T05:55:05.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.176+0000 7f06f3e00640 1 -- 172.21.15.112:0/573008726 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f4109b70 con 0x7f06f4102850 2024-05-08T05:55:05.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.178+0000 7f06ebe00640 1 -- 172.21.15.112:0/573008726 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f06e40037a0 con 0x7f06f4102850 2024-05-08T05:55:05.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.180+0000 7f06ebe00640 1 --2- 172.21.15.112:0/573008726 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f06d80b27f0 0x7f06d80b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:05.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.180+0000 7f06ebe00640 1 -- 172.21.15.112:0/573008726 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f06e40e27c0 con 0x7f06f4102850 2024-05-08T05:55:05.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.180+0000 7f06ea000640 1 --2- 172.21.15.112:0/573008726 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f06d80b27f0 0x7f06d80b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:05.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.181+0000 7f06ea000640 1 --2- 172.21.15.112:0/573008726 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f06d80b27f0 0x7f06d80b4cb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f06f40764d0 tx=0x7f06dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:05.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.182+0000 7f06ebe00640 1 -- 172.21.15.112:0/573008726 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f06e40aa450 con 0x7f06f4102850 2024-05-08T05:55:05.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:04 smithi150 ceph-mon[34315]: pgmap v434: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-08T05:55:05.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.384+0000 7f06f3e00640 1 -- 172.21.15.112:0/573008726 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f06f4062df0 con 0x7f06d80b27f0 2024-05-08T05:55:05.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.394+0000 7f06ebe00640 1 -- 172.21.15.112:0/573008726 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258834 (secure 0 0 0) 0x7f06f4062df0 con 0x7f06d80b27f0 2024-05-08T05:55:05.398 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:55:05.399 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-05-08T05:55:05.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.399+0000 7f06f3e00640 1 -- 172.21.15.112:0/573008726 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f06d80b27f0 msgr2=0x7f06d80b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:05.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.399+0000 7f06f3e00640 1 --2- 172.21.15.112:0/573008726 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f06d80b27f0 0x7f06d80b4cb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f06f40764d0 tx=0x7f06dc005f90 comp rx=0 tx=0).stop 2024-05-08T05:55:05.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.399+0000 7f06f3e00640 1 -- 172.21.15.112:0/573008726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06f4102850 msgr2=0x7f06f4076760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:05.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.399+0000 7f06f3e00640 1 --2- 172.21.15.112:0/573008726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06f4102850 0x7f06f4076760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f06e400e9c0 tx=0x7f06e400ee80 comp rx=0 tx=0).stop 2024-05-08T05:55:05.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.399+0000 7f06f3e00640 1 -- 172.21.15.112:0/573008726 shutdown_connections 2024-05-08T05:55:05.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.399+0000 7f06f3e00640 1 --2- 172.21.15.112:0/573008726 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f06d80b27f0 0x7f06d80b4cb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.399+0000 7f06f3e00640 1 --2- 172.21.15.112:0/573008726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06f4104400 0x7f06f40752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.399+0000 7f06f3e00640 1 --2- 172.21.15.112:0/573008726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06f4103a40 0x7f06f4074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.399+0000 7f06f3e00640 1 --2- 172.21.15.112:0/573008726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06f4102850 0x7f06f4076760 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.399+0000 7f06f3e00640 1 -- 172.21.15.112:0/573008726 >> 172.21.15.112:0/573008726 conn(0x7f06f40fddc0 msgr2=0x7f06f4106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:05.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.399+0000 7f06f3e00640 1 -- 172.21.15.112:0/573008726 shutdown_connections 2024-05-08T05:55:05.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.399+0000 7f06f3e00640 1 -- 172.21.15.112:0/573008726 wait complete. 2024-05-08T05:55:05.420 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-05-08T05:55:03.567795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661677,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1381,"num_read_kb":793,"num_write":306,"num_write_kb":2280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951974,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":747,"ondisk_log_size":747,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366380,"kb_used_data":13044,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575444,"statfs":{"total":1151940427776,"available":1151565254656,"internally_reserved":0,"allocated":13357056,"data_stored":7408239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4934,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":37,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004939"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.580018+0000","last_change":"2024-05-08T05:51:04.320319+0000","last_active":"2024-05-08T05:53:07.580018+0000","last_peered":"2024-05-08T05:53:07.580018+0000","last_clean":"2024-05-08T05:53:07.580018+0000","last_became_active":"2024-05-08T05:51:04.319961+0000","last_became_peered":"2024-05-08T05:51:04.319961+0000","last_unstale":"2024-05-08T05:53:07.580018+0000","last_undegraded":"2024-05-08T05:53:07.580018+0000","last_fullsized":"2024-05-08T05:53:07.580018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:32.463261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032466+0000","last_change":"2024-05-08T05:51:04.315531+0000","last_active":"2024-05-08T05:53:07.032466+0000","last_peered":"2024-05-08T05:53:07.032466+0000","last_clean":"2024-05-08T05:53:07.032466+0000","last_became_active":"2024-05-08T05:51:04.314116+0000","last_became_peered":"2024-05-08T05:51:04.314116+0000","last_unstale":"2024-05-08T05:53:07.032466+0000","last_undegraded":"2024-05-08T05:53:07.032466+0000","last_fullsized":"2024-05-08T05:53:07.032466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:12:03.505888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032236+0000","last_change":"2024-05-08T05:51:04.315826+0000","last_active":"2024-05-08T05:53:07.032236+0000","last_peered":"2024-05-08T05:53:07.032236+0000","last_clean":"2024-05-08T05:53:07.032236+0000","last_became_active":"2024-05-08T05:51:04.315460+0000","last_became_peered":"2024-05-08T05:51:04.315460+0000","last_unstale":"2024-05-08T05:53:07.032236+0000","last_undegraded":"2024-05-08T05:53:07.032236+0000","last_fullsized":"2024-05-08T05:53:07.032236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:22:20.412368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715439+0000","last_change":"2024-05-08T05:51:04.297401+0000","last_active":"2024-05-08T05:53:05.715439+0000","last_peered":"2024-05-08T05:53:05.715439+0000","last_clean":"2024-05-08T05:53:05.715439+0000","last_became_active":"2024-05-08T05:51:04.297061+0000","last_became_peered":"2024-05-08T05:51:04.297061+0000","last_unstale":"2024-05-08T05:53:05.715439+0000","last_undegraded":"2024-05-08T05:53:05.715439+0000","last_fullsized":"2024-05-08T05:53:05.715439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:27.967517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268821+0000","last_change":"2024-05-08T05:51:04.315069+0000","last_active":"2024-05-08T05:53:05.268821+0000","last_peered":"2024-05-08T05:53:05.268821+0000","last_clean":"2024-05-08T05:53:05.268821+0000","last_became_active":"2024-05-08T05:51:04.314513+0000","last_became_peered":"2024-05-08T05:51:04.314513+0000","last_unstale":"2024-05-08T05:53:05.268821+0000","last_undegraded":"2024-05-08T05:53:05.268821+0000","last_fullsized":"2024-05-08T05:53:05.268821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:02.520130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265219+0000","last_change":"2024-05-08T05:51:04.302851+0000","last_active":"2024-05-08T05:53:05.265219+0000","last_peered":"2024-05-08T05:53:05.265219+0000","last_clean":"2024-05-08T05:53:05.265219+0000","last_became_active":"2024-05-08T05:51:04.302256+0000","last_became_peered":"2024-05-08T05:51:04.302256+0000","last_unstale":"2024-05-08T05:53:05.265219+0000","last_undegraded":"2024-05-08T05:53:05.265219+0000","last_fullsized":"2024-05-08T05:53:05.265219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:45.587319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139305+0000","last_change":"2024-05-08T05:51:04.308674+0000","last_active":"2024-05-08T05:53:07.139305+0000","last_peered":"2024-05-08T05:53:07.139305+0000","last_clean":"2024-05-08T05:53:07.139305+0000","last_became_active":"2024-05-08T05:51:04.308295+0000","last_became_peered":"2024-05-08T05:51:04.308295+0000","last_unstale":"2024-05-08T05:53:07.139305+0000","last_undegraded":"2024-05-08T05:53:07.139305+0000","last_fullsized":"2024-05-08T05:53:07.139305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:59:17.561655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571645+0000","last_change":"2024-05-08T05:51:04.314511+0000","last_active":"2024-05-08T05:53:05.571645+0000","last_peered":"2024-05-08T05:53:05.571645+0000","last_clean":"2024-05-08T05:53:05.571645+0000","last_became_active":"2024-05-08T05:51:04.314082+0000","last_became_peered":"2024-05-08T05:51:04.314082+0000","last_unstale":"2024-05-08T05:53:05.571645+0000","last_undegraded":"2024-05-08T05:53:05.571645+0000","last_fullsized":"2024-05-08T05:53:05.571645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:24.182391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269793+0000","last_change":"2024-05-08T05:51:04.293745+0000","last_active":"2024-05-08T05:53:05.269793+0000","last_peered":"2024-05-08T05:53:05.269793+0000","last_clean":"2024-05-08T05:53:05.269793+0000","last_became_active":"2024-05-08T05:51:04.293334+0000","last_became_peered":"2024-05-08T05:51:04.293334+0000","last_unstale":"2024-05-08T05:53:05.269793+0000","last_undegraded":"2024-05-08T05:53:05.269793+0000","last_fullsized":"2024-05-08T05:53:05.269793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:16.576967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265481+0000","last_change":"2024-05-08T05:51:04.297800+0000","last_active":"2024-05-08T05:53:05.265481+0000","last_peered":"2024-05-08T05:53:05.265481+0000","last_clean":"2024-05-08T05:53:05.265481+0000","last_became_active":"2024-05-08T05:51:04.297349+0000","last_became_peered":"2024-05-08T05:51:04.297349+0000","last_unstale":"2024-05-08T05:53:05.265481+0000","last_undegraded":"2024-05-08T05:53:05.265481+0000","last_fullsized":"2024-05-08T05:53:05.265481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:55.784512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139645+0000","last_change":"2024-05-08T05:51:04.297997+0000","last_active":"2024-05-08T05:53:07.139645+0000","last_peered":"2024-05-08T05:53:07.139645+0000","last_clean":"2024-05-08T05:53:07.139645+0000","last_became_active":"2024-05-08T05:51:04.297516+0000","last_became_peered":"2024-05-08T05:51:04.297516+0000","last_unstale":"2024-05-08T05:53:07.139645+0000","last_undegraded":"2024-05-08T05:53:07.139645+0000","last_fullsized":"2024-05-08T05:53:07.139645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:25.991555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715483+0000","last_change":"2024-05-08T05:51:04.295610+0000","last_active":"2024-05-08T05:53:05.715483+0000","last_peered":"2024-05-08T05:53:05.715483+0000","last_clean":"2024-05-08T05:53:05.715483+0000","last_became_active":"2024-05-08T05:51:04.294553+0000","last_became_peered":"2024-05-08T05:51:04.294553+0000","last_unstale":"2024-05-08T05:53:05.715483+0000","last_undegraded":"2024-05-08T05:53:05.715483+0000","last_fullsized":"2024-05-08T05:53:05.715483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:40:14.492057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820666+0000","last_change":"2024-05-08T05:51:04.303879+0000","last_active":"2024-05-08T05:53:05.820666+0000","last_peered":"2024-05-08T05:53:05.820666+0000","last_clean":"2024-05-08T05:53:05.820666+0000","last_became_active":"2024-05-08T05:51:04.303423+0000","last_became_peered":"2024-05-08T05:51:04.303423+0000","last_unstale":"2024-05-08T05:53:05.820666+0000","last_undegraded":"2024-05-08T05:53:05.820666+0000","last_fullsized":"2024-05-08T05:53:05.820666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:50:55.905858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.031822+0000","last_change":"2024-05-08T05:51:04.309818+0000","last_active":"2024-05-08T05:53:07.031822+0000","last_peered":"2024-05-08T05:53:07.031822+0000","last_clean":"2024-05-08T05:53:07.031822+0000","last_became_active":"2024-05-08T05:51:04.309434+0000","last_became_peered":"2024-05-08T05:51:04.309434+0000","last_unstale":"2024-05-08T05:53:07.031822+0000","last_undegraded":"2024-05-08T05:53:07.031822+0000","last_fullsized":"2024-05-08T05:53:07.031822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:55.832239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.031573+0000","last_change":"2024-05-08T05:51:04.316708+0000","last_active":"2024-05-08T05:53:07.031573+0000","last_peered":"2024-05-08T05:53:07.031573+0000","last_clean":"2024-05-08T05:53:07.031573+0000","last_became_active":"2024-05-08T05:51:04.316525+0000","last_became_peered":"2024-05-08T05:51:04.316525+0000","last_unstale":"2024-05-08T05:53:07.031573+0000","last_undegraded":"2024-05-08T05:53:07.031573+0000","last_fullsized":"2024-05-08T05:53:07.031573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:33:38.246611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269307+0000","last_change":"2024-05-08T05:51:04.314780+0000","last_active":"2024-05-08T05:53:05.269307+0000","last_peered":"2024-05-08T05:53:05.269307+0000","last_clean":"2024-05-08T05:53:05.269307+0000","last_became_active":"2024-05-08T05:51:04.314406+0000","last_became_peered":"2024-05-08T05:51:04.314406+0000","last_unstale":"2024-05-08T05:53:05.269307+0000","last_undegraded":"2024-05-08T05:53:05.269307+0000","last_fullsized":"2024-05-08T05:53:05.269307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:39:26.926238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579947+0000","last_change":"2024-05-08T05:51:04.319425+0000","last_active":"2024-05-08T05:53:07.579947+0000","last_peered":"2024-05-08T05:53:07.579947+0000","last_clean":"2024-05-08T05:53:07.579947+0000","last_became_active":"2024-05-08T05:51:04.319106+0000","last_became_peered":"2024-05-08T05:51:04.319106+0000","last_unstale":"2024-05-08T05:53:07.579947+0000","last_undegraded":"2024-05-08T05:53:07.579947+0000","last_fullsized":"2024-05-08T05:53:07.579947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:59:54.023162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267374+0000","last_change":"2024-05-08T05:51:04.303547+0000","last_active":"2024-05-08T05:53:05.267374+0000","last_peered":"2024-05-08T05:53:05.267374+0000","last_clean":"2024-05-08T05:53:05.267374+0000","last_became_active":"2024-05-08T05:51:04.303182+0000","last_became_peered":"2024-05-08T05:51:04.303182+0000","last_unstale":"2024-05-08T05:53:05.267374+0000","last_undegraded":"2024-05-08T05:53:05.267374+0000","last_fullsized":"2024-05-08T05:53:05.267374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:11:59.624329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505589+0000","last_change":"2024-05-08T05:51:04.315674+0000","last_active":"2024-05-08T05:53:05.505589+0000","last_peered":"2024-05-08T05:53:05.505589+0000","last_clean":"2024-05-08T05:53:05.505589+0000","last_became_active":"2024-05-08T05:51:04.314911+0000","last_became_peered":"2024-05-08T05:51:04.314911+0000","last_unstale":"2024-05-08T05:53:05.505589+0000","last_undegraded":"2024-05-08T05:53:05.505589+0000","last_fullsized":"2024-05-08T05:53:05.505589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:49.501508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267963+0000","last_change":"2024-05-08T05:51:00.273554+0000","last_active":"2024-05-08T05:53:05.267963+0000","last_peered":"2024-05-08T05:53:05.267963+0000","last_clean":"2024-05-08T05:53:05.267963+0000","last_became_active":"2024-05-08T05:51:00.272858+0000","last_became_peered":"2024-05-08T05:51:00.272858+0000","last_unstale":"2024-05-08T05:53:05.267963+0000","last_undegraded":"2024-05-08T05:53:05.267963+0000","last_fullsized":"2024-05-08T05:53:05.267963+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:47:37.418887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572441+0000","last_change":"2024-05-08T05:51:04.306392+0000","last_active":"2024-05-08T05:53:05.572441+0000","last_peered":"2024-05-08T05:53:05.572441+0000","last_clean":"2024-05-08T05:53:05.572441+0000","last_became_active":"2024-05-08T05:51:04.304248+0000","last_became_peered":"2024-05-08T05:51:04.304248+0000","last_unstale":"2024-05-08T05:53:05.572441+0000","last_undegraded":"2024-05-08T05:53:05.572441+0000","last_fullsized":"2024-05-08T05:53:05.572441+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:49:26.034883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572774+0000","last_change":"2024-05-08T05:51:01.187292+0000","last_active":"2024-05-08T05:53:05.572774+0000","last_peered":"2024-05-08T05:53:05.572774+0000","last_clean":"2024-05-08T05:53:05.572774+0000","last_became_active":"2024-05-08T05:51:01.185948+0000","last_became_peered":"2024-05-08T05:51:01.185948+0000","last_unstale":"2024-05-08T05:53:05.572774+0000","last_undegraded":"2024-05-08T05:53:05.572774+0000","last_fullsized":"2024-05-08T05:53:05.572774+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:02.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032756+0000","last_change":"2024-05-08T05:51:04.318439+0000","last_active":"2024-05-08T05:53:07.032756+0000","last_peered":"2024-05-08T05:53:07.032756+0000","last_clean":"2024-05-08T05:53:07.032756+0000","last_became_active":"2024-05-08T05:51:04.317722+0000","last_became_peered":"2024-05-08T05:51:04.317722+0000","last_unstale":"2024-05-08T05:53:07.032756+0000","last_undegraded":"2024-05-08T05:53:07.032756+0000","last_fullsized":"2024-05-08T05:53:07.032756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:36:43.648643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269296+0000","last_change":"2024-05-08T05:51:00.269684+0000","last_active":"2024-05-08T05:53:05.269296+0000","last_peered":"2024-05-08T05:53:05.269296+0000","last_clean":"2024-05-08T05:53:05.269296+0000","last_became_active":"2024-05-08T05:51:00.269329+0000","last_became_peered":"2024-05-08T05:51:00.269329+0000","last_unstale":"2024-05-08T05:53:05.269296+0000","last_undegraded":"2024-05-08T05:53:05.269296+0000","last_fullsized":"2024-05-08T05:53:05.269296+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:27:10.422994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269410+0000","last_change":"2024-05-08T05:51:04.308048+0000","last_active":"2024-05-08T05:53:05.269410+0000","last_peered":"2024-05-08T05:53:05.269410+0000","last_clean":"2024-05-08T05:53:05.269410+0000","last_became_active":"2024-05-08T05:51:04.307550+0000","last_became_peered":"2024-05-08T05:51:04.307550+0000","last_unstale":"2024-05-08T05:53:05.269410+0000","last_undegraded":"2024-05-08T05:53:05.269410+0000","last_fullsized":"2024-05-08T05:53:05.269410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:13.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269033+0000","last_change":"2024-05-08T05:51:00.268142+0000","last_active":"2024-05-08T05:53:05.269033+0000","last_peered":"2024-05-08T05:53:05.269033+0000","last_clean":"2024-05-08T05:53:05.269033+0000","last_became_active":"2024-05-08T05:51:00.267756+0000","last_became_peered":"2024-05-08T05:51:00.267756+0000","last_unstale":"2024-05-08T05:53:05.269033+0000","last_undegraded":"2024-05-08T05:53:05.269033+0000","last_fullsized":"2024-05-08T05:53:05.269033+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:01.439680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270314+0000","last_change":"2024-05-08T05:51:04.292341+0000","last_active":"2024-05-08T05:53:05.270314+0000","last_peered":"2024-05-08T05:53:05.270314+0000","last_clean":"2024-05-08T05:53:05.270314+0000","last_became_active":"2024-05-08T05:51:04.291086+0000","last_became_peered":"2024-05-08T05:51:04.291086+0000","last_unstale":"2024-05-08T05:53:05.270314+0000","last_undegraded":"2024-05-08T05:53:05.270314+0000","last_fullsized":"2024-05-08T05:53:05.270314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:47.108100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":252,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:54:57.975810+0000","last_change":"2024-05-08T05:51:00.268100+0000","last_active":"2024-05-08T05:54:57.975810+0000","last_peered":"2024-05-08T05:54:57.975810+0000","last_clean":"2024-05-08T05:54:57.975810+0000","last_became_active":"2024-05-08T05:51:00.267678+0000","last_became_peered":"2024-05-08T05:51:00.267678+0000","last_unstale":"2024-05-08T05:54:57.975810+0000","last_undegraded":"2024-05-08T05:54:57.975810+0000","last_fullsized":"2024-05-08T05:54:57.975810+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:11:31.731548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":215,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268766+0000","last_change":"2024-05-08T05:51:04.303872+0000","last_active":"2024-05-08T05:53:05.268766+0000","last_peered":"2024-05-08T05:53:05.268766+0000","last_clean":"2024-05-08T05:53:05.268766+0000","last_became_active":"2024-05-08T05:51:04.303456+0000","last_became_peered":"2024-05-08T05:51:04.303456+0000","last_unstale":"2024-05-08T05:53:05.268766+0000","last_undegraded":"2024-05-08T05:53:05.268766+0000","last_fullsized":"2024-05-08T05:53:05.268766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:42:05.098488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265391+0000","last_change":"2024-05-08T05:51:00.275657+0000","last_active":"2024-05-08T05:53:05.265391+0000","last_peered":"2024-05-08T05:53:05.265391+0000","last_clean":"2024-05-08T05:53:05.265391+0000","last_became_active":"2024-05-08T05:51:00.275254+0000","last_became_peered":"2024-05-08T05:51:00.275254+0000","last_unstale":"2024-05-08T05:53:05.265391+0000","last_undegraded":"2024-05-08T05:53:05.265391+0000","last_fullsized":"2024-05-08T05:53:05.265391+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:28.555698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270118+0000","last_change":"2024-05-08T05:51:04.293791+0000","last_active":"2024-05-08T05:53:05.270118+0000","last_peered":"2024-05-08T05:53:05.270118+0000","last_clean":"2024-05-08T05:53:05.270118+0000","last_became_active":"2024-05-08T05:51:04.293405+0000","last_became_peered":"2024-05-08T05:51:04.293405+0000","last_unstale":"2024-05-08T05:53:05.270118+0000","last_undegraded":"2024-05-08T05:53:05.270118+0000","last_fullsized":"2024-05-08T05:53:05.270118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:28.064452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820147+0000","last_change":"2024-05-08T05:51:00.280295+0000","last_active":"2024-05-08T05:53:05.820147+0000","last_peered":"2024-05-08T05:53:05.820147+0000","last_clean":"2024-05-08T05:53:05.820147+0000","last_became_active":"2024-05-08T05:51:00.279721+0000","last_became_peered":"2024-05-08T05:51:00.279721+0000","last_unstale":"2024-05-08T05:53:05.820147+0000","last_undegraded":"2024-05-08T05:53:05.820147+0000","last_fullsized":"2024-05-08T05:53:05.820147+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:06:38.064872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571814+0000","last_change":"2024-05-08T05:51:04.307546+0000","last_active":"2024-05-08T05:53:05.571814+0000","last_peered":"2024-05-08T05:53:05.571814+0000","last_clean":"2024-05-08T05:53:05.571814+0000","last_became_active":"2024-05-08T05:51:04.307164+0000","last_became_peered":"2024-05-08T05:51:04.307164+0000","last_unstale":"2024-05-08T05:53:05.571814+0000","last_undegraded":"2024-05-08T05:53:05.571814+0000","last_fullsized":"2024-05-08T05:53:05.571814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:36:02.269040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269777+0000","last_change":"2024-05-08T05:51:00.267987+0000","last_active":"2024-05-08T05:53:05.269777+0000","last_peered":"2024-05-08T05:53:05.269777+0000","last_clean":"2024-05-08T05:53:05.269777+0000","last_became_active":"2024-05-08T05:51:00.267633+0000","last_became_peered":"2024-05-08T05:51:00.267633+0000","last_unstale":"2024-05-08T05:53:05.269777+0000","last_undegraded":"2024-05-08T05:53:05.269777+0000","last_fullsized":"2024-05-08T05:53:05.269777+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:13.034300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.266492+0000","last_change":"2024-05-08T05:51:04.311422+0000","last_active":"2024-05-08T05:53:05.266492+0000","last_peered":"2024-05-08T05:53:05.266492+0000","last_clean":"2024-05-08T05:53:05.266492+0000","last_became_active":"2024-05-08T05:51:04.311052+0000","last_became_peered":"2024-05-08T05:51:04.311052+0000","last_unstale":"2024-05-08T05:53:05.266492+0000","last_undegraded":"2024-05-08T05:53:05.266492+0000","last_fullsized":"2024-05-08T05:53:05.266492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:47.790583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":246,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:54:57.974211+0000","last_change":"2024-05-08T05:51:00.274252+0000","last_active":"2024-05-08T05:54:57.974211+0000","last_peered":"2024-05-08T05:54:57.974211+0000","last_clean":"2024-05-08T05:54:57.974211+0000","last_became_active":"2024-05-08T05:51:00.273942+0000","last_became_peered":"2024-05-08T05:51:00.273942+0000","last_unstale":"2024-05-08T05:54:57.974211+0000","last_undegraded":"2024-05-08T05:54:57.974211+0000","last_fullsized":"2024-05-08T05:54:57.974211+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:36.778032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":212,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269700+0000","last_change":"2024-05-08T05:51:04.296414+0000","last_active":"2024-05-08T05:53:05.269700+0000","last_peered":"2024-05-08T05:53:05.269700+0000","last_clean":"2024-05-08T05:53:05.269700+0000","last_became_active":"2024-05-08T05:51:04.296015+0000","last_became_peered":"2024-05-08T05:51:04.296015+0000","last_unstale":"2024-05-08T05:53:05.269700+0000","last_undegraded":"2024-05-08T05:53:05.269700+0000","last_fullsized":"2024-05-08T05:53:05.269700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:48:15.609385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139516+0000","last_change":"2024-05-08T05:51:00.268234+0000","last_active":"2024-05-08T05:53:07.139516+0000","last_peered":"2024-05-08T05:53:07.139516+0000","last_clean":"2024-05-08T05:53:07.139516+0000","last_became_active":"2024-05-08T05:51:00.267846+0000","last_became_peered":"2024-05-08T05:51:00.267846+0000","last_unstale":"2024-05-08T05:53:07.139516+0000","last_undegraded":"2024-05-08T05:53:07.139516+0000","last_fullsized":"2024-05-08T05:53:07.139516+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:26.919004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032653+0000","last_change":"2024-05-08T05:51:04.318215+0000","last_active":"2024-05-08T05:53:07.032653+0000","last_peered":"2024-05-08T05:53:07.032653+0000","last_clean":"2024-05-08T05:53:07.032653+0000","last_became_active":"2024-05-08T05:51:04.317309+0000","last_became_peered":"2024-05-08T05:51:04.317309+0000","last_unstale":"2024-05-08T05:53:07.032653+0000","last_undegraded":"2024-05-08T05:53:07.032653+0000","last_fullsized":"2024-05-08T05:53:07.032653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:11.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270484+0000","last_change":"2024-05-08T05:51:01.186438+0000","last_active":"2024-05-08T05:53:05.270484+0000","last_peered":"2024-05-08T05:53:05.270484+0000","last_clean":"2024-05-08T05:53:05.270484+0000","last_became_active":"2024-05-08T05:51:01.186039+0000","last_became_peered":"2024-05-08T05:51:01.186039+0000","last_unstale":"2024-05-08T05:53:05.270484+0000","last_undegraded":"2024-05-08T05:53:05.270484+0000","last_fullsized":"2024-05-08T05:53:05.270484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:34.652184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269557+0000","last_change":"2024-05-08T05:52:05.575531+0000","last_active":"2024-05-08T05:53:05.269557+0000","last_peered":"2024-05-08T05:53:05.269557+0000","last_clean":"2024-05-08T05:53:05.269557+0000","last_became_active":"2024-05-08T05:52:05.575042+0000","last_became_peered":"2024-05-08T05:52:05.575042+0000","last_unstale":"2024-05-08T05:53:05.269557+0000","last_undegraded":"2024-05-08T05:53:05.269557+0000","last_fullsized":"2024-05-08T05:53:05.269557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:56.096772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'36","reported_seq":502,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:54:58.977550+0000","last_change":"2024-05-08T05:51:00.263385+0000","last_active":"2024-05-08T05:54:58.977550+0000","last_peered":"2024-05-08T05:54:58.977550+0000","last_clean":"2024-05-08T05:54:58.977550+0000","last_became_active":"2024-05-08T05:51:00.262997+0000","last_became_peered":"2024-05-08T05:51:00.262997+0000","last_unstale":"2024-05-08T05:54:58.977550+0000","last_undegraded":"2024-05-08T05:54:58.977550+0000","last_fullsized":"2024-05-08T05:54:58.977550+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:22:09.727481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66623,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":436,"num_read_kb":283,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269140+0000","last_change":"2024-05-08T05:51:04.294790+0000","last_active":"2024-05-08T05:53:05.269140+0000","last_peered":"2024-05-08T05:53:05.269140+0000","last_clean":"2024-05-08T05:53:05.269140+0000","last_became_active":"2024-05-08T05:51:04.294418+0000","last_became_peered":"2024-05-08T05:51:04.294418+0000","last_unstale":"2024-05-08T05:53:05.269140+0000","last_undegraded":"2024-05-08T05:53:05.269140+0000","last_fullsized":"2024-05-08T05:53:05.269140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:11:10.624434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505362+0000","last_change":"2024-05-08T05:51:00.258507+0000","last_active":"2024-05-08T05:53:05.505362+0000","last_peered":"2024-05-08T05:53:05.505362+0000","last_clean":"2024-05-08T05:53:05.505362+0000","last_became_active":"2024-05-08T05:51:00.258179+0000","last_became_peered":"2024-05-08T05:51:00.258179+0000","last_unstale":"2024-05-08T05:53:05.505362+0000","last_undegraded":"2024-05-08T05:53:05.505362+0000","last_fullsized":"2024-05-08T05:53:05.505362+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:22.376499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820379+0000","last_change":"2024-05-08T05:51:04.310947+0000","last_active":"2024-05-08T05:53:05.820379+0000","last_peered":"2024-05-08T05:53:05.820379+0000","last_clean":"2024-05-08T05:53:05.820379+0000","last_became_active":"2024-05-08T05:51:04.310378+0000","last_became_peered":"2024-05-08T05:51:04.310378+0000","last_unstale":"2024-05-08T05:53:05.820379+0000","last_undegraded":"2024-05-08T05:53:05.820379+0000","last_fullsized":"2024-05-08T05:53:05.820379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:19.367660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270609+0000","last_change":"2024-05-08T05:51:00.271104+0000","last_active":"2024-05-08T05:53:05.270609+0000","last_peered":"2024-05-08T05:53:05.270609+0000","last_clean":"2024-05-08T05:53:05.270609+0000","last_became_active":"2024-05-08T05:51:00.270497+0000","last_became_peered":"2024-05-08T05:51:00.270497+0000","last_unstale":"2024-05-08T05:53:05.270609+0000","last_undegraded":"2024-05-08T05:53:05.270609+0000","last_fullsized":"2024-05-08T05:53:05.270609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:40.959423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139512+0000","last_change":"2024-05-08T05:51:04.310243+0000","last_active":"2024-05-08T05:53:07.139512+0000","last_peered":"2024-05-08T05:53:07.139512+0000","last_clean":"2024-05-08T05:53:07.139512+0000","last_became_active":"2024-05-08T05:51:04.309871+0000","last_became_peered":"2024-05-08T05:51:04.309871+0000","last_unstale":"2024-05-08T05:53:07.139512+0000","last_undegraded":"2024-05-08T05:53:07.139512+0000","last_fullsized":"2024-05-08T05:53:07.139512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:18:21.045621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":251,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:54:57.082440+0000","last_change":"2024-05-08T05:51:00.266096+0000","last_active":"2024-05-08T05:54:57.082440+0000","last_peered":"2024-05-08T05:54:57.082440+0000","last_clean":"2024-05-08T05:54:57.082440+0000","last_became_active":"2024-05-08T05:51:00.265706+0000","last_became_peered":"2024-05-08T05:51:00.265706+0000","last_unstale":"2024-05-08T05:54:57.082440+0000","last_undegraded":"2024-05-08T05:54:57.082440+0000","last_fullsized":"2024-05-08T05:54:57.082440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:47.388732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":216,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139267+0000","last_change":"2024-05-08T05:51:04.293141+0000","last_active":"2024-05-08T05:53:07.139267+0000","last_peered":"2024-05-08T05:53:07.139267+0000","last_clean":"2024-05-08T05:53:07.139267+0000","last_became_active":"2024-05-08T05:51:04.292772+0000","last_became_peered":"2024-05-08T05:51:04.292772+0000","last_unstale":"2024-05-08T05:53:07.139267+0000","last_undegraded":"2024-05-08T05:53:07.139267+0000","last_fullsized":"2024-05-08T05:53:07.139267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:25:32.801209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268837+0000","last_change":"2024-05-08T05:52:05.042102+0000","last_active":"2024-05-08T05:53:05.268837+0000","last_peered":"2024-05-08T05:53:05.268837+0000","last_clean":"2024-05-08T05:53:05.268837+0000","last_became_active":"2024-05-08T05:52:05.027787+0000","last_became_peered":"2024-05-08T05:52:05.027787+0000","last_unstale":"2024-05-08T05:53:05.268837+0000","last_undegraded":"2024-05-08T05:53:05.268837+0000","last_fullsized":"2024-05-08T05:53:05.268837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:05.580309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269940+0000","last_change":"2024-05-08T05:51:04.290096+0000","last_active":"2024-05-08T05:53:05.269940+0000","last_peered":"2024-05-08T05:53:05.269940+0000","last_clean":"2024-05-08T05:53:05.269940+0000","last_became_active":"2024-05-08T05:51:04.289684+0000","last_became_peered":"2024-05-08T05:51:04.289684+0000","last_unstale":"2024-05-08T05:53:05.269940+0000","last_undegraded":"2024-05-08T05:53:05.269940+0000","last_fullsized":"2024-05-08T05:53:05.269940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:02.400348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579795+0000","last_change":"2024-05-08T05:52:05.034004+0000","last_active":"2024-05-08T05:53:07.579795+0000","last_peered":"2024-05-08T05:53:07.579795+0000","last_clean":"2024-05-08T05:53:07.579795+0000","last_became_active":"2024-05-08T05:52:05.023497+0000","last_became_peered":"2024-05-08T05:52:05.023497+0000","last_unstale":"2024-05-08T05:53:07.579795+0000","last_undegraded":"2024-05-08T05:53:07.579795+0000","last_fullsized":"2024-05-08T05:53:07.579795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:06.462940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579850+0000","last_change":"2024-05-08T05:52:05.027648+0000","last_active":"2024-05-08T05:53:07.579850+0000","last_peered":"2024-05-08T05:53:07.579850+0000","last_clean":"2024-05-08T05:53:07.579850+0000","last_became_active":"2024-05-08T05:52:05.027262+0000","last_became_peered":"2024-05-08T05:52:05.027262+0000","last_unstale":"2024-05-08T05:53:07.579850+0000","last_undegraded":"2024-05-08T05:53:07.579850+0000","last_fullsized":"2024-05-08T05:53:07.579850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:36:18.588967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267116+0000","last_change":"2024-05-08T05:51:01.189324+0000","last_active":"2024-05-08T05:53:05.267116+0000","last_peered":"2024-05-08T05:53:05.267116+0000","last_clean":"2024-05-08T05:53:05.267116+0000","last_became_active":"2024-05-08T05:51:01.188651+0000","last_became_peered":"2024-05-08T05:51:01.188651+0000","last_unstale":"2024-05-08T05:53:05.267116+0000","last_undegraded":"2024-05-08T05:53:05.267116+0000","last_fullsized":"2024-05-08T05:53:05.267116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:45.149213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265294+0000","last_change":"2024-05-08T05:51:04.306811+0000","last_active":"2024-05-08T05:53:05.265294+0000","last_peered":"2024-05-08T05:53:05.265294+0000","last_clean":"2024-05-08T05:53:05.265294+0000","last_became_active":"2024-05-08T05:51:04.306423+0000","last_became_peered":"2024-05-08T05:51:04.306423+0000","last_unstale":"2024-05-08T05:53:05.265294+0000","last_undegraded":"2024-05-08T05:53:05.265294+0000","last_fullsized":"2024-05-08T05:53:05.265294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:08.830682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139525+0000","last_change":"2024-05-08T05:51:00.276543+0000","last_active":"2024-05-08T05:53:07.139525+0000","last_peered":"2024-05-08T05:53:07.139525+0000","last_clean":"2024-05-08T05:53:07.139525+0000","last_became_active":"2024-05-08T05:51:00.276233+0000","last_became_peered":"2024-05-08T05:51:00.276233+0000","last_unstale":"2024-05-08T05:53:07.139525+0000","last_undegraded":"2024-05-08T05:53:07.139525+0000","last_fullsized":"2024-05-08T05:53:07.139525+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:11.229942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265173+0000","last_change":"2024-05-08T05:51:04.322033+0000","last_active":"2024-05-08T05:53:05.265173+0000","last_peered":"2024-05-08T05:53:05.265173+0000","last_clean":"2024-05-08T05:53:05.265173+0000","last_became_active":"2024-05-08T05:51:04.321555+0000","last_became_peered":"2024-05-08T05:51:04.321555+0000","last_unstale":"2024-05-08T05:53:05.265173+0000","last_undegraded":"2024-05-08T05:53:05.265173+0000","last_fullsized":"2024-05-08T05:53:05.265173+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:21:50.820709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.266462+0000","last_change":"2024-05-08T05:52:05.027723+0000","last_active":"2024-05-08T05:53:05.266462+0000","last_peered":"2024-05-08T05:53:05.266462+0000","last_clean":"2024-05-08T05:53:05.266462+0000","last_became_active":"2024-05-08T05:52:05.026508+0000","last_became_peered":"2024-05-08T05:52:05.026508+0000","last_unstale":"2024-05-08T05:53:05.266462+0000","last_undegraded":"2024-05-08T05:53:05.266462+0000","last_fullsized":"2024-05-08T05:53:05.266462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:14:32.399433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":250,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:55:00.087804+0000","last_change":"2024-05-08T05:51:01.187556+0000","last_active":"2024-05-08T05:55:00.087804+0000","last_peered":"2024-05-08T05:55:00.087804+0000","last_clean":"2024-05-08T05:55:00.087804+0000","last_became_active":"2024-05-08T05:51:01.186584+0000","last_became_peered":"2024-05-08T05:51:01.186584+0000","last_unstale":"2024-05-08T05:55:00.087804+0000","last_undegraded":"2024-05-08T05:55:00.087804+0000","last_fullsized":"2024-05-08T05:55:00.087804+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:29.036633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.274091+0000","last_change":"2024-05-08T05:53:05.272875+0000","last_active":"2024-05-08T05:53:05.274091+0000","last_peered":"2024-05-08T05:53:05.274091+0000","last_clean":"2024-05-08T05:53:05.274091+0000","last_became_active":"2024-05-08T05:53:05.271672+0000","last_became_peered":"2024-05-08T05:53:05.271672+0000","last_unstale":"2024-05-08T05:53:05.274091+0000","last_undegraded":"2024-05-08T05:53:05.274091+0000","last_fullsized":"2024-05-08T05:53:05.274091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:08:28.776982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715396+0000","last_change":"2024-05-08T05:51:00.272501+0000","last_active":"2024-05-08T05:53:05.715396+0000","last_peered":"2024-05-08T05:53:05.715396+0000","last_clean":"2024-05-08T05:53:05.715396+0000","last_became_active":"2024-05-08T05:51:00.272166+0000","last_became_peered":"2024-05-08T05:51:00.272166+0000","last_unstale":"2024-05-08T05:53:05.715396+0000","last_undegraded":"2024-05-08T05:53:05.715396+0000","last_fullsized":"2024-05-08T05:53:05.715396+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:30.980047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":104,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139425+0000","last_change":"2024-05-08T05:49:31.820799+0000","last_active":"2024-05-08T05:53:07.139425+0000","last_peered":"2024-05-08T05:53:07.139425+0000","last_clean":"2024-05-08T05:53:07.139425+0000","last_became_active":"2024-05-08T05:49:31.815568+0000","last_became_peered":"2024-05-08T05:49:31.815568+0000","last_unstale":"2024-05-08T05:53:07.139425+0000","last_undegraded":"2024-05-08T05:53:07.139425+0000","last_fullsized":"2024-05-08T05:53:07.139425+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:48.170242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:48.170242+0000","last_clean_scrub_stamp":"2024-05-08T05:45:48.170242+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:39.527036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820012+0000","last_change":"2024-05-08T05:51:04.301375+0000","last_active":"2024-05-08T05:53:05.820012+0000","last_peered":"2024-05-08T05:53:05.820012+0000","last_clean":"2024-05-08T05:53:05.820012+0000","last_became_active":"2024-05-08T05:51:04.301027+0000","last_became_peered":"2024-05-08T05:51:04.301027+0000","last_unstale":"2024-05-08T05:53:05.820012+0000","last_undegraded":"2024-05-08T05:53:05.820012+0000","last_fullsized":"2024-05-08T05:53:05.820012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:21.393242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579619+0000","last_change":"2024-05-08T05:52:05.037427+0000","last_active":"2024-05-08T05:53:07.579619+0000","last_peered":"2024-05-08T05:53:07.579619+0000","last_clean":"2024-05-08T05:53:07.579619+0000","last_became_active":"2024-05-08T05:52:05.023644+0000","last_became_peered":"2024-05-08T05:52:05.023644+0000","last_unstale":"2024-05-08T05:53:07.579619+0000","last_undegraded":"2024-05-08T05:53:07.579619+0000","last_fullsized":"2024-05-08T05:53:07.579619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:49.193375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269166+0000","last_change":"2024-05-08T05:51:01.187171+0000","last_active":"2024-05-08T05:53:05.269166+0000","last_peered":"2024-05-08T05:53:05.269166+0000","last_clean":"2024-05-08T05:53:05.269166+0000","last_became_active":"2024-05-08T05:51:01.186142+0000","last_became_peered":"2024-05-08T05:51:01.186142+0000","last_unstale":"2024-05-08T05:53:05.269166+0000","last_undegraded":"2024-05-08T05:53:05.269166+0000","last_fullsized":"2024-05-08T05:53:05.269166+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:37:17.021905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820912+0000","last_change":"2024-05-08T05:51:04.311538+0000","last_active":"2024-05-08T05:53:05.820912+0000","last_peered":"2024-05-08T05:53:05.820912+0000","last_clean":"2024-05-08T05:53:05.820912+0000","last_became_active":"2024-05-08T05:51:04.310535+0000","last_became_peered":"2024-05-08T05:51:04.310535+0000","last_unstale":"2024-05-08T05:53:05.820912+0000","last_undegraded":"2024-05-08T05:53:05.820912+0000","last_fullsized":"2024-05-08T05:53:05.820912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:01:04.497986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'13","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:13.203771+0000","last_change":"2024-05-08T05:51:00.277923+0000","last_active":"2024-05-08T05:53:13.203771+0000","last_peered":"2024-05-08T05:53:13.203771+0000","last_clean":"2024-05-08T05:53:13.203771+0000","last_became_active":"2024-05-08T05:51:00.274896+0000","last_became_peered":"2024-05-08T05:51:00.274896+0000","last_unstale":"2024-05-08T05:53:13.203771+0000","last_undegraded":"2024-05-08T05:53:13.203771+0000","last_fullsized":"2024-05-08T05:53:13.203771+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:04:32.543451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.573064+0000","last_change":"2024-05-08T05:51:04.306185+0000","last_active":"2024-05-08T05:53:05.573064+0000","last_peered":"2024-05-08T05:53:05.573064+0000","last_clean":"2024-05-08T05:53:05.573064+0000","last_became_active":"2024-05-08T05:51:04.305229+0000","last_became_peered":"2024-05-08T05:51:04.305229+0000","last_unstale":"2024-05-08T05:53:05.573064+0000","last_undegraded":"2024-05-08T05:53:05.573064+0000","last_fullsized":"2024-05-08T05:53:05.573064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:48.859978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715588+0000","last_change":"2024-05-08T05:51:01.140178+0000","last_active":"2024-05-08T05:53:05.715588+0000","last_peered":"2024-05-08T05:53:05.715588+0000","last_clean":"2024-05-08T05:53:05.715588+0000","last_became_active":"2024-05-08T05:51:01.139772+0000","last_became_peered":"2024-05-08T05:51:01.139772+0000","last_unstale":"2024-05-08T05:53:05.715588+0000","last_undegraded":"2024-05-08T05:53:05.715588+0000","last_fullsized":"2024-05-08T05:53:05.715588+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:54:40.132683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032654+0000","last_change":"2024-05-08T05:51:04.319015+0000","last_active":"2024-05-08T05:53:07.032654+0000","last_peered":"2024-05-08T05:53:07.032654+0000","last_clean":"2024-05-08T05:53:07.032654+0000","last_became_active":"2024-05-08T05:51:04.318465+0000","last_became_peered":"2024-05-08T05:51:04.318465+0000","last_unstale":"2024-05-08T05:53:07.032654+0000","last_undegraded":"2024-05-08T05:53:07.032654+0000","last_fullsized":"2024-05-08T05:53:07.032654+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:07:19.336656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269322+0000","last_change":"2024-05-08T05:51:01.185619+0000","last_active":"2024-05-08T05:53:05.269322+0000","last_peered":"2024-05-08T05:53:05.269322+0000","last_clean":"2024-05-08T05:53:05.269322+0000","last_became_active":"2024-05-08T05:51:01.185325+0000","last_became_peered":"2024-05-08T05:51:01.185325+0000","last_unstale":"2024-05-08T05:53:05.269322+0000","last_undegraded":"2024-05-08T05:53:05.269322+0000","last_fullsized":"2024-05-08T05:53:05.269322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:41.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268756+0000","last_change":"2024-05-08T05:51:04.295018+0000","last_active":"2024-05-08T05:53:05.268756+0000","last_peered":"2024-05-08T05:53:05.268756+0000","last_clean":"2024-05-08T05:53:05.268756+0000","last_became_active":"2024-05-08T05:51:04.294432+0000","last_became_peered":"2024-05-08T05:51:04.294432+0000","last_unstale":"2024-05-08T05:53:05.268756+0000","last_undegraded":"2024-05-08T05:53:05.268756+0000","last_fullsized":"2024-05-08T05:53:05.268756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:01.925456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571832+0000","last_change":"2024-05-08T05:51:00.280527+0000","last_active":"2024-05-08T05:53:05.571832+0000","last_peered":"2024-05-08T05:53:05.571832+0000","last_clean":"2024-05-08T05:53:05.571832+0000","last_became_active":"2024-05-08T05:51:00.278718+0000","last_became_peered":"2024-05-08T05:51:00.278718+0000","last_unstale":"2024-05-08T05:53:05.571832+0000","last_undegraded":"2024-05-08T05:53:05.571832+0000","last_fullsized":"2024-05-08T05:53:05.571832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:48:27.149713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579777+0000","last_change":"2024-05-08T05:51:04.314543+0000","last_active":"2024-05-08T05:53:07.579777+0000","last_peered":"2024-05-08T05:53:07.579777+0000","last_clean":"2024-05-08T05:53:07.579777+0000","last_became_active":"2024-05-08T05:51:04.314168+0000","last_became_peered":"2024-05-08T05:51:04.314168+0000","last_unstale":"2024-05-08T05:53:07.579777+0000","last_undegraded":"2024-05-08T05:53:07.579777+0000","last_fullsized":"2024-05-08T05:53:07.579777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:10:23.215322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:11.750983+0000","last_change":"2024-05-08T05:51:00.268081+0000","last_active":"2024-05-08T05:53:11.750983+0000","last_peered":"2024-05-08T05:53:11.750983+0000","last_clean":"2024-05-08T05:53:11.750983+0000","last_became_active":"2024-05-08T05:51:00.267470+0000","last_became_peered":"2024-05-08T05:51:00.267470+0000","last_unstale":"2024-05-08T05:53:11.750983+0000","last_undegraded":"2024-05-08T05:53:11.750983+0000","last_fullsized":"2024-05-08T05:53:11.750983+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:45.796019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572395+0000","last_change":"2024-05-08T05:51:04.322010+0000","last_active":"2024-05-08T05:53:05.572395+0000","last_peered":"2024-05-08T05:53:05.572395+0000","last_clean":"2024-05-08T05:53:05.572395+0000","last_became_active":"2024-05-08T05:51:04.320867+0000","last_became_peered":"2024-05-08T05:51:04.320867+0000","last_unstale":"2024-05-08T05:53:05.572395+0000","last_undegraded":"2024-05-08T05:53:05.572395+0000","last_fullsized":"2024-05-08T05:53:05.572395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:49.853518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268990+0000","last_change":"2024-05-08T05:52:05.045290+0000","last_active":"2024-05-08T05:53:05.268990+0000","last_peered":"2024-05-08T05:53:05.268990+0000","last_clean":"2024-05-08T05:53:05.268990+0000","last_became_active":"2024-05-08T05:52:05.029284+0000","last_became_peered":"2024-05-08T05:52:05.029284+0000","last_unstale":"2024-05-08T05:53:05.268990+0000","last_undegraded":"2024-05-08T05:53:05.268990+0000","last_fullsized":"2024-05-08T05:53:05.268990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:20.491862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139647+0000","last_change":"2024-05-08T05:51:04.302813+0000","last_active":"2024-05-08T05:53:07.139647+0000","last_peered":"2024-05-08T05:53:07.139647+0000","last_clean":"2024-05-08T05:53:07.139647+0000","last_became_active":"2024-05-08T05:51:04.302424+0000","last_became_peered":"2024-05-08T05:51:04.302424+0000","last_unstale":"2024-05-08T05:53:07.139647+0000","last_undegraded":"2024-05-08T05:53:07.139647+0000","last_fullsized":"2024-05-08T05:53:07.139647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:52:17.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505505+0000","last_change":"2024-05-08T05:51:01.187513+0000","last_active":"2024-05-08T05:53:05.505505+0000","last_peered":"2024-05-08T05:53:05.505505+0000","last_clean":"2024-05-08T05:53:05.505505+0000","last_became_active":"2024-05-08T05:51:01.186780+0000","last_became_peered":"2024-05-08T05:51:01.186780+0000","last_unstale":"2024-05-08T05:53:05.505505+0000","last_undegraded":"2024-05-08T05:53:05.505505+0000","last_fullsized":"2024-05-08T05:53:05.505505+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:42:26.206351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139833+0000","last_change":"2024-05-08T05:51:04.295338+0000","last_active":"2024-05-08T05:53:07.139833+0000","last_peered":"2024-05-08T05:53:07.139833+0000","last_clean":"2024-05-08T05:53:07.139833+0000","last_became_active":"2024-05-08T05:51:04.294990+0000","last_became_peered":"2024-05-08T05:51:04.294990+0000","last_unstale":"2024-05-08T05:53:07.139833+0000","last_undegraded":"2024-05-08T05:53:07.139833+0000","last_fullsized":"2024-05-08T05:53:07.139833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:29.411662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270051+0000","last_change":"2024-05-08T05:51:00.273052+0000","last_active":"2024-05-08T05:53:05.270051+0000","last_peered":"2024-05-08T05:53:05.270051+0000","last_clean":"2024-05-08T05:53:05.270051+0000","last_became_active":"2024-05-08T05:51:00.272718+0000","last_became_peered":"2024-05-08T05:51:00.272718+0000","last_unstale":"2024-05-08T05:53:05.270051+0000","last_undegraded":"2024-05-08T05:53:05.270051+0000","last_fullsized":"2024-05-08T05:53:05.270051+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:44.494378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139455+0000","last_change":"2024-05-08T05:51:04.306209+0000","last_active":"2024-05-08T05:53:07.139455+0000","last_peered":"2024-05-08T05:53:07.139455+0000","last_clean":"2024-05-08T05:53:07.139455+0000","last_became_active":"2024-05-08T05:51:04.305752+0000","last_became_peered":"2024-05-08T05:51:04.305752+0000","last_unstale":"2024-05-08T05:53:07.139455+0000","last_undegraded":"2024-05-08T05:53:07.139455+0000","last_fullsized":"2024-05-08T05:53:07.139455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:24.586111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820689+0000","last_change":"2024-05-08T05:51:00.278426+0000","last_active":"2024-05-08T05:53:05.820689+0000","last_peered":"2024-05-08T05:53:05.820689+0000","last_clean":"2024-05-08T05:53:05.820689+0000","last_became_active":"2024-05-08T05:51:00.276764+0000","last_became_peered":"2024-05-08T05:51:00.276764+0000","last_unstale":"2024-05-08T05:53:05.820689+0000","last_undegraded":"2024-05-08T05:53:05.820689+0000","last_fullsized":"2024-05-08T05:53:05.820689+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:34.158628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139683+0000","last_change":"2024-05-08T05:51:04.294511+0000","last_active":"2024-05-08T05:53:07.139683+0000","last_peered":"2024-05-08T05:53:07.139683+0000","last_clean":"2024-05-08T05:53:07.139683+0000","last_became_active":"2024-05-08T05:51:04.294145+0000","last_became_peered":"2024-05-08T05:51:04.294145+0000","last_unstale":"2024-05-08T05:53:07.139683+0000","last_undegraded":"2024-05-08T05:53:07.139683+0000","last_fullsized":"2024-05-08T05:53:07.139683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:10.034060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579833+0000","last_change":"2024-05-08T05:51:04.298905+0000","last_active":"2024-05-08T05:53:07.579833+0000","last_peered":"2024-05-08T05:53:07.579833+0000","last_clean":"2024-05-08T05:53:07.579833+0000","last_became_active":"2024-05-08T05:51:04.298586+0000","last_became_peered":"2024-05-08T05:51:04.298586+0000","last_unstale":"2024-05-08T05:53:07.579833+0000","last_undegraded":"2024-05-08T05:53:07.579833+0000","last_fullsized":"2024-05-08T05:53:07.579833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:43:24.899015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505323+0000","last_change":"2024-05-08T05:51:04.300503+0000","last_active":"2024-05-08T05:53:05.505323+0000","last_peered":"2024-05-08T05:53:05.505323+0000","last_clean":"2024-05-08T05:53:05.505323+0000","last_became_active":"2024-05-08T05:51:04.299771+0000","last_became_peered":"2024-05-08T05:51:04.299771+0000","last_unstale":"2024-05-08T05:53:05.505323+0000","last_undegraded":"2024-05-08T05:53:05.505323+0000","last_fullsized":"2024-05-08T05:53:05.505323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:19.790501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505267+0000","last_change":"2024-05-08T05:52:05.027724+0000","last_active":"2024-05-08T05:53:05.505267+0000","last_peered":"2024-05-08T05:53:05.505267+0000","last_clean":"2024-05-08T05:53:05.505267+0000","last_became_active":"2024-05-08T05:52:05.027338+0000","last_became_peered":"2024-05-08T05:52:05.027338+0000","last_unstale":"2024-05-08T05:53:05.505267+0000","last_undegraded":"2024-05-08T05:53:05.505267+0000","last_fullsized":"2024-05-08T05:53:05.505267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:03.324348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032152+0000","last_change":"2024-05-08T05:52:05.574645+0000","last_active":"2024-05-08T05:53:07.032152+0000","last_peered":"2024-05-08T05:53:07.032152+0000","last_clean":"2024-05-08T05:53:07.032152+0000","last_became_active":"2024-05-08T05:52:05.573751+0000","last_became_peered":"2024-05-08T05:52:05.573751+0000","last_unstale":"2024-05-08T05:53:07.032152+0000","last_undegraded":"2024-05-08T05:53:07.032152+0000","last_fullsized":"2024-05-08T05:53:07.032152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:57:40.656001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820990+0000","last_change":"2024-05-08T05:51:04.315769+0000","last_active":"2024-05-08T05:53:05.820990+0000","last_peered":"2024-05-08T05:53:05.820990+0000","last_clean":"2024-05-08T05:53:05.820990+0000","last_became_active":"2024-05-08T05:51:04.315322+0000","last_became_peered":"2024-05-08T05:51:04.315322+0000","last_unstale":"2024-05-08T05:53:05.820990+0000","last_undegraded":"2024-05-08T05:53:05.820990+0000","last_fullsized":"2024-05-08T05:53:05.820990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:52.096556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579878+0000","last_change":"2024-05-08T05:51:04.298128+0000","last_active":"2024-05-08T05:53:07.579878+0000","last_peered":"2024-05-08T05:53:07.579878+0000","last_clean":"2024-05-08T05:53:07.579878+0000","last_became_active":"2024-05-08T05:51:04.297743+0000","last_became_peered":"2024-05-08T05:51:04.297743+0000","last_unstale":"2024-05-08T05:53:07.579878+0000","last_undegraded":"2024-05-08T05:53:07.579878+0000","last_fullsized":"2024-05-08T05:53:07.579878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:18.334026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505656+0000","last_change":"2024-05-08T05:51:04.301056+0000","last_active":"2024-05-08T05:53:05.505656+0000","last_peered":"2024-05-08T05:53:05.505656+0000","last_clean":"2024-05-08T05:53:05.505656+0000","last_became_active":"2024-05-08T05:51:04.300733+0000","last_became_peered":"2024-05-08T05:51:04.300733+0000","last_unstale":"2024-05-08T05:53:05.505656+0000","last_undegraded":"2024-05-08T05:53:05.505656+0000","last_fullsized":"2024-05-08T05:53:05.505656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:21.616867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572880+0000","last_change":"2024-05-08T05:51:04.299079+0000","last_active":"2024-05-08T05:53:05.572880+0000","last_peered":"2024-05-08T05:53:05.572880+0000","last_clean":"2024-05-08T05:53:05.572880+0000","last_became_active":"2024-05-08T05:51:04.298628+0000","last_became_peered":"2024-05-08T05:51:04.298628+0000","last_unstale":"2024-05-08T05:53:05.572880+0000","last_undegraded":"2024-05-08T05:53:05.572880+0000","last_fullsized":"2024-05-08T05:53:05.572880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:45.031695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270220+0000","last_change":"2024-05-08T05:51:04.296140+0000","last_active":"2024-05-08T05:53:05.270220+0000","last_peered":"2024-05-08T05:53:05.270220+0000","last_clean":"2024-05-08T05:53:05.270220+0000","last_became_active":"2024-05-08T05:51:04.294940+0000","last_became_peered":"2024-05-08T05:51:04.294940+0000","last_unstale":"2024-05-08T05:53:05.270220+0000","last_undegraded":"2024-05-08T05:53:05.270220+0000","last_fullsized":"2024-05-08T05:53:05.270220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:37.192639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571649+0000","last_change":"2024-05-08T05:51:04.305803+0000","last_active":"2024-05-08T05:53:05.571649+0000","last_peered":"2024-05-08T05:53:05.571649+0000","last_clean":"2024-05-08T05:53:05.571649+0000","last_became_active":"2024-05-08T05:51:04.305482+0000","last_became_peered":"2024-05-08T05:51:04.305482+0000","last_unstale":"2024-05-08T05:53:05.571649+0000","last_undegraded":"2024-05-08T05:53:05.571649+0000","last_fullsized":"2024-05-08T05:53:05.571649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:02.050153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269746+0000","last_change":"2024-05-08T05:51:04.293787+0000","last_active":"2024-05-08T05:53:05.269746+0000","last_peered":"2024-05-08T05:53:05.269746+0000","last_clean":"2024-05-08T05:53:05.269746+0000","last_became_active":"2024-05-08T05:51:04.293455+0000","last_became_peered":"2024-05-08T05:51:04.293455+0000","last_unstale":"2024-05-08T05:53:05.269746+0000","last_undegraded":"2024-05-08T05:53:05.269746+0000","last_fullsized":"2024-05-08T05:53:05.269746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:09:53.211411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820895+0000","last_change":"2024-05-08T05:51:04.321442+0000","last_active":"2024-05-08T05:53:05.820895+0000","last_peered":"2024-05-08T05:53:05.820895+0000","last_clean":"2024-05-08T05:53:05.820895+0000","last_became_active":"2024-05-08T05:51:04.320982+0000","last_became_peered":"2024-05-08T05:51:04.320982+0000","last_unstale":"2024-05-08T05:53:05.820895+0000","last_undegraded":"2024-05-08T05:53:05.820895+0000","last_fullsized":"2024-05-08T05:53:05.820895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:45.089609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579671+0000","last_change":"2024-05-08T05:51:04.294866+0000","last_active":"2024-05-08T05:53:07.579671+0000","last_peered":"2024-05-08T05:53:07.579671+0000","last_clean":"2024-05-08T05:53:07.579671+0000","last_became_active":"2024-05-08T05:51:04.294504+0000","last_became_peered":"2024-05-08T05:51:04.294504+0000","last_unstale":"2024-05-08T05:53:07.579671+0000","last_undegraded":"2024-05-08T05:53:07.579671+0000","last_fullsized":"2024-05-08T05:53:07.579671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:33.139864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71075,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1305,"num_read_kb":729,"num_write":181,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":134,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":213225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645376,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":452538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841604,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1111152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005131,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":452752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201360,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":452650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430293,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1540096,"data_stored":1045053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593819,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":452141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":518669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":453412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":520105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182507,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":452452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986278,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":453110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790051,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36852,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708300,"statfs":{"total":95995035648,"available":95957299200,"internally_reserved":0,"allocated":1552384,"data_stored":1044205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:55:05.424 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-08T05:55:05.424 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:55:05.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.668+0000 7f67ca800640 1 -- 172.21.15.112:0/512297886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67c4078f50 msgr2=0x7f67c40797f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:05.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.668+0000 7f67ca800640 1 --2- 172.21.15.112:0/512297886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67c4078f50 0x7f67c40797f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009be0 tx=0x7f67bc02a6c0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.668+0000 7f67ca800640 1 -- 172.21.15.112:0/512297886 shutdown_connections 2024-05-08T05:55:05.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.668+0000 7f67ca800640 1 --2- 172.21.15.112:0/512297886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67c4078f50 0x7f67c40797f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.668+0000 7f67ca800640 1 --2- 172.21.15.112:0/512297886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67c40785b0 0x7f67c4078a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.668+0000 7f67ca800640 1 --2- 172.21.15.112:0/512297886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c4077370 0x7f67c4077790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.668+0000 7f67ca800640 1 -- 172.21.15.112:0/512297886 >> 172.21.15.112:0/512297886 conn(0x7f67c41001a0 msgr2=0x7f67c41025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:05.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.668+0000 7f67ca800640 1 -- 172.21.15.112:0/512297886 shutdown_connections 2024-05-08T05:55:05.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.668+0000 7f67ca800640 1 -- 172.21.15.112:0/512297886 wait complete. 2024-05-08T05:55:05.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.669+0000 7f67ca800640 1 Processor -- start 2024-05-08T05:55:05.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.669+0000 7f67ca800640 1 -- start start 2024-05-08T05:55:05.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.670+0000 7f67ca800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67c4077370 0x7f67c419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:05.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.670+0000 7f67ca800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67c40785b0 0x7f67c419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:05.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.670+0000 7f67ca800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c4078f50 0x7f67c41a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:05.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.670+0000 7f67ca800640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4114b70 con 0x7f67c4078f50 2024-05-08T05:55:05.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.670+0000 7f67ca800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41149f0 con 0x7f67c4077370 2024-05-08T05:55:05.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.670+0000 7f67ca800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4114cf0 con 0x7f67c40785b0 2024-05-08T05:55:05.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.670+0000 7f67c3e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c4078f50 0x7f67c41a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:05.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.670+0000 7f67c3e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c4078f50 0x7f67c41a67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33096/0 (socket says 172.21.15.112:33096) 2024-05-08T05:55:05.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.670+0000 7f67c3e00640 1 -- 172.21.15.112:0/3505530604 learned_addr learned my addr 172.21.15.112:0/3505530604 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:05.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.670+0000 7f67c2a00640 1 --2- 172.21.15.112:0/3505530604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67c40785b0 0x7f67c419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:05.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.670+0000 7f67c3400640 1 --2- 172.21.15.112:0/3505530604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67c4077370 0x7f67c419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:05.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.671+0000 7f67c2a00640 1 -- 172.21.15.112:0/3505530604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67c4077370 msgr2=0x7f67c419f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:05.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.671+0000 7f67c2a00640 1 --2- 172.21.15.112:0/3505530604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67c4077370 0x7f67c419f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.671+0000 7f67c2a00640 1 -- 172.21.15.112:0/3505530604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c4078f50 msgr2=0x7f67c41a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:05.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.671+0000 7f67c2a00640 1 --2- 172.21.15.112:0/3505530604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c4078f50 0x7f67c41a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.671+0000 7f67c2a00640 1 -- 172.21.15.112:0/3505530604 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c41a6e10 con 0x7f67c40785b0 2024-05-08T05:55:05.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.671+0000 7f67c2a00640 1 --2- 172.21.15.112:0/3505530604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67c40785b0 0x7f67c419f730 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f67b000e9f0 tx=0x7f67b000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:05.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.671+0000 7f67bbe00640 1 -- 172.21.15.112:0/3505530604 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b000ccc0 con 0x7f67c40785b0 2024-05-08T05:55:05.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.671+0000 7f67bbe00640 1 -- 172.21.15.112:0/3505530604 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67b00108e0 con 0x7f67c40785b0 2024-05-08T05:55:05.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.671+0000 7f67ca800640 1 -- 172.21.15.112:0/3505530604 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c41a7100 con 0x7f67c40785b0 2024-05-08T05:55:05.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.671+0000 7f67bbe00640 1 -- 172.21.15.112:0/3505530604 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0018840 con 0x7f67c40785b0 2024-05-08T05:55:05.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.671+0000 7f67ca800640 1 -- 172.21.15.112:0/3505530604 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c41a76a0 con 0x7f67c40785b0 2024-05-08T05:55:05.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.673+0000 7f67ca800640 1 -- 172.21.15.112:0/3505530604 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c41115a0 con 0x7f67c40785b0 2024-05-08T05:55:05.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.676+0000 7f67bbe00640 1 -- 172.21.15.112:0/3505530604 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f67b0010430 con 0x7f67c40785b0 2024-05-08T05:55:05.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.677+0000 7f67bbe00640 1 --2- 172.21.15.112:0/3505530604 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f67a80b2870 0x7f67a80b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:05.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.678+0000 7f67bbe00640 1 -- 172.21.15.112:0/3505530604 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f67b00e2970 con 0x7f67c40785b0 2024-05-08T05:55:05.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.678+0000 7f67c3400640 1 --2- 172.21.15.112:0/3505530604 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f67a80b2870 0x7f67a80b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:05.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.678+0000 7f67bbe00640 1 -- 172.21.15.112:0/3505530604 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f67b00aa600 con 0x7f67c40785b0 2024-05-08T05:55:05.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.678+0000 7f67c3400640 1 --2- 172.21.15.112:0/3505530604 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f67a80b2870 0x7f67a80b4d30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f67b4005fd0 tx=0x7f67b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:05.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.884+0000 7f67ca800640 1 -- 172.21.15.112:0/3505530604 --> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67c4062df0 con 0x7f67a80b2870 2024-05-08T05:55:05.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.907+0000 7f67bbe00640 1 -- 172.21.15.112:0/3505530604 <== mgr.14152 v2:172.21.15.112:6800/1211679607 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258834 (secure 0 0 0) 0x7f67c4062df0 con 0x7f67a80b2870 2024-05-08T05:55:05.908 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:55:05.912 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-05-08T05:55:05.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.913+0000 7f67ca800640 1 -- 172.21.15.112:0/3505530604 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f67a80b2870 msgr2=0x7f67a80b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:05.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.913+0000 7f67ca800640 1 --2- 172.21.15.112:0/3505530604 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f67a80b2870 0x7f67a80b4d30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f67b4005fd0 tx=0x7f67b4009260 comp rx=0 tx=0).stop 2024-05-08T05:55:05.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.913+0000 7f67ca800640 1 -- 172.21.15.112:0/3505530604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67c40785b0 msgr2=0x7f67c419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:05.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.914+0000 7f67ca800640 1 --2- 172.21.15.112:0/3505530604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67c40785b0 0x7f67c419f730 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f67b000e9f0 tx=0x7f67b000edf0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.914+0000 7f67ca800640 1 -- 172.21.15.112:0/3505530604 shutdown_connections 2024-05-08T05:55:05.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.914+0000 7f67ca800640 1 --2- 172.21.15.112:0/3505530604 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f67a80b2870 0x7f67a80b4d30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.914+0000 7f67ca800640 1 --2- 172.21.15.112:0/3505530604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c4078f50 0x7f67c41a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.914+0000 7f67ca800640 1 --2- 172.21.15.112:0/3505530604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67c40785b0 0x7f67c419f730 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.914+0000 7f67ca800640 1 --2- 172.21.15.112:0/3505530604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67c4077370 0x7f67c419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.914+0000 7f67ca800640 1 -- 172.21.15.112:0/3505530604 >> 172.21.15.112:0/3505530604 conn(0x7f67c41001a0 msgr2=0x7f67c4101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:05.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.914+0000 7f67ca800640 1 -- 172.21.15.112:0/3505530604 shutdown_connections 2024-05-08T05:55:05.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:05.914+0000 7f67ca800640 1 -- 172.21.15.112:0/3505530604 wait complete. 2024-05-08T05:55:05.934 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-05-08T05:55:05.568559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661677,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1406,"num_read_kb":803,"num_write":306,"num_write_kb":2280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951974,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":747,"ondisk_log_size":747,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366380,"kb_used_data":13044,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575444,"statfs":{"total":1151940427776,"available":1151565254656,"internally_reserved":0,"allocated":13357056,"data_stored":7408239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4934,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":47,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004776"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.580018+0000","last_change":"2024-05-08T05:51:04.320319+0000","last_active":"2024-05-08T05:53:07.580018+0000","last_peered":"2024-05-08T05:53:07.580018+0000","last_clean":"2024-05-08T05:53:07.580018+0000","last_became_active":"2024-05-08T05:51:04.319961+0000","last_became_peered":"2024-05-08T05:51:04.319961+0000","last_unstale":"2024-05-08T05:53:07.580018+0000","last_undegraded":"2024-05-08T05:53:07.580018+0000","last_fullsized":"2024-05-08T05:53:07.580018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:32.463261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032466+0000","last_change":"2024-05-08T05:51:04.315531+0000","last_active":"2024-05-08T05:53:07.032466+0000","last_peered":"2024-05-08T05:53:07.032466+0000","last_clean":"2024-05-08T05:53:07.032466+0000","last_became_active":"2024-05-08T05:51:04.314116+0000","last_became_peered":"2024-05-08T05:51:04.314116+0000","last_unstale":"2024-05-08T05:53:07.032466+0000","last_undegraded":"2024-05-08T05:53:07.032466+0000","last_fullsized":"2024-05-08T05:53:07.032466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:12:03.505888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032236+0000","last_change":"2024-05-08T05:51:04.315826+0000","last_active":"2024-05-08T05:53:07.032236+0000","last_peered":"2024-05-08T05:53:07.032236+0000","last_clean":"2024-05-08T05:53:07.032236+0000","last_became_active":"2024-05-08T05:51:04.315460+0000","last_became_peered":"2024-05-08T05:51:04.315460+0000","last_unstale":"2024-05-08T05:53:07.032236+0000","last_undegraded":"2024-05-08T05:53:07.032236+0000","last_fullsized":"2024-05-08T05:53:07.032236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:22:20.412368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715495+0000","last_change":"2024-05-08T05:51:04.297401+0000","last_active":"2024-05-08T05:53:05.715495+0000","last_peered":"2024-05-08T05:53:05.715495+0000","last_clean":"2024-05-08T05:53:05.715495+0000","last_became_active":"2024-05-08T05:51:04.297061+0000","last_became_peered":"2024-05-08T05:51:04.297061+0000","last_unstale":"2024-05-08T05:53:05.715495+0000","last_undegraded":"2024-05-08T05:53:05.715495+0000","last_fullsized":"2024-05-08T05:53:05.715495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:27.967517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268821+0000","last_change":"2024-05-08T05:51:04.315069+0000","last_active":"2024-05-08T05:53:05.268821+0000","last_peered":"2024-05-08T05:53:05.268821+0000","last_clean":"2024-05-08T05:53:05.268821+0000","last_became_active":"2024-05-08T05:51:04.314513+0000","last_became_peered":"2024-05-08T05:51:04.314513+0000","last_unstale":"2024-05-08T05:53:05.268821+0000","last_undegraded":"2024-05-08T05:53:05.268821+0000","last_fullsized":"2024-05-08T05:53:05.268821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:02.520130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265219+0000","last_change":"2024-05-08T05:51:04.302851+0000","last_active":"2024-05-08T05:53:05.265219+0000","last_peered":"2024-05-08T05:53:05.265219+0000","last_clean":"2024-05-08T05:53:05.265219+0000","last_became_active":"2024-05-08T05:51:04.302256+0000","last_became_peered":"2024-05-08T05:51:04.302256+0000","last_unstale":"2024-05-08T05:53:05.265219+0000","last_undegraded":"2024-05-08T05:53:05.265219+0000","last_fullsized":"2024-05-08T05:53:05.265219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:45.587319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139305+0000","last_change":"2024-05-08T05:51:04.308674+0000","last_active":"2024-05-08T05:53:07.139305+0000","last_peered":"2024-05-08T05:53:07.139305+0000","last_clean":"2024-05-08T05:53:07.139305+0000","last_became_active":"2024-05-08T05:51:04.308295+0000","last_became_peered":"2024-05-08T05:51:04.308295+0000","last_unstale":"2024-05-08T05:53:07.139305+0000","last_undegraded":"2024-05-08T05:53:07.139305+0000","last_fullsized":"2024-05-08T05:53:07.139305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:59:17.561655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571645+0000","last_change":"2024-05-08T05:51:04.314511+0000","last_active":"2024-05-08T05:53:05.571645+0000","last_peered":"2024-05-08T05:53:05.571645+0000","last_clean":"2024-05-08T05:53:05.571645+0000","last_became_active":"2024-05-08T05:51:04.314082+0000","last_became_peered":"2024-05-08T05:51:04.314082+0000","last_unstale":"2024-05-08T05:53:05.571645+0000","last_undegraded":"2024-05-08T05:53:05.571645+0000","last_fullsized":"2024-05-08T05:53:05.571645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:03:24.182391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269793+0000","last_change":"2024-05-08T05:51:04.293745+0000","last_active":"2024-05-08T05:53:05.269793+0000","last_peered":"2024-05-08T05:53:05.269793+0000","last_clean":"2024-05-08T05:53:05.269793+0000","last_became_active":"2024-05-08T05:51:04.293334+0000","last_became_peered":"2024-05-08T05:51:04.293334+0000","last_unstale":"2024-05-08T05:53:05.269793+0000","last_undegraded":"2024-05-08T05:53:05.269793+0000","last_fullsized":"2024-05-08T05:53:05.269793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:16.576967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265481+0000","last_change":"2024-05-08T05:51:04.297800+0000","last_active":"2024-05-08T05:53:05.265481+0000","last_peered":"2024-05-08T05:53:05.265481+0000","last_clean":"2024-05-08T05:53:05.265481+0000","last_became_active":"2024-05-08T05:51:04.297349+0000","last_became_peered":"2024-05-08T05:51:04.297349+0000","last_unstale":"2024-05-08T05:53:05.265481+0000","last_undegraded":"2024-05-08T05:53:05.265481+0000","last_fullsized":"2024-05-08T05:53:05.265481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:55.784512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139645+0000","last_change":"2024-05-08T05:51:04.297997+0000","last_active":"2024-05-08T05:53:07.139645+0000","last_peered":"2024-05-08T05:53:07.139645+0000","last_clean":"2024-05-08T05:53:07.139645+0000","last_became_active":"2024-05-08T05:51:04.297516+0000","last_became_peered":"2024-05-08T05:51:04.297516+0000","last_unstale":"2024-05-08T05:53:07.139645+0000","last_undegraded":"2024-05-08T05:53:07.139645+0000","last_fullsized":"2024-05-08T05:53:07.139645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:25.991555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715483+0000","last_change":"2024-05-08T05:51:04.295610+0000","last_active":"2024-05-08T05:53:05.715483+0000","last_peered":"2024-05-08T05:53:05.715483+0000","last_clean":"2024-05-08T05:53:05.715483+0000","last_became_active":"2024-05-08T05:51:04.294553+0000","last_became_peered":"2024-05-08T05:51:04.294553+0000","last_unstale":"2024-05-08T05:53:05.715483+0000","last_undegraded":"2024-05-08T05:53:05.715483+0000","last_fullsized":"2024-05-08T05:53:05.715483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:40:14.492057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820666+0000","last_change":"2024-05-08T05:51:04.303879+0000","last_active":"2024-05-08T05:53:05.820666+0000","last_peered":"2024-05-08T05:53:05.820666+0000","last_clean":"2024-05-08T05:53:05.820666+0000","last_became_active":"2024-05-08T05:51:04.303423+0000","last_became_peered":"2024-05-08T05:51:04.303423+0000","last_unstale":"2024-05-08T05:53:05.820666+0000","last_undegraded":"2024-05-08T05:53:05.820666+0000","last_fullsized":"2024-05-08T05:53:05.820666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:50:55.905858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.031822+0000","last_change":"2024-05-08T05:51:04.309818+0000","last_active":"2024-05-08T05:53:07.031822+0000","last_peered":"2024-05-08T05:53:07.031822+0000","last_clean":"2024-05-08T05:53:07.031822+0000","last_became_active":"2024-05-08T05:51:04.309434+0000","last_became_peered":"2024-05-08T05:51:04.309434+0000","last_unstale":"2024-05-08T05:53:07.031822+0000","last_undegraded":"2024-05-08T05:53:07.031822+0000","last_fullsized":"2024-05-08T05:53:07.031822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:55.832239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.031573+0000","last_change":"2024-05-08T05:51:04.316708+0000","last_active":"2024-05-08T05:53:07.031573+0000","last_peered":"2024-05-08T05:53:07.031573+0000","last_clean":"2024-05-08T05:53:07.031573+0000","last_became_active":"2024-05-08T05:51:04.316525+0000","last_became_peered":"2024-05-08T05:51:04.316525+0000","last_unstale":"2024-05-08T05:53:07.031573+0000","last_undegraded":"2024-05-08T05:53:07.031573+0000","last_fullsized":"2024-05-08T05:53:07.031573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:33:38.246611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269307+0000","last_change":"2024-05-08T05:51:04.314780+0000","last_active":"2024-05-08T05:53:05.269307+0000","last_peered":"2024-05-08T05:53:05.269307+0000","last_clean":"2024-05-08T05:53:05.269307+0000","last_became_active":"2024-05-08T05:51:04.314406+0000","last_became_peered":"2024-05-08T05:51:04.314406+0000","last_unstale":"2024-05-08T05:53:05.269307+0000","last_undegraded":"2024-05-08T05:53:05.269307+0000","last_fullsized":"2024-05-08T05:53:05.269307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:39:26.926238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579947+0000","last_change":"2024-05-08T05:51:04.319425+0000","last_active":"2024-05-08T05:53:07.579947+0000","last_peered":"2024-05-08T05:53:07.579947+0000","last_clean":"2024-05-08T05:53:07.579947+0000","last_became_active":"2024-05-08T05:51:04.319106+0000","last_became_peered":"2024-05-08T05:51:04.319106+0000","last_unstale":"2024-05-08T05:53:07.579947+0000","last_undegraded":"2024-05-08T05:53:07.579947+0000","last_fullsized":"2024-05-08T05:53:07.579947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:59:54.023162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267374+0000","last_change":"2024-05-08T05:51:04.303547+0000","last_active":"2024-05-08T05:53:05.267374+0000","last_peered":"2024-05-08T05:53:05.267374+0000","last_clean":"2024-05-08T05:53:05.267374+0000","last_became_active":"2024-05-08T05:51:04.303182+0000","last_became_peered":"2024-05-08T05:51:04.303182+0000","last_unstale":"2024-05-08T05:53:05.267374+0000","last_undegraded":"2024-05-08T05:53:05.267374+0000","last_fullsized":"2024-05-08T05:53:05.267374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:11:59.624329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505589+0000","last_change":"2024-05-08T05:51:04.315674+0000","last_active":"2024-05-08T05:53:05.505589+0000","last_peered":"2024-05-08T05:53:05.505589+0000","last_clean":"2024-05-08T05:53:05.505589+0000","last_became_active":"2024-05-08T05:51:04.314911+0000","last_became_peered":"2024-05-08T05:51:04.314911+0000","last_unstale":"2024-05-08T05:53:05.505589+0000","last_undegraded":"2024-05-08T05:53:05.505589+0000","last_fullsized":"2024-05-08T05:53:05.505589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:49.501508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267963+0000","last_change":"2024-05-08T05:51:00.273554+0000","last_active":"2024-05-08T05:53:05.267963+0000","last_peered":"2024-05-08T05:53:05.267963+0000","last_clean":"2024-05-08T05:53:05.267963+0000","last_became_active":"2024-05-08T05:51:00.272858+0000","last_became_peered":"2024-05-08T05:51:00.272858+0000","last_unstale":"2024-05-08T05:53:05.267963+0000","last_undegraded":"2024-05-08T05:53:05.267963+0000","last_fullsized":"2024-05-08T05:53:05.267963+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:47:37.418887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572498+0000","last_change":"2024-05-08T05:51:04.306392+0000","last_active":"2024-05-08T05:53:05.572498+0000","last_peered":"2024-05-08T05:53:05.572498+0000","last_clean":"2024-05-08T05:53:05.572498+0000","last_became_active":"2024-05-08T05:51:04.304248+0000","last_became_peered":"2024-05-08T05:51:04.304248+0000","last_unstale":"2024-05-08T05:53:05.572498+0000","last_undegraded":"2024-05-08T05:53:05.572498+0000","last_fullsized":"2024-05-08T05:53:05.572498+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:49:26.034883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572828+0000","last_change":"2024-05-08T05:51:01.187292+0000","last_active":"2024-05-08T05:53:05.572828+0000","last_peered":"2024-05-08T05:53:05.572828+0000","last_clean":"2024-05-08T05:53:05.572828+0000","last_became_active":"2024-05-08T05:51:01.185948+0000","last_became_peered":"2024-05-08T05:51:01.185948+0000","last_unstale":"2024-05-08T05:53:05.572828+0000","last_undegraded":"2024-05-08T05:53:05.572828+0000","last_fullsized":"2024-05-08T05:53:05.572828+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:02.927029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032756+0000","last_change":"2024-05-08T05:51:04.318439+0000","last_active":"2024-05-08T05:53:07.032756+0000","last_peered":"2024-05-08T05:53:07.032756+0000","last_clean":"2024-05-08T05:53:07.032756+0000","last_became_active":"2024-05-08T05:51:04.317722+0000","last_became_peered":"2024-05-08T05:51:04.317722+0000","last_unstale":"2024-05-08T05:53:07.032756+0000","last_undegraded":"2024-05-08T05:53:07.032756+0000","last_fullsized":"2024-05-08T05:53:07.032756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:36:43.648643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269296+0000","last_change":"2024-05-08T05:51:00.269684+0000","last_active":"2024-05-08T05:53:05.269296+0000","last_peered":"2024-05-08T05:53:05.269296+0000","last_clean":"2024-05-08T05:53:05.269296+0000","last_became_active":"2024-05-08T05:51:00.269329+0000","last_became_peered":"2024-05-08T05:51:00.269329+0000","last_unstale":"2024-05-08T05:53:05.269296+0000","last_undegraded":"2024-05-08T05:53:05.269296+0000","last_fullsized":"2024-05-08T05:53:05.269296+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:27:10.422994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269410+0000","last_change":"2024-05-08T05:51:04.308048+0000","last_active":"2024-05-08T05:53:05.269410+0000","last_peered":"2024-05-08T05:53:05.269410+0000","last_clean":"2024-05-08T05:53:05.269410+0000","last_became_active":"2024-05-08T05:51:04.307550+0000","last_became_peered":"2024-05-08T05:51:04.307550+0000","last_unstale":"2024-05-08T05:53:05.269410+0000","last_undegraded":"2024-05-08T05:53:05.269410+0000","last_fullsized":"2024-05-08T05:53:05.269410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:13.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269033+0000","last_change":"2024-05-08T05:51:00.268142+0000","last_active":"2024-05-08T05:53:05.269033+0000","last_peered":"2024-05-08T05:53:05.269033+0000","last_clean":"2024-05-08T05:53:05.269033+0000","last_became_active":"2024-05-08T05:51:00.267756+0000","last_became_peered":"2024-05-08T05:51:00.267756+0000","last_unstale":"2024-05-08T05:53:05.269033+0000","last_undegraded":"2024-05-08T05:53:05.269033+0000","last_fullsized":"2024-05-08T05:53:05.269033+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:01.439680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270314+0000","last_change":"2024-05-08T05:51:04.292341+0000","last_active":"2024-05-08T05:53:05.270314+0000","last_peered":"2024-05-08T05:53:05.270314+0000","last_clean":"2024-05-08T05:53:05.270314+0000","last_became_active":"2024-05-08T05:51:04.291086+0000","last_became_peered":"2024-05-08T05:51:04.291086+0000","last_unstale":"2024-05-08T05:53:05.270314+0000","last_undegraded":"2024-05-08T05:53:05.270314+0000","last_fullsized":"2024-05-08T05:53:05.270314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:47.108100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":257,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:55:02.984822+0000","last_change":"2024-05-08T05:51:00.268100+0000","last_active":"2024-05-08T05:55:02.984822+0000","last_peered":"2024-05-08T05:55:02.984822+0000","last_clean":"2024-05-08T05:55:02.984822+0000","last_became_active":"2024-05-08T05:51:00.267678+0000","last_became_peered":"2024-05-08T05:51:00.267678+0000","last_unstale":"2024-05-08T05:55:02.984822+0000","last_undegraded":"2024-05-08T05:55:02.984822+0000","last_fullsized":"2024-05-08T05:55:02.984822+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:11:31.731548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":220,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268766+0000","last_change":"2024-05-08T05:51:04.303872+0000","last_active":"2024-05-08T05:53:05.268766+0000","last_peered":"2024-05-08T05:53:05.268766+0000","last_clean":"2024-05-08T05:53:05.268766+0000","last_became_active":"2024-05-08T05:51:04.303456+0000","last_became_peered":"2024-05-08T05:51:04.303456+0000","last_unstale":"2024-05-08T05:53:05.268766+0000","last_undegraded":"2024-05-08T05:53:05.268766+0000","last_fullsized":"2024-05-08T05:53:05.268766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:42:05.098488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265391+0000","last_change":"2024-05-08T05:51:00.275657+0000","last_active":"2024-05-08T05:53:05.265391+0000","last_peered":"2024-05-08T05:53:05.265391+0000","last_clean":"2024-05-08T05:53:05.265391+0000","last_became_active":"2024-05-08T05:51:00.275254+0000","last_became_peered":"2024-05-08T05:51:00.275254+0000","last_unstale":"2024-05-08T05:53:05.265391+0000","last_undegraded":"2024-05-08T05:53:05.265391+0000","last_fullsized":"2024-05-08T05:53:05.265391+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:28.555698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270118+0000","last_change":"2024-05-08T05:51:04.293791+0000","last_active":"2024-05-08T05:53:05.270118+0000","last_peered":"2024-05-08T05:53:05.270118+0000","last_clean":"2024-05-08T05:53:05.270118+0000","last_became_active":"2024-05-08T05:51:04.293405+0000","last_became_peered":"2024-05-08T05:51:04.293405+0000","last_unstale":"2024-05-08T05:53:05.270118+0000","last_undegraded":"2024-05-08T05:53:05.270118+0000","last_fullsized":"2024-05-08T05:53:05.270118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:28.064452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820147+0000","last_change":"2024-05-08T05:51:00.280295+0000","last_active":"2024-05-08T05:53:05.820147+0000","last_peered":"2024-05-08T05:53:05.820147+0000","last_clean":"2024-05-08T05:53:05.820147+0000","last_became_active":"2024-05-08T05:51:00.279721+0000","last_became_peered":"2024-05-08T05:51:00.279721+0000","last_unstale":"2024-05-08T05:53:05.820147+0000","last_undegraded":"2024-05-08T05:53:05.820147+0000","last_fullsized":"2024-05-08T05:53:05.820147+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:06:38.064872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571814+0000","last_change":"2024-05-08T05:51:04.307546+0000","last_active":"2024-05-08T05:53:05.571814+0000","last_peered":"2024-05-08T05:53:05.571814+0000","last_clean":"2024-05-08T05:53:05.571814+0000","last_became_active":"2024-05-08T05:51:04.307164+0000","last_became_peered":"2024-05-08T05:51:04.307164+0000","last_unstale":"2024-05-08T05:53:05.571814+0000","last_undegraded":"2024-05-08T05:53:05.571814+0000","last_fullsized":"2024-05-08T05:53:05.571814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:36:02.269040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269777+0000","last_change":"2024-05-08T05:51:00.267987+0000","last_active":"2024-05-08T05:53:05.269777+0000","last_peered":"2024-05-08T05:53:05.269777+0000","last_clean":"2024-05-08T05:53:05.269777+0000","last_became_active":"2024-05-08T05:51:00.267633+0000","last_became_peered":"2024-05-08T05:51:00.267633+0000","last_unstale":"2024-05-08T05:53:05.269777+0000","last_undegraded":"2024-05-08T05:53:05.269777+0000","last_fullsized":"2024-05-08T05:53:05.269777+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:13.034300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.266492+0000","last_change":"2024-05-08T05:51:04.311422+0000","last_active":"2024-05-08T05:53:05.266492+0000","last_peered":"2024-05-08T05:53:05.266492+0000","last_clean":"2024-05-08T05:53:05.266492+0000","last_became_active":"2024-05-08T05:51:04.311052+0000","last_became_peered":"2024-05-08T05:51:04.311052+0000","last_unstale":"2024-05-08T05:53:05.266492+0000","last_undegraded":"2024-05-08T05:53:05.266492+0000","last_fullsized":"2024-05-08T05:53:05.266492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:47.790583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":251,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:55:02.984113+0000","last_change":"2024-05-08T05:51:00.274252+0000","last_active":"2024-05-08T05:55:02.984113+0000","last_peered":"2024-05-08T05:55:02.984113+0000","last_clean":"2024-05-08T05:55:02.984113+0000","last_became_active":"2024-05-08T05:51:00.273942+0000","last_became_peered":"2024-05-08T05:51:00.273942+0000","last_unstale":"2024-05-08T05:55:02.984113+0000","last_undegraded":"2024-05-08T05:55:02.984113+0000","last_fullsized":"2024-05-08T05:55:02.984113+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:36.778032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":217,"num_read_kb":217,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269700+0000","last_change":"2024-05-08T05:51:04.296414+0000","last_active":"2024-05-08T05:53:05.269700+0000","last_peered":"2024-05-08T05:53:05.269700+0000","last_clean":"2024-05-08T05:53:05.269700+0000","last_became_active":"2024-05-08T05:51:04.296015+0000","last_became_peered":"2024-05-08T05:51:04.296015+0000","last_unstale":"2024-05-08T05:53:05.269700+0000","last_undegraded":"2024-05-08T05:53:05.269700+0000","last_fullsized":"2024-05-08T05:53:05.269700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:48:15.609385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139516+0000","last_change":"2024-05-08T05:51:00.268234+0000","last_active":"2024-05-08T05:53:07.139516+0000","last_peered":"2024-05-08T05:53:07.139516+0000","last_clean":"2024-05-08T05:53:07.139516+0000","last_became_active":"2024-05-08T05:51:00.267846+0000","last_became_peered":"2024-05-08T05:51:00.267846+0000","last_unstale":"2024-05-08T05:53:07.139516+0000","last_undegraded":"2024-05-08T05:53:07.139516+0000","last_fullsized":"2024-05-08T05:53:07.139516+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:26.919004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032653+0000","last_change":"2024-05-08T05:51:04.318215+0000","last_active":"2024-05-08T05:53:07.032653+0000","last_peered":"2024-05-08T05:53:07.032653+0000","last_clean":"2024-05-08T05:53:07.032653+0000","last_became_active":"2024-05-08T05:51:04.317309+0000","last_became_peered":"2024-05-08T05:51:04.317309+0000","last_unstale":"2024-05-08T05:53:07.032653+0000","last_undegraded":"2024-05-08T05:53:07.032653+0000","last_fullsized":"2024-05-08T05:53:07.032653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:11.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270484+0000","last_change":"2024-05-08T05:51:01.186438+0000","last_active":"2024-05-08T05:53:05.270484+0000","last_peered":"2024-05-08T05:53:05.270484+0000","last_clean":"2024-05-08T05:53:05.270484+0000","last_became_active":"2024-05-08T05:51:01.186039+0000","last_became_peered":"2024-05-08T05:51:01.186039+0000","last_unstale":"2024-05-08T05:53:05.270484+0000","last_undegraded":"2024-05-08T05:53:05.270484+0000","last_fullsized":"2024-05-08T05:53:05.270484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:34.652184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269557+0000","last_change":"2024-05-08T05:52:05.575531+0000","last_active":"2024-05-08T05:53:05.269557+0000","last_peered":"2024-05-08T05:53:05.269557+0000","last_clean":"2024-05-08T05:53:05.269557+0000","last_became_active":"2024-05-08T05:52:05.575042+0000","last_became_peered":"2024-05-08T05:52:05.575042+0000","last_unstale":"2024-05-08T05:53:05.269557+0000","last_undegraded":"2024-05-08T05:53:05.269557+0000","last_fullsized":"2024-05-08T05:53:05.269557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:56.096772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'36","reported_seq":512,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:55:03.985920+0000","last_change":"2024-05-08T05:51:00.263385+0000","last_active":"2024-05-08T05:55:03.985920+0000","last_peered":"2024-05-08T05:55:03.985920+0000","last_clean":"2024-05-08T05:55:03.985920+0000","last_became_active":"2024-05-08T05:51:00.262997+0000","last_became_peered":"2024-05-08T05:51:00.262997+0000","last_unstale":"2024-05-08T05:55:03.985920+0000","last_undegraded":"2024-05-08T05:55:03.985920+0000","last_fullsized":"2024-05-08T05:55:03.985920+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:22:09.727481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66623,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":446,"num_read_kb":288,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269140+0000","last_change":"2024-05-08T05:51:04.294790+0000","last_active":"2024-05-08T05:53:05.269140+0000","last_peered":"2024-05-08T05:53:05.269140+0000","last_clean":"2024-05-08T05:53:05.269140+0000","last_became_active":"2024-05-08T05:51:04.294418+0000","last_became_peered":"2024-05-08T05:51:04.294418+0000","last_unstale":"2024-05-08T05:53:05.269140+0000","last_undegraded":"2024-05-08T05:53:05.269140+0000","last_fullsized":"2024-05-08T05:53:05.269140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:11:10.624434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505362+0000","last_change":"2024-05-08T05:51:00.258507+0000","last_active":"2024-05-08T05:53:05.505362+0000","last_peered":"2024-05-08T05:53:05.505362+0000","last_clean":"2024-05-08T05:53:05.505362+0000","last_became_active":"2024-05-08T05:51:00.258179+0000","last_became_peered":"2024-05-08T05:51:00.258179+0000","last_unstale":"2024-05-08T05:53:05.505362+0000","last_undegraded":"2024-05-08T05:53:05.505362+0000","last_fullsized":"2024-05-08T05:53:05.505362+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:22.376499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820379+0000","last_change":"2024-05-08T05:51:04.310947+0000","last_active":"2024-05-08T05:53:05.820379+0000","last_peered":"2024-05-08T05:53:05.820379+0000","last_clean":"2024-05-08T05:53:05.820379+0000","last_became_active":"2024-05-08T05:51:04.310378+0000","last_became_peered":"2024-05-08T05:51:04.310378+0000","last_unstale":"2024-05-08T05:53:05.820379+0000","last_undegraded":"2024-05-08T05:53:05.820379+0000","last_fullsized":"2024-05-08T05:53:05.820379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:19.367660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270609+0000","last_change":"2024-05-08T05:51:00.271104+0000","last_active":"2024-05-08T05:53:05.270609+0000","last_peered":"2024-05-08T05:53:05.270609+0000","last_clean":"2024-05-08T05:53:05.270609+0000","last_became_active":"2024-05-08T05:51:00.270497+0000","last_became_peered":"2024-05-08T05:51:00.270497+0000","last_unstale":"2024-05-08T05:53:05.270609+0000","last_undegraded":"2024-05-08T05:53:05.270609+0000","last_fullsized":"2024-05-08T05:53:05.270609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:40.959423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139512+0000","last_change":"2024-05-08T05:51:04.310243+0000","last_active":"2024-05-08T05:53:07.139512+0000","last_peered":"2024-05-08T05:53:07.139512+0000","last_clean":"2024-05-08T05:53:07.139512+0000","last_became_active":"2024-05-08T05:51:04.309871+0000","last_became_peered":"2024-05-08T05:51:04.309871+0000","last_unstale":"2024-05-08T05:53:07.139512+0000","last_undegraded":"2024-05-08T05:53:07.139512+0000","last_fullsized":"2024-05-08T05:53:07.139512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:18:21.045621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":256,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:55:02.093168+0000","last_change":"2024-05-08T05:51:00.266096+0000","last_active":"2024-05-08T05:55:02.093168+0000","last_peered":"2024-05-08T05:55:02.093168+0000","last_clean":"2024-05-08T05:55:02.093168+0000","last_became_active":"2024-05-08T05:51:00.265706+0000","last_became_peered":"2024-05-08T05:51:00.265706+0000","last_unstale":"2024-05-08T05:55:02.093168+0000","last_undegraded":"2024-05-08T05:55:02.093168+0000","last_fullsized":"2024-05-08T05:55:02.093168+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:47.388732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":221,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139267+0000","last_change":"2024-05-08T05:51:04.293141+0000","last_active":"2024-05-08T05:53:07.139267+0000","last_peered":"2024-05-08T05:53:07.139267+0000","last_clean":"2024-05-08T05:53:07.139267+0000","last_became_active":"2024-05-08T05:51:04.292772+0000","last_became_peered":"2024-05-08T05:51:04.292772+0000","last_unstale":"2024-05-08T05:53:07.139267+0000","last_undegraded":"2024-05-08T05:53:07.139267+0000","last_fullsized":"2024-05-08T05:53:07.139267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:25:32.801209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268837+0000","last_change":"2024-05-08T05:52:05.042102+0000","last_active":"2024-05-08T05:53:05.268837+0000","last_peered":"2024-05-08T05:53:05.268837+0000","last_clean":"2024-05-08T05:53:05.268837+0000","last_became_active":"2024-05-08T05:52:05.027787+0000","last_became_peered":"2024-05-08T05:52:05.027787+0000","last_unstale":"2024-05-08T05:53:05.268837+0000","last_undegraded":"2024-05-08T05:53:05.268837+0000","last_fullsized":"2024-05-08T05:53:05.268837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:05.580309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269940+0000","last_change":"2024-05-08T05:51:04.290096+0000","last_active":"2024-05-08T05:53:05.269940+0000","last_peered":"2024-05-08T05:53:05.269940+0000","last_clean":"2024-05-08T05:53:05.269940+0000","last_became_active":"2024-05-08T05:51:04.289684+0000","last_became_peered":"2024-05-08T05:51:04.289684+0000","last_unstale":"2024-05-08T05:53:05.269940+0000","last_undegraded":"2024-05-08T05:53:05.269940+0000","last_fullsized":"2024-05-08T05:53:05.269940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:02.400348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579795+0000","last_change":"2024-05-08T05:52:05.034004+0000","last_active":"2024-05-08T05:53:07.579795+0000","last_peered":"2024-05-08T05:53:07.579795+0000","last_clean":"2024-05-08T05:53:07.579795+0000","last_became_active":"2024-05-08T05:52:05.023497+0000","last_became_peered":"2024-05-08T05:52:05.023497+0000","last_unstale":"2024-05-08T05:53:07.579795+0000","last_undegraded":"2024-05-08T05:53:07.579795+0000","last_fullsized":"2024-05-08T05:53:07.579795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:06.462940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579850+0000","last_change":"2024-05-08T05:52:05.027648+0000","last_active":"2024-05-08T05:53:07.579850+0000","last_peered":"2024-05-08T05:53:07.579850+0000","last_clean":"2024-05-08T05:53:07.579850+0000","last_became_active":"2024-05-08T05:52:05.027262+0000","last_became_peered":"2024-05-08T05:52:05.027262+0000","last_unstale":"2024-05-08T05:53:07.579850+0000","last_undegraded":"2024-05-08T05:53:07.579850+0000","last_fullsized":"2024-05-08T05:53:07.579850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:36:18.588967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.267116+0000","last_change":"2024-05-08T05:51:01.189324+0000","last_active":"2024-05-08T05:53:05.267116+0000","last_peered":"2024-05-08T05:53:05.267116+0000","last_clean":"2024-05-08T05:53:05.267116+0000","last_became_active":"2024-05-08T05:51:01.188651+0000","last_became_peered":"2024-05-08T05:51:01.188651+0000","last_unstale":"2024-05-08T05:53:05.267116+0000","last_undegraded":"2024-05-08T05:53:05.267116+0000","last_fullsized":"2024-05-08T05:53:05.267116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:45.149213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265294+0000","last_change":"2024-05-08T05:51:04.306811+0000","last_active":"2024-05-08T05:53:05.265294+0000","last_peered":"2024-05-08T05:53:05.265294+0000","last_clean":"2024-05-08T05:53:05.265294+0000","last_became_active":"2024-05-08T05:51:04.306423+0000","last_became_peered":"2024-05-08T05:51:04.306423+0000","last_unstale":"2024-05-08T05:53:05.265294+0000","last_undegraded":"2024-05-08T05:53:05.265294+0000","last_fullsized":"2024-05-08T05:53:05.265294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:08.830682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139525+0000","last_change":"2024-05-08T05:51:00.276543+0000","last_active":"2024-05-08T05:53:07.139525+0000","last_peered":"2024-05-08T05:53:07.139525+0000","last_clean":"2024-05-08T05:53:07.139525+0000","last_became_active":"2024-05-08T05:51:00.276233+0000","last_became_peered":"2024-05-08T05:51:00.276233+0000","last_unstale":"2024-05-08T05:53:07.139525+0000","last_undegraded":"2024-05-08T05:53:07.139525+0000","last_fullsized":"2024-05-08T05:53:07.139525+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:11.229942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.265173+0000","last_change":"2024-05-08T05:51:04.322033+0000","last_active":"2024-05-08T05:53:05.265173+0000","last_peered":"2024-05-08T05:53:05.265173+0000","last_clean":"2024-05-08T05:53:05.265173+0000","last_became_active":"2024-05-08T05:51:04.321555+0000","last_became_peered":"2024-05-08T05:51:04.321555+0000","last_unstale":"2024-05-08T05:53:05.265173+0000","last_undegraded":"2024-05-08T05:53:05.265173+0000","last_fullsized":"2024-05-08T05:53:05.265173+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:21:50.820709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.266462+0000","last_change":"2024-05-08T05:52:05.027723+0000","last_active":"2024-05-08T05:53:05.266462+0000","last_peered":"2024-05-08T05:53:05.266462+0000","last_clean":"2024-05-08T05:53:05.266462+0000","last_became_active":"2024-05-08T05:52:05.026508+0000","last_became_peered":"2024-05-08T05:52:05.026508+0000","last_unstale":"2024-05-08T05:53:05.266462+0000","last_undegraded":"2024-05-08T05:53:05.266462+0000","last_fullsized":"2024-05-08T05:53:05.266462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:14:32.399433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":250,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:55:00.087804+0000","last_change":"2024-05-08T05:51:01.187556+0000","last_active":"2024-05-08T05:55:00.087804+0000","last_peered":"2024-05-08T05:55:00.087804+0000","last_clean":"2024-05-08T05:55:00.087804+0000","last_became_active":"2024-05-08T05:51:01.186584+0000","last_became_peered":"2024-05-08T05:51:01.186584+0000","last_unstale":"2024-05-08T05:55:00.087804+0000","last_undegraded":"2024-05-08T05:55:00.087804+0000","last_fullsized":"2024-05-08T05:55:00.087804+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:29.036633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.274091+0000","last_change":"2024-05-08T05:53:05.272875+0000","last_active":"2024-05-08T05:53:05.274091+0000","last_peered":"2024-05-08T05:53:05.274091+0000","last_clean":"2024-05-08T05:53:05.274091+0000","last_became_active":"2024-05-08T05:53:05.271672+0000","last_became_peered":"2024-05-08T05:53:05.271672+0000","last_unstale":"2024-05-08T05:53:05.274091+0000","last_undegraded":"2024-05-08T05:53:05.274091+0000","last_fullsized":"2024-05-08T05:53:05.274091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:08:28.776982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715396+0000","last_change":"2024-05-08T05:51:00.272501+0000","last_active":"2024-05-08T05:53:05.715396+0000","last_peered":"2024-05-08T05:53:05.715396+0000","last_clean":"2024-05-08T05:53:05.715396+0000","last_became_active":"2024-05-08T05:51:00.272166+0000","last_became_peered":"2024-05-08T05:51:00.272166+0000","last_unstale":"2024-05-08T05:53:05.715396+0000","last_undegraded":"2024-05-08T05:53:05.715396+0000","last_fullsized":"2024-05-08T05:53:05.715396+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:30.980047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":104,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139425+0000","last_change":"2024-05-08T05:49:31.820799+0000","last_active":"2024-05-08T05:53:07.139425+0000","last_peered":"2024-05-08T05:53:07.139425+0000","last_clean":"2024-05-08T05:53:07.139425+0000","last_became_active":"2024-05-08T05:49:31.815568+0000","last_became_peered":"2024-05-08T05:49:31.815568+0000","last_unstale":"2024-05-08T05:53:07.139425+0000","last_undegraded":"2024-05-08T05:53:07.139425+0000","last_fullsized":"2024-05-08T05:53:07.139425+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:48.170242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:48.170242+0000","last_clean_scrub_stamp":"2024-05-08T05:45:48.170242+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:39.527036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820012+0000","last_change":"2024-05-08T05:51:04.301375+0000","last_active":"2024-05-08T05:53:05.820012+0000","last_peered":"2024-05-08T05:53:05.820012+0000","last_clean":"2024-05-08T05:53:05.820012+0000","last_became_active":"2024-05-08T05:51:04.301027+0000","last_became_peered":"2024-05-08T05:51:04.301027+0000","last_unstale":"2024-05-08T05:53:05.820012+0000","last_undegraded":"2024-05-08T05:53:05.820012+0000","last_fullsized":"2024-05-08T05:53:05.820012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:21.393242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579619+0000","last_change":"2024-05-08T05:52:05.037427+0000","last_active":"2024-05-08T05:53:07.579619+0000","last_peered":"2024-05-08T05:53:07.579619+0000","last_clean":"2024-05-08T05:53:07.579619+0000","last_became_active":"2024-05-08T05:52:05.023644+0000","last_became_peered":"2024-05-08T05:52:05.023644+0000","last_unstale":"2024-05-08T05:53:07.579619+0000","last_undegraded":"2024-05-08T05:53:07.579619+0000","last_fullsized":"2024-05-08T05:53:07.579619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:49.193375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269166+0000","last_change":"2024-05-08T05:51:01.187171+0000","last_active":"2024-05-08T05:53:05.269166+0000","last_peered":"2024-05-08T05:53:05.269166+0000","last_clean":"2024-05-08T05:53:05.269166+0000","last_became_active":"2024-05-08T05:51:01.186142+0000","last_became_peered":"2024-05-08T05:51:01.186142+0000","last_unstale":"2024-05-08T05:53:05.269166+0000","last_undegraded":"2024-05-08T05:53:05.269166+0000","last_fullsized":"2024-05-08T05:53:05.269166+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:37:17.021905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820912+0000","last_change":"2024-05-08T05:51:04.311538+0000","last_active":"2024-05-08T05:53:05.820912+0000","last_peered":"2024-05-08T05:53:05.820912+0000","last_clean":"2024-05-08T05:53:05.820912+0000","last_became_active":"2024-05-08T05:51:04.310535+0000","last_became_peered":"2024-05-08T05:51:04.310535+0000","last_unstale":"2024-05-08T05:53:05.820912+0000","last_undegraded":"2024-05-08T05:53:05.820912+0000","last_fullsized":"2024-05-08T05:53:05.820912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:01:04.497986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'13","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:13.203771+0000","last_change":"2024-05-08T05:51:00.277923+0000","last_active":"2024-05-08T05:53:13.203771+0000","last_peered":"2024-05-08T05:53:13.203771+0000","last_clean":"2024-05-08T05:53:13.203771+0000","last_became_active":"2024-05-08T05:51:00.274896+0000","last_became_peered":"2024-05-08T05:51:00.274896+0000","last_unstale":"2024-05-08T05:53:13.203771+0000","last_undegraded":"2024-05-08T05:53:13.203771+0000","last_fullsized":"2024-05-08T05:53:13.203771+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:04:32.543451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.573064+0000","last_change":"2024-05-08T05:51:04.306185+0000","last_active":"2024-05-08T05:53:05.573064+0000","last_peered":"2024-05-08T05:53:05.573064+0000","last_clean":"2024-05-08T05:53:05.573064+0000","last_became_active":"2024-05-08T05:51:04.305229+0000","last_became_peered":"2024-05-08T05:51:04.305229+0000","last_unstale":"2024-05-08T05:53:05.573064+0000","last_undegraded":"2024-05-08T05:53:05.573064+0000","last_fullsized":"2024-05-08T05:53:05.573064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:48.859978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.715588+0000","last_change":"2024-05-08T05:51:01.140178+0000","last_active":"2024-05-08T05:53:05.715588+0000","last_peered":"2024-05-08T05:53:05.715588+0000","last_clean":"2024-05-08T05:53:05.715588+0000","last_became_active":"2024-05-08T05:51:01.139772+0000","last_became_peered":"2024-05-08T05:51:01.139772+0000","last_unstale":"2024-05-08T05:53:05.715588+0000","last_undegraded":"2024-05-08T05:53:05.715588+0000","last_fullsized":"2024-05-08T05:53:05.715588+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:54:40.132683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032654+0000","last_change":"2024-05-08T05:51:04.319015+0000","last_active":"2024-05-08T05:53:07.032654+0000","last_peered":"2024-05-08T05:53:07.032654+0000","last_clean":"2024-05-08T05:53:07.032654+0000","last_became_active":"2024-05-08T05:51:04.318465+0000","last_became_peered":"2024-05-08T05:51:04.318465+0000","last_unstale":"2024-05-08T05:53:07.032654+0000","last_undegraded":"2024-05-08T05:53:07.032654+0000","last_fullsized":"2024-05-08T05:53:07.032654+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:07:19.336656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269322+0000","last_change":"2024-05-08T05:51:01.185619+0000","last_active":"2024-05-08T05:53:05.269322+0000","last_peered":"2024-05-08T05:53:05.269322+0000","last_clean":"2024-05-08T05:53:05.269322+0000","last_became_active":"2024-05-08T05:51:01.185325+0000","last_became_peered":"2024-05-08T05:51:01.185325+0000","last_unstale":"2024-05-08T05:53:05.269322+0000","last_undegraded":"2024-05-08T05:53:05.269322+0000","last_fullsized":"2024-05-08T05:53:05.269322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:41.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268756+0000","last_change":"2024-05-08T05:51:04.295018+0000","last_active":"2024-05-08T05:53:05.268756+0000","last_peered":"2024-05-08T05:53:05.268756+0000","last_clean":"2024-05-08T05:53:05.268756+0000","last_became_active":"2024-05-08T05:51:04.294432+0000","last_became_peered":"2024-05-08T05:51:04.294432+0000","last_unstale":"2024-05-08T05:53:05.268756+0000","last_undegraded":"2024-05-08T05:53:05.268756+0000","last_fullsized":"2024-05-08T05:53:05.268756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:01.925456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571832+0000","last_change":"2024-05-08T05:51:00.280527+0000","last_active":"2024-05-08T05:53:05.571832+0000","last_peered":"2024-05-08T05:53:05.571832+0000","last_clean":"2024-05-08T05:53:05.571832+0000","last_became_active":"2024-05-08T05:51:00.278718+0000","last_became_peered":"2024-05-08T05:51:00.278718+0000","last_unstale":"2024-05-08T05:53:05.571832+0000","last_undegraded":"2024-05-08T05:53:05.571832+0000","last_fullsized":"2024-05-08T05:53:05.571832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:48:27.149713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579777+0000","last_change":"2024-05-08T05:51:04.314543+0000","last_active":"2024-05-08T05:53:07.579777+0000","last_peered":"2024-05-08T05:53:07.579777+0000","last_clean":"2024-05-08T05:53:07.579777+0000","last_became_active":"2024-05-08T05:51:04.314168+0000","last_became_peered":"2024-05-08T05:51:04.314168+0000","last_unstale":"2024-05-08T05:53:07.579777+0000","last_undegraded":"2024-05-08T05:53:07.579777+0000","last_fullsized":"2024-05-08T05:53:07.579777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:10:23.215322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:11.750983+0000","last_change":"2024-05-08T05:51:00.268081+0000","last_active":"2024-05-08T05:53:11.750983+0000","last_peered":"2024-05-08T05:53:11.750983+0000","last_clean":"2024-05-08T05:53:11.750983+0000","last_became_active":"2024-05-08T05:51:00.267470+0000","last_became_peered":"2024-05-08T05:51:00.267470+0000","last_unstale":"2024-05-08T05:53:11.750983+0000","last_undegraded":"2024-05-08T05:53:11.750983+0000","last_fullsized":"2024-05-08T05:53:11.750983+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:45.796019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572395+0000","last_change":"2024-05-08T05:51:04.322010+0000","last_active":"2024-05-08T05:53:05.572395+0000","last_peered":"2024-05-08T05:53:05.572395+0000","last_clean":"2024-05-08T05:53:05.572395+0000","last_became_active":"2024-05-08T05:51:04.320867+0000","last_became_peered":"2024-05-08T05:51:04.320867+0000","last_unstale":"2024-05-08T05:53:05.572395+0000","last_undegraded":"2024-05-08T05:53:05.572395+0000","last_fullsized":"2024-05-08T05:53:05.572395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:49.853518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.268990+0000","last_change":"2024-05-08T05:52:05.045290+0000","last_active":"2024-05-08T05:53:05.268990+0000","last_peered":"2024-05-08T05:53:05.268990+0000","last_clean":"2024-05-08T05:53:05.268990+0000","last_became_active":"2024-05-08T05:52:05.029284+0000","last_became_peered":"2024-05-08T05:52:05.029284+0000","last_unstale":"2024-05-08T05:53:05.268990+0000","last_undegraded":"2024-05-08T05:53:05.268990+0000","last_fullsized":"2024-05-08T05:53:05.268990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:20.491862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139647+0000","last_change":"2024-05-08T05:51:04.302813+0000","last_active":"2024-05-08T05:53:07.139647+0000","last_peered":"2024-05-08T05:53:07.139647+0000","last_clean":"2024-05-08T05:53:07.139647+0000","last_became_active":"2024-05-08T05:51:04.302424+0000","last_became_peered":"2024-05-08T05:51:04.302424+0000","last_unstale":"2024-05-08T05:53:07.139647+0000","last_undegraded":"2024-05-08T05:53:07.139647+0000","last_fullsized":"2024-05-08T05:53:07.139647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:52:17.752351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505505+0000","last_change":"2024-05-08T05:51:01.187513+0000","last_active":"2024-05-08T05:53:05.505505+0000","last_peered":"2024-05-08T05:53:05.505505+0000","last_clean":"2024-05-08T05:53:05.505505+0000","last_became_active":"2024-05-08T05:51:01.186780+0000","last_became_peered":"2024-05-08T05:51:01.186780+0000","last_unstale":"2024-05-08T05:53:05.505505+0000","last_undegraded":"2024-05-08T05:53:05.505505+0000","last_fullsized":"2024-05-08T05:53:05.505505+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:42:26.206351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139833+0000","last_change":"2024-05-08T05:51:04.295338+0000","last_active":"2024-05-08T05:53:07.139833+0000","last_peered":"2024-05-08T05:53:07.139833+0000","last_clean":"2024-05-08T05:53:07.139833+0000","last_became_active":"2024-05-08T05:51:04.294990+0000","last_became_peered":"2024-05-08T05:51:04.294990+0000","last_unstale":"2024-05-08T05:53:07.139833+0000","last_undegraded":"2024-05-08T05:53:07.139833+0000","last_fullsized":"2024-05-08T05:53:07.139833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:29.411662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270051+0000","last_change":"2024-05-08T05:51:00.273052+0000","last_active":"2024-05-08T05:53:05.270051+0000","last_peered":"2024-05-08T05:53:05.270051+0000","last_clean":"2024-05-08T05:53:05.270051+0000","last_became_active":"2024-05-08T05:51:00.272718+0000","last_became_peered":"2024-05-08T05:51:00.272718+0000","last_unstale":"2024-05-08T05:53:05.270051+0000","last_undegraded":"2024-05-08T05:53:05.270051+0000","last_fullsized":"2024-05-08T05:53:05.270051+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:44.494378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139455+0000","last_change":"2024-05-08T05:51:04.306209+0000","last_active":"2024-05-08T05:53:07.139455+0000","last_peered":"2024-05-08T05:53:07.139455+0000","last_clean":"2024-05-08T05:53:07.139455+0000","last_became_active":"2024-05-08T05:51:04.305752+0000","last_became_peered":"2024-05-08T05:51:04.305752+0000","last_unstale":"2024-05-08T05:53:07.139455+0000","last_undegraded":"2024-05-08T05:53:07.139455+0000","last_fullsized":"2024-05-08T05:53:07.139455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:24.586111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820689+0000","last_change":"2024-05-08T05:51:00.278426+0000","last_active":"2024-05-08T05:53:05.820689+0000","last_peered":"2024-05-08T05:53:05.820689+0000","last_clean":"2024-05-08T05:53:05.820689+0000","last_became_active":"2024-05-08T05:51:00.276764+0000","last_became_peered":"2024-05-08T05:51:00.276764+0000","last_unstale":"2024-05-08T05:53:05.820689+0000","last_undegraded":"2024-05-08T05:53:05.820689+0000","last_fullsized":"2024-05-08T05:53:05.820689+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:59.214286+0000","last_clean_scrub_stamp":"2024-05-08T05:50:59.214286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:34.158628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.139683+0000","last_change":"2024-05-08T05:51:04.294511+0000","last_active":"2024-05-08T05:53:07.139683+0000","last_peered":"2024-05-08T05:53:07.139683+0000","last_clean":"2024-05-08T05:53:07.139683+0000","last_became_active":"2024-05-08T05:51:04.294145+0000","last_became_peered":"2024-05-08T05:51:04.294145+0000","last_unstale":"2024-05-08T05:53:07.139683+0000","last_undegraded":"2024-05-08T05:53:07.139683+0000","last_fullsized":"2024-05-08T05:53:07.139683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:10.034060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579833+0000","last_change":"2024-05-08T05:51:04.298905+0000","last_active":"2024-05-08T05:53:07.579833+0000","last_peered":"2024-05-08T05:53:07.579833+0000","last_clean":"2024-05-08T05:53:07.579833+0000","last_became_active":"2024-05-08T05:51:04.298586+0000","last_became_peered":"2024-05-08T05:51:04.298586+0000","last_unstale":"2024-05-08T05:53:07.579833+0000","last_undegraded":"2024-05-08T05:53:07.579833+0000","last_fullsized":"2024-05-08T05:53:07.579833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:43:24.899015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505323+0000","last_change":"2024-05-08T05:51:04.300503+0000","last_active":"2024-05-08T05:53:05.505323+0000","last_peered":"2024-05-08T05:53:05.505323+0000","last_clean":"2024-05-08T05:53:05.505323+0000","last_became_active":"2024-05-08T05:51:04.299771+0000","last_became_peered":"2024-05-08T05:51:04.299771+0000","last_unstale":"2024-05-08T05:53:05.505323+0000","last_undegraded":"2024-05-08T05:53:05.505323+0000","last_fullsized":"2024-05-08T05:53:05.505323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:19.790501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505267+0000","last_change":"2024-05-08T05:52:05.027724+0000","last_active":"2024-05-08T05:53:05.505267+0000","last_peered":"2024-05-08T05:53:05.505267+0000","last_clean":"2024-05-08T05:53:05.505267+0000","last_became_active":"2024-05-08T05:52:05.027338+0000","last_became_peered":"2024-05-08T05:52:05.027338+0000","last_unstale":"2024-05-08T05:53:05.505267+0000","last_undegraded":"2024-05-08T05:53:05.505267+0000","last_fullsized":"2024-05-08T05:53:05.505267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:03.324348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.032152+0000","last_change":"2024-05-08T05:52:05.574645+0000","last_active":"2024-05-08T05:53:07.032152+0000","last_peered":"2024-05-08T05:53:07.032152+0000","last_clean":"2024-05-08T05:53:07.032152+0000","last_became_active":"2024-05-08T05:52:05.573751+0000","last_became_peered":"2024-05-08T05:52:05.573751+0000","last_unstale":"2024-05-08T05:53:07.032152+0000","last_undegraded":"2024-05-08T05:53:07.032152+0000","last_fullsized":"2024-05-08T05:53:07.032152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:57:40.656001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820990+0000","last_change":"2024-05-08T05:51:04.315769+0000","last_active":"2024-05-08T05:53:05.820990+0000","last_peered":"2024-05-08T05:53:05.820990+0000","last_clean":"2024-05-08T05:53:05.820990+0000","last_became_active":"2024-05-08T05:51:04.315322+0000","last_became_peered":"2024-05-08T05:51:04.315322+0000","last_unstale":"2024-05-08T05:53:05.820990+0000","last_undegraded":"2024-05-08T05:53:05.820990+0000","last_fullsized":"2024-05-08T05:53:05.820990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:52.096556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579878+0000","last_change":"2024-05-08T05:51:04.298128+0000","last_active":"2024-05-08T05:53:07.579878+0000","last_peered":"2024-05-08T05:53:07.579878+0000","last_clean":"2024-05-08T05:53:07.579878+0000","last_became_active":"2024-05-08T05:51:04.297743+0000","last_became_peered":"2024-05-08T05:51:04.297743+0000","last_unstale":"2024-05-08T05:53:07.579878+0000","last_undegraded":"2024-05-08T05:53:07.579878+0000","last_fullsized":"2024-05-08T05:53:07.579878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:18.334026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.505656+0000","last_change":"2024-05-08T05:51:04.301056+0000","last_active":"2024-05-08T05:53:05.505656+0000","last_peered":"2024-05-08T05:53:05.505656+0000","last_clean":"2024-05-08T05:53:05.505656+0000","last_became_active":"2024-05-08T05:51:04.300733+0000","last_became_peered":"2024-05-08T05:51:04.300733+0000","last_unstale":"2024-05-08T05:53:05.505656+0000","last_undegraded":"2024-05-08T05:53:05.505656+0000","last_fullsized":"2024-05-08T05:53:05.505656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:21.616867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.572880+0000","last_change":"2024-05-08T05:51:04.299079+0000","last_active":"2024-05-08T05:53:05.572880+0000","last_peered":"2024-05-08T05:53:05.572880+0000","last_clean":"2024-05-08T05:53:05.572880+0000","last_became_active":"2024-05-08T05:51:04.298628+0000","last_became_peered":"2024-05-08T05:51:04.298628+0000","last_unstale":"2024-05-08T05:53:05.572880+0000","last_undegraded":"2024-05-08T05:53:05.572880+0000","last_fullsized":"2024-05-08T05:53:05.572880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:45.031695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.270220+0000","last_change":"2024-05-08T05:51:04.296140+0000","last_active":"2024-05-08T05:53:05.270220+0000","last_peered":"2024-05-08T05:53:05.270220+0000","last_clean":"2024-05-08T05:53:05.270220+0000","last_became_active":"2024-05-08T05:51:04.294940+0000","last_became_peered":"2024-05-08T05:51:04.294940+0000","last_unstale":"2024-05-08T05:53:05.270220+0000","last_undegraded":"2024-05-08T05:53:05.270220+0000","last_fullsized":"2024-05-08T05:53:05.270220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:37.192639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.571649+0000","last_change":"2024-05-08T05:51:04.305803+0000","last_active":"2024-05-08T05:53:05.571649+0000","last_peered":"2024-05-08T05:53:05.571649+0000","last_clean":"2024-05-08T05:53:05.571649+0000","last_became_active":"2024-05-08T05:51:04.305482+0000","last_became_peered":"2024-05-08T05:51:04.305482+0000","last_unstale":"2024-05-08T05:53:05.571649+0000","last_undegraded":"2024-05-08T05:53:05.571649+0000","last_fullsized":"2024-05-08T05:53:05.571649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:02.050153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.269746+0000","last_change":"2024-05-08T05:51:04.293787+0000","last_active":"2024-05-08T05:53:05.269746+0000","last_peered":"2024-05-08T05:53:05.269746+0000","last_clean":"2024-05-08T05:53:05.269746+0000","last_became_active":"2024-05-08T05:51:04.293455+0000","last_became_peered":"2024-05-08T05:51:04.293455+0000","last_unstale":"2024-05-08T05:53:05.269746+0000","last_undegraded":"2024-05-08T05:53:05.269746+0000","last_fullsized":"2024-05-08T05:53:05.269746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:09:53.211411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:05.820895+0000","last_change":"2024-05-08T05:51:04.321442+0000","last_active":"2024-05-08T05:53:05.820895+0000","last_peered":"2024-05-08T05:53:05.820895+0000","last_clean":"2024-05-08T05:53:05.820895+0000","last_became_active":"2024-05-08T05:51:04.320982+0000","last_became_peered":"2024-05-08T05:51:04.320982+0000","last_unstale":"2024-05-08T05:53:05.820895+0000","last_undegraded":"2024-05-08T05:53:05.820895+0000","last_fullsized":"2024-05-08T05:53:05.820895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:45.089609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T05:53:07.579671+0000","last_change":"2024-05-08T05:51:04.294866+0000","last_active":"2024-05-08T05:53:07.579671+0000","last_peered":"2024-05-08T05:53:07.579671+0000","last_clean":"2024-05-08T05:53:07.579671+0000","last_became_active":"2024-05-08T05:51:04.294504+0000","last_became_peered":"2024-05-08T05:51:04.294504+0000","last_unstale":"2024-05-08T05:53:07.579671+0000","last_undegraded":"2024-05-08T05:53:07.579671+0000","last_fullsized":"2024-05-08T05:53:07.579671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.268495+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.268495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:33.139864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71075,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1330,"num_read_kb":739,"num_write":181,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":134,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":213225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645377,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":452538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841605,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1111152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005131,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":452752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201360,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":452650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430294,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1540096,"data_stored":1045053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593819,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":452141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":518669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":453412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":520105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182508,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":452452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986279,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":453110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790052,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36852,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708300,"statfs":{"total":95995035648,"available":95957299200,"internally_reserved":0,"allocated":1552384,"data_stored":1044205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:55:05.938 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:55:06.049 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:05 smithi112 ceph-mon[30003]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:55:06.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:05 smithi162 ceph-mon[34800]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:55:06.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:05 smithi150 ceph-mon[34315]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:55:06.421 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:55:06.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:06 smithi112 ceph-mon[30003]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:55:06.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:06 smithi112 ceph-mon[30003]: pgmap v435: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T05:55:06.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:06 smithi112 ceph-mon[30003]: from='client.25099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:55:07.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:06 smithi162 ceph-mon[34800]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:55:07.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:06 smithi162 ceph-mon[34800]: pgmap v435: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T05:55:07.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:06 smithi162 ceph-mon[34800]: from='client.25099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:55:07.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:06 smithi150 ceph-mon[34315]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:55:07.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:06 smithi150 ceph-mon[34315]: pgmap v435: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T05:55:07.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:06 smithi150 ceph-mon[34315]: from='client.25099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:55:08.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.634+0000 7fec05600700 1 -- 172.21.15.112:0/2215331340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec000756d0 msgr2=0x7fec00075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:08.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.634+0000 7fec05600700 1 --2- 172.21.15.112:0/2215331340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec000756d0 0x7fec00075ab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7febf000b0a0 tx=0x7febf000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.635+0000 7fec05600700 1 -- 172.21.15.112:0/2215331340 shutdown_connections 2024-05-08T05:55:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.635+0000 7fec05600700 1 --2- 172.21.15.112:0/2215331340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec00114670 0x7fec00116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.635+0000 7fec05600700 1 --2- 172.21.15.112:0/2215331340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec000756d0 0x7fec00075ab0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.635+0000 7fec05600700 1 --2- 172.21.15.112:0/2215331340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec00075ff0 0x7fec00114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.635+0000 7fec05600700 1 -- 172.21.15.112:0/2215331340 >> 172.21.15.112:0/2215331340 conn(0x7fec0007a600 msgr2=0x7fec0007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.635+0000 7fec05600700 1 -- 172.21.15.112:0/2215331340 shutdown_connections 2024-05-08T05:55:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.635+0000 7fec05600700 1 -- 172.21.15.112:0/2215331340 wait complete. 2024-05-08T05:55:08.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.636+0000 7fec05600700 1 Processor -- start 2024-05-08T05:55:08.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.636+0000 7fec05600700 1 -- start start 2024-05-08T05:55:08.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.636+0000 7fec05600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec000756d0 0x7fec001a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.636+0000 7fec05600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec00075ff0 0x7fec001a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.636+0000 7fec05600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec00114670 0x7fec001a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.636+0000 7fec05600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec00119c50 con 0x7fec00075ff0 2024-05-08T05:55:08.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.636+0000 7fec05600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fec00119ad0 con 0x7fec000756d0 2024-05-08T05:55:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.636+0000 7fec05600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fec00119dd0 con 0x7fec00114670 2024-05-08T05:55:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.637+0000 7febfea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec000756d0 0x7fec001a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.637+0000 7febfea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec000756d0 0x7fec001a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:40792/0 (socket says 172.21.15.112:40792) 2024-05-08T05:55:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.637+0000 7febfea00700 1 -- 172.21.15.112:0/1423005846 learned_addr learned my addr 172.21.15.112:0/1423005846 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.637+0000 7febff400700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec00114670 0x7fec001a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.637+0000 7febf6000700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec00075ff0 0x7fec001a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.637+0000 7febff400700 1 -- 172.21.15.112:0/1423005846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec000756d0 msgr2=0x7fec001a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.637+0000 7febff400700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec000756d0 0x7fec001a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.637+0000 7febff400700 1 -- 172.21.15.112:0/1423005846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec00075ff0 msgr2=0x7fec001a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.637+0000 7febff400700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec00075ff0 0x7fec001a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.637+0000 7febff400700 1 -- 172.21.15.112:0/1423005846 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec001a5170 con 0x7fec00114670 2024-05-08T05:55:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.637+0000 7febfea00700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec000756d0 0x7fec001a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.637+0000 7febf6000700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec00075ff0 0x7fec001a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.638+0000 7febff400700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec00114670 0x7fec001a4a30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7febf800e940 tx=0x7febf800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.638+0000 7febf7e00700 1 -- 172.21.15.112:0/1423005846 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8004220 con 0x7fec00114670 2024-05-08T05:55:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.638+0000 7fec05600700 1 -- 172.21.15.112:0/1423005846 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec001a5460 con 0x7fec00114670 2024-05-08T05:55:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.638+0000 7febf7e00700 1 -- 172.21.15.112:0/1423005846 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf80043c0 con 0x7fec00114670 2024-05-08T05:55:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.638+0000 7febf7e00700 1 -- 172.21.15.112:0/1423005846 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8010700 con 0x7fec00114670 2024-05-08T05:55:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.638+0000 7fec05600700 1 -- 172.21.15.112:0/1423005846 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec001acce0 con 0x7fec00114670 2024-05-08T05:55:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.639+0000 7fec05600700 1 -- 172.21.15.112:0/1423005846 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0006a190 con 0x7fec00114670 2024-05-08T05:55:08.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.641+0000 7febf7e00700 1 -- 172.21.15.112:0/1423005846 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7febf8004750 con 0x7fec00114670 2024-05-08T05:55:08.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.643+0000 7febf7e00700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7febe40b2880 0x7febe40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.643+0000 7febf7e00700 1 -- 172.21.15.112:0/1423005846 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7febf80e2090 con 0x7fec00114670 2024-05-08T05:55:08.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.643+0000 7febfea00700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7febe40b2880 0x7febe40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.644+0000 7febfea00700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7febe40b2880 0x7febe40b4d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7febf0009e60 tx=0x7febf000b740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:08.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.645+0000 7febf7e00700 1 -- 172.21.15.112:0/1423005846 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7febf80a9db0 con 0x7fec00114670 2024-05-08T05:55:08.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:08 smithi112 ceph-mon[30003]: pgmap v436: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T05:55:08.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.926+0000 7fec05600700 1 -- 172.21.15.112:0/1423005846 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec001acee0 con 0x7fec00114670 2024-05-08T05:55:08.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.928+0000 7febf7e00700 1 -- 172.21.15.112:0/1423005846 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11891 (secure 0 0 0) 0x7febf80a96b0 con 0x7fec00114670 2024-05-08T05:55:08.928 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:55:08.929 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T05:54:58.248303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24518,24511]},"id":1}]} 2024-05-08T05:55:08.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.930+0000 7fec05600700 1 -- 172.21.15.112:0/1423005846 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7febe40b2880 msgr2=0x7febe40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:08.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.930+0000 7fec05600700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7febe40b2880 0x7febe40b4d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7febf0009e60 tx=0x7febf000b740 comp rx=0 tx=0).stop 2024-05-08T05:55:08.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.930+0000 7fec05600700 1 -- 172.21.15.112:0/1423005846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec00114670 msgr2=0x7fec001a4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:08.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.930+0000 7fec05600700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec00114670 0x7fec001a4a30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7febf800e940 tx=0x7febf800ec50 comp rx=0 tx=0).stop 2024-05-08T05:55:08.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.931+0000 7fec05600700 1 -- 172.21.15.112:0/1423005846 shutdown_connections 2024-05-08T05:55:08.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.931+0000 7fec05600700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7febe40b2880 0x7febe40b4d40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.931+0000 7fec05600700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec00114670 0x7fec001a4a30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.931+0000 7fec05600700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec000756d0 0x7fec001a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.931+0000 7fec05600700 1 --2- 172.21.15.112:0/1423005846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec00075ff0 0x7fec001a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.931+0000 7fec05600700 1 -- 172.21.15.112:0/1423005846 >> 172.21.15.112:0/1423005846 conn(0x7fec0007a600 msgr2=0x7fec00110110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:08.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.931+0000 7fec05600700 1 -- 172.21.15.112:0/1423005846 shutdown_connections 2024-05-08T05:55:08.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:08.931+0000 7fec05600700 1 -- 172.21.15.112:0/1423005846 wait complete. 2024-05-08T05:55:08.933 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-05-08T05:55:09.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:08 smithi162 ceph-mon[34800]: pgmap v436: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T05:55:09.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:08 smithi150 ceph-mon[34315]: pgmap v436: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T05:55:09.831 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:55:10.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:09 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1423005846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:10.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:09 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1423005846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:10.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:09 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1423005846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:10.283 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:55:11.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:10 smithi112 ceph-mon[30003]: pgmap v437: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T05:55:11.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:10 smithi162 ceph-mon[34800]: pgmap v437: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T05:55:11.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:10 smithi150 ceph-mon[34315]: pgmap v437: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T05:55:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.420+0000 7f0124a00700 1 -- 172.21.15.112:0/3821384920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01200779d0 msgr2=0x7f0120077db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.420+0000 7f0124a00700 1 --2- 172.21.15.112:0/3821384920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01200779d0 0x7f0120077db0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0110009990 tx=0x7f0110009ca0 comp rx=0 tx=0).stop 2024-05-08T05:55:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.420+0000 7f0124a00700 1 -- 172.21.15.112:0/3821384920 shutdown_connections 2024-05-08T05:55:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.420+0000 7f0124a00700 1 --2- 172.21.15.112:0/3821384920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0120114330 0x7f0120116720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.420+0000 7f0124a00700 1 --2- 172.21.15.112:0/3821384920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01200779d0 0x7f0120077db0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.420+0000 7f0124a00700 1 --2- 172.21.15.112:0/3821384920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01200782f0 0x7f0120113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:12.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.420+0000 7f0124a00700 1 -- 172.21.15.112:0/3821384920 >> 172.21.15.112:0/3821384920 conn(0x7f0120101180 msgr2=0x7f01201035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:12.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.421+0000 7f0124a00700 1 -- 172.21.15.112:0/3821384920 shutdown_connections 2024-05-08T05:55:12.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.421+0000 7f0124a00700 1 -- 172.21.15.112:0/3821384920 wait complete. 2024-05-08T05:55:12.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.421+0000 7f0124a00700 1 Processor -- start 2024-05-08T05:55:12.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.422+0000 7f0124a00700 1 -- start start 2024-05-08T05:55:12.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.422+0000 7f0124a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01200779d0 0x7f01201a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:12.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.422+0000 7f0124a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01200782f0 0x7f01201a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:12.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.422+0000 7f0124a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0120114330 0x7f01201a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:12.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.422+0000 7f0124a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0120119d00 con 0x7f01200779d0 2024-05-08T05:55:12.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.422+0000 7f0124a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0120119b80 con 0x7f0120114330 2024-05-08T05:55:12.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.422+0000 7f0124a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0120119e80 con 0x7f01200782f0 2024-05-08T05:55:12.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.422+0000 7f011de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01200779d0 0x7f01201a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:12.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.422+0000 7f011d400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01200782f0 0x7f01201a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:12.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.422+0000 7f011d400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01200782f0 0x7f01201a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:57314/0 (socket says 172.21.15.112:57314) 2024-05-08T05:55:12.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.422+0000 7f011d400700 1 -- 172.21.15.112:0/1375629508 learned_addr learned my addr 172.21.15.112:0/1375629508 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:12.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.422+0000 7f011e800700 1 --2- 172.21.15.112:0/1375629508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0120114330 0x7f01201a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:12.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.423+0000 7f011d400700 1 -- 172.21.15.112:0/1375629508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0120114330 msgr2=0x7f01201a4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:12.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.423+0000 7f011d400700 1 --2- 172.21.15.112:0/1375629508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0120114330 0x7f01201a4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:12.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.423+0000 7f011d400700 1 -- 172.21.15.112:0/1375629508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01200779d0 msgr2=0x7f01201a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:12.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.423+0000 7f011d400700 1 --2- 172.21.15.112:0/1375629508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01200779d0 0x7f01201a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:12.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.423+0000 7f011d400700 1 -- 172.21.15.112:0/1375629508 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01201a51e0 con 0x7f01200782f0 2024-05-08T05:55:12.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.423+0000 7f011d400700 1 --2- 172.21.15.112:0/1375629508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01200782f0 0x7f01201a0710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f010c00e9a0 tx=0x7f010c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:12.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.423+0000 7f0116a00700 1 -- 172.21.15.112:0/1375629508 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c00ca10 con 0x7f01200782f0 2024-05-08T05:55:12.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.423+0000 7f0116a00700 1 -- 172.21.15.112:0/1375629508 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f010c00cbb0 con 0x7f01200782f0 2024-05-08T05:55:12.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.424+0000 7f0124a00700 1 -- 172.21.15.112:0/1375629508 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01201a54d0 con 0x7f01200782f0 2024-05-08T05:55:12.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.424+0000 7f0116a00700 1 -- 172.21.15.112:0/1375629508 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c018680 con 0x7f01200782f0 2024-05-08T05:55:12.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.424+0000 7f0124a00700 1 -- 172.21.15.112:0/1375629508 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01201acd50 con 0x7f01200782f0 2024-05-08T05:55:12.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.425+0000 7f0124a00700 1 -- 172.21.15.112:0/1375629508 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f012006a190 con 0x7f01200782f0 2024-05-08T05:55:12.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.428+0000 7f0116a00700 1 -- 172.21.15.112:0/1375629508 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f010c010430 con 0x7f01200782f0 2024-05-08T05:55:12.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.429+0000 7f0116a00700 1 --2- 172.21.15.112:0/1375629508 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f01040b2900 0x7f01040b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:12.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.429+0000 7f0116a00700 1 -- 172.21.15.112:0/1375629508 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f010c0e26d0 con 0x7f01200782f0 2024-05-08T05:55:12.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.430+0000 7f011de00700 1 --2- 172.21.15.112:0/1375629508 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f01040b2900 0x7f01040b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:12.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.430+0000 7f011de00700 1 --2- 172.21.15.112:0/1375629508 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f01040b2900 0x7f01040b4dc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0110006010 tx=0x7f011000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:12.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.431+0000 7f0116a00700 1 -- 172.21.15.112:0/1375629508 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f010c0aa360 con 0x7f01200782f0 2024-05-08T05:55:12.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.670+0000 7f0124a00700 1 -- 172.21.15.112:0/1375629508 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f012002e510 con 0x7f01200782f0 2024-05-08T05:55:12.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.672+0000 7f0116a00700 1 -- 172.21.15.112:0/1375629508 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22829 (secure 0 0 0) 0x7f010c0b23e0 con 0x7f01200782f0 2024-05-08T05:55:12.673 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:55:12.673 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":84,"fsid":"7a07ff60-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:48.498871+0000","modified":"2024-05-08T05:53:05.251932+0000","last_up_change":"2024-05-08T05:49:54.039380+0000","last_in_change":"2024-05-08T05:49:34.443850+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:47.485105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:50:58.918909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.175022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b378542-1bbe-4614-8805-8a6b875ada55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6803","nonce":3419091820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6805","nonce":3419091820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6809","nonce":3419091820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6807","nonce":3419091820}]},"public_addr":"172.21.15.112:6803/3419091820","cluster_addr":"172.21.15.112:6805/3419091820","heartbeat_back_addr":"172.21.15.112:6809/3419091820","heartbeat_front_addr":"172.21.15.112:6807/3419091820","state":["exists","up"]},{"osd":1,"uuid":"826ef036-2db7-470e-9956-d9a58b9181e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6801","nonce":2728419797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6803","nonce":2728419797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6807","nonce":2728419797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6805","nonce":2728419797}]},"public_addr":"172.21.15.150:6801/2728419797","cluster_addr":"172.21.15.150:6803/2728419797","heartbeat_back_addr":"172.21.15.150:6807/2728419797","heartbeat_front_addr":"172.21.15.150:6805/2728419797","state":["exists","up"]},{"osd":2,"uuid":"32e710e7-2a87-4863-bc7a-906d31aa4ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6801","nonce":1556203215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6803","nonce":1556203215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6807","nonce":1556203215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6805","nonce":1556203215}]},"public_addr":"172.21.15.162:6801/1556203215","cluster_addr":"172.21.15.162:6803/1556203215","heartbeat_back_addr":"172.21.15.162:6807/1556203215","heartbeat_front_addr":"172.21.15.162:6805/1556203215","state":["exists","up"]},{"osd":3,"uuid":"ba1d4a5f-6a95-4b9e-8a48-8524a899b004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6811","nonce":3325718755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6813","nonce":3325718755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6817","nonce":3325718755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6815","nonce":3325718755}]},"public_addr":"172.21.15.112:6811/3325718755","cluster_addr":"172.21.15.112:6813/3325718755","heartbeat_back_addr":"172.21.15.112:6817/3325718755","heartbeat_front_addr":"172.21.15.112:6815/3325718755","state":["exists","up"]},{"osd":4,"uuid":"cb4f1a42-8de8-4b50-b3bc-602de9f4803a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6809","nonce":1548914683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6811","nonce":1548914683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6815","nonce":1548914683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6813","nonce":1548914683}]},"public_addr":"172.21.15.150:6809/1548914683","cluster_addr":"172.21.15.150:6811/1548914683","heartbeat_back_addr":"172.21.15.150:6815/1548914683","heartbeat_front_addr":"172.21.15.150:6813/1548914683","state":["exists","up"]},{"osd":5,"uuid":"a26c03b8-050c-4a2c-8f23-5801fffb7c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6809","nonce":409836750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6811","nonce":409836750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6815","nonce":409836750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6813","nonce":409836750}]},"public_addr":"172.21.15.162:6809/409836750","cluster_addr":"172.21.15.162:6811/409836750","heartbeat_back_addr":"172.21.15.162:6815/409836750","heartbeat_front_addr":"172.21.15.162:6813/409836750","state":["exists","up"]},{"osd":6,"uuid":"8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6819","nonce":1098984186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6821","nonce":1098984186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6825","nonce":1098984186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6823","nonce":1098984186}]},"public_addr":"172.21.15.112:6819/1098984186","cluster_addr":"172.21.15.112:6821/1098984186","heartbeat_back_addr":"172.21.15.112:6825/1098984186","heartbeat_front_addr":"172.21.15.112:6823/1098984186","state":["exists","up"]},{"osd":7,"uuid":"af3629b4-9bed-4669-97a6-48deba503a96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6817","nonce":4009836564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6819","nonce":4009836564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6823","nonce":4009836564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6821","nonce":4009836564}]},"public_addr":"172.21.15.150:6817/4009836564","cluster_addr":"172.21.15.150:6819/4009836564","heartbeat_back_addr":"172.21.15.150:6823/4009836564","heartbeat_front_addr":"172.21.15.150:6821/4009836564","state":["exists","up"]},{"osd":8,"uuid":"9333e87d-dd78-4467-bbde-c2b44f4ea07f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6817","nonce":852791606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6819","nonce":852791606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6823","nonce":852791606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6821","nonce":852791606}]},"public_addr":"172.21.15.162:6817/852791606","cluster_addr":"172.21.15.162:6819/852791606","heartbeat_back_addr":"172.21.15.162:6823/852791606","heartbeat_front_addr":"172.21.15.162:6821/852791606","state":["exists","up"]},{"osd":9,"uuid":"09723581-1995-42e0-8db7-776ea98a918c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6827","nonce":2186764674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6829","nonce":2186764674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6833","nonce":2186764674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6831","nonce":2186764674}]},"public_addr":"172.21.15.112:6827/2186764674","cluster_addr":"172.21.15.112:6829/2186764674","heartbeat_back_addr":"172.21.15.112:6833/2186764674","heartbeat_front_addr":"172.21.15.112:6831/2186764674","state":["exists","up"]},{"osd":10,"uuid":"34e08e3e-1231-4899-a5b3-76d449105bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6825","nonce":2838995474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6827","nonce":2838995474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6831","nonce":2838995474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6829","nonce":2838995474}]},"public_addr":"172.21.15.150:6825/2838995474","cluster_addr":"172.21.15.150:6827/2838995474","heartbeat_back_addr":"172.21.15.150:6831/2838995474","heartbeat_front_addr":"172.21.15.150:6829/2838995474","state":["exists","up"]},{"osd":11,"uuid":"6fc34ec0-78ca-403c-acb7-b29d83ff64e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6825","nonce":3702614331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6827","nonce":3702614331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6831","nonce":3702614331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6829","nonce":3702614331}]},"public_addr":"172.21.15.162:6825/3702614331","cluster_addr":"172.21.15.162:6827/3702614331","heartbeat_back_addr":"172.21.15.162:6831/3702614331","heartbeat_front_addr":"172.21.15.162:6829/3702614331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:54.518702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:19.939543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:44.425586+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:16.126471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:41.696071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:05.517405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:38.549318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:03.799649+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:28.007282+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:01.395459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:27.963637+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:52.181212+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3667148497":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6800/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3425327158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1404611205":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3765334043":"2024-05-09T05:42:29.819307+0000","172.21.15.112:6800/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/2814303790":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1091751583":"2024-05-09T05:42:29.819307+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:55:12.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.675+0000 7f0124a00700 1 -- 172.21.15.112:0/1375629508 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f01040b2900 msgr2=0x7f01040b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:12.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.675+0000 7f0124a00700 1 --2- 172.21.15.112:0/1375629508 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f01040b2900 0x7f01040b4dc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0110006010 tx=0x7f011000b260 comp rx=0 tx=0).stop 2024-05-08T05:55:12.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.675+0000 7f0124a00700 1 -- 172.21.15.112:0/1375629508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01200782f0 msgr2=0x7f01201a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:12.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.675+0000 7f0124a00700 1 --2- 172.21.15.112:0/1375629508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01200782f0 0x7f01201a0710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f010c00e9a0 tx=0x7f010c00ecb0 comp rx=0 tx=0).stop 2024-05-08T05:55:12.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.675+0000 7f0124a00700 1 -- 172.21.15.112:0/1375629508 shutdown_connections 2024-05-08T05:55:12.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.675+0000 7f0124a00700 1 --2- 172.21.15.112:0/1375629508 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f01040b2900 0x7f01040b4dc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:12.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.675+0000 7f0124a00700 1 --2- 172.21.15.112:0/1375629508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0120114330 0x7f01201a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:12.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.675+0000 7f0124a00700 1 --2- 172.21.15.112:0/1375629508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01200779d0 0x7f01201a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:12.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.675+0000 7f0124a00700 1 --2- 172.21.15.112:0/1375629508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01200782f0 0x7f01201a0710 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:12.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.676+0000 7f0124a00700 1 -- 172.21.15.112:0/1375629508 >> 172.21.15.112:0/1375629508 conn(0x7f0120101180 msgr2=0x7f0120103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:12.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.676+0000 7f0124a00700 1 -- 172.21.15.112:0/1375629508 shutdown_connections 2024-05-08T05:55:12.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:12.676+0000 7f0124a00700 1 -- 172.21.15.112:0/1375629508 wait complete. 2024-05-08T05:55:12.972 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:12 smithi112 ceph-mon[30003]: pgmap v438: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T05:55:12.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:12 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1375629508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:55:13.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:12 smithi162 ceph-mon[34800]: pgmap v438: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T05:55:13.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:12 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1375629508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:55:13.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:12 smithi150 ceph-mon[34315]: pgmap v438: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T05:55:13.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:12 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1375629508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:55:13.550 DEBUG:tasks.fwd_scrub:Yielding 2024-05-08T05:55:13.551 INFO:teuthology.run_tasks:Running task workunit... 2024-05-08T05:55:13.563 INFO:tasks.workunit:Pulling workunits from ref bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:55:13.564 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-08T05:55:13.564 INFO:tasks.workunit:timeout=3h 2024-05-08T05:55:13.564 INFO:tasks.workunit:cleanup=True 2024-05-08T05:55:13.564 DEBUG:teuthology.orchestra.run.smithi112:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-08T05:55:13.564 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-08T05:55:13.565 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T05:55:13.565 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T05:55:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-08T05:55:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-08T05:55:13.617 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-05-08T05:55:13.617 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:55:13.617 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-08T05:55:13.617 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-05-08 05:51:46.688291027 +0000 2024-05-08T05:55:13.617 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-05-08 05:51:46.688291027 +0000 2024-05-08T05:55:13.617 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-05-08 05:54:57.156660889 +0000 2024-05-08T05:55:13.617 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-05-08 05:51:46.688291027 +0000 2024-05-08T05:55:13.618 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-08T05:55:13.618 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-08T05:55:13.743 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:55:13.795 INFO:tasks.workunit.client.0.smithi112.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-08T05:55:14.006 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:55:14.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:14 smithi112 ceph-mon[30003]: pgmap v439: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T05:55:15.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:14 smithi162 ceph-mon[34800]: pgmap v439: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T05:55:15.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:14 smithi150 ceph-mon[34315]: pgmap v439: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T05:55:16.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.384+0000 7f9ee9400700 1 -- 172.21.15.112:0/2679912449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee4106080 msgr2=0x7f9ee4112590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:16.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.384+0000 7f9ee9400700 1 --2- 172.21.15.112:0/2679912449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee4106080 0x7f9ee4112590 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00b0a0 tx=0x7f9edc00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.384+0000 7f9ee9400700 1 -- 172.21.15.112:0/2679912449 shutdown_connections 2024-05-08T05:55:16.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.384+0000 7f9ee9400700 1 --2- 172.21.15.112:0/2679912449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee4106080 0x7f9ee4112590 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.384+0000 7f9ee9400700 1 --2- 172.21.15.112:0/2679912449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee410b6c0 0x7f9ee410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.384+0000 7f9ee9400700 1 --2- 172.21.15.112:0/2679912449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ee41056c0 0x7f9ee4105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.384+0000 7f9ee9400700 1 -- 172.21.15.112:0/2679912449 >> 172.21.15.112:0/2679912449 conn(0x7f9ee41011b0 msgr2=0x7f9ee41035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:16.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.384+0000 7f9ee9400700 1 -- 172.21.15.112:0/2679912449 shutdown_connections 2024-05-08T05:55:16.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.384+0000 7f9ee9400700 1 -- 172.21.15.112:0/2679912449 wait complete. 2024-05-08T05:55:16.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.385+0000 7f9ee9400700 1 Processor -- start 2024-05-08T05:55:16.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.385+0000 7f9ee9400700 1 -- start start 2024-05-08T05:55:16.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.386+0000 7f9ee9400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ee41056c0 0x7f9ee419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.386+0000 7f9ee9400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee4106080 0x7f9ee419c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.386+0000 7f9ee9400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee410b6c0 0x7f9ee41a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.386+0000 7f9ee9400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4115910 con 0x7f9ee410b6c0 2024-05-08T05:55:16.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.386+0000 7f9ee9400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4115790 con 0x7f9ee4106080 2024-05-08T05:55:16.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.386+0000 7f9ee9400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4115a90 con 0x7f9ee41056c0 2024-05-08T05:55:16.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.386+0000 7f9ee3200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee410b6c0 0x7f9ee41a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.386+0000 7f9ee3200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee410b6c0 0x7f9ee41a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34544/0 (socket says 172.21.15.112:34544) 2024-05-08T05:55:16.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.386+0000 7f9ee1e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee4106080 0x7f9ee419c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.386+0000 7f9ee3200700 1 -- 172.21.15.112:0/936163370 learned_addr learned my addr 172.21.15.112:0/936163370 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:16.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.386+0000 7f9ee2800700 1 --2- 172.21.15.112:0/936163370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ee41056c0 0x7f9ee419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.387+0000 7f9ee3200700 1 -- 172.21.15.112:0/936163370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ee41056c0 msgr2=0x7f9ee419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:16.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.387+0000 7f9ee3200700 1 --2- 172.21.15.112:0/936163370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ee41056c0 0x7f9ee419bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.387+0000 7f9ee3200700 1 -- 172.21.15.112:0/936163370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee4106080 msgr2=0x7f9ee419c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:16.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.387+0000 7f9ee3200700 1 --2- 172.21.15.112:0/936163370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee4106080 0x7f9ee419c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.387+0000 7f9ee3200700 1 -- 172.21.15.112:0/936163370 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee41a0d70 con 0x7f9ee410b6c0 2024-05-08T05:55:16.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.387+0000 7f9ee1e00700 1 --2- 172.21.15.112:0/936163370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee4106080 0x7f9ee419c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:16.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.387+0000 7f9ee3200700 1 --2- 172.21.15.112:0/936163370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee410b6c0 0x7f9ee41a0630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00b070 tx=0x7f9edc009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:16.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.387+0000 7f9edb400700 1 -- 172.21.15.112:0/936163370 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc021070 con 0x7f9ee410b6c0 2024-05-08T05:55:16.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.387+0000 7f9edb400700 1 -- 172.21.15.112:0/936163370 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9edc004e40 con 0x7f9ee410b6c0 2024-05-08T05:55:16.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.387+0000 7f9ee9400700 1 -- 172.21.15.112:0/936163370 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee41a1000 con 0x7f9ee410b6c0 2024-05-08T05:55:16.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.387+0000 7f9edb400700 1 -- 172.21.15.112:0/936163370 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc004630 con 0x7f9ee410b6c0 2024-05-08T05:55:16.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.387+0000 7f9ee9400700 1 -- 172.21.15.112:0/936163370 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee41a88e0 con 0x7f9ee410b6c0 2024-05-08T05:55:16.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.391+0000 7f9edb400700 1 -- 172.21.15.112:0/936163370 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9edc007980 con 0x7f9ee410b6c0 2024-05-08T05:55:16.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.393+0000 7f9ef3d9db80 1 Processor -- start 2024-05-08T05:55:16.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.393+0000 7f9ef3d9db80 1 -- start start 2024-05-08T05:55:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.394+0000 7f9edb400700 1 --2- 172.21.15.112:0/936163370 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9ec80b2880 0x7f9ec80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.394+0000 7f9ef3d9db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b116c40 0x55f46b117020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.394+0000 7f9ef3d9db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b117560 0x55f46b111350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.394+0000 7f9ef3d9db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b111970 0x55f46b111df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.394+0000 7f9ef3d9db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f46b063aa0 con 0x55f46b117560 2024-05-08T05:55:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.394+0000 7f9ef3d9db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f46b0017e0 con 0x55f46b116c40 2024-05-08T05:55:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.394+0000 7f9ef3d9db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55f46b05fa60 con 0x55f46b111970 2024-05-08T05:55:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.394+0000 7f9edb400700 1 -- 172.21.15.112:0/936163370 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f9edc00f020 con 0x7f9ee410b6c0 2024-05-08T05:55:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.394+0000 7f9ee1e00700 1 --2- 172.21.15.112:0/936163370 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9ec80b2880 0x7f9ec80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.394+0000 7f9ee3200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b111970 0x55f46b111df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.394+0000 7f9ee2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b116c40 0x55f46b117020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.394+0000 7f9ee1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b117560 0x55f46b111350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.394+0000 7f9ee3200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b111970 0x55f46b111df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50764/0 (socket says 172.21.15.112:50764) 2024-05-08T05:55:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.394+0000 7f9ee3200700 1 -- 172.21.15.112:0/3405430119 learned_addr learned my addr 172.21.15.112:0/3405430119 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.395+0000 7f9ee3200700 1 -- 172.21.15.112:0/3405430119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b116c40 msgr2=0x55f46b117020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:16.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.395+0000 7f9ee3200700 1 --2- 172.21.15.112:0/3405430119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b116c40 0x55f46b117020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.395+0000 7f9ee3200700 1 -- 172.21.15.112:0/3405430119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b117560 msgr2=0x55f46b111350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:16.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.395+0000 7f9ee3200700 1 --2- 172.21.15.112:0/3405430119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b117560 0x55f46b111350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.395+0000 7f9ee3200700 1 -- 172.21.15.112:0/3405430119 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f46afb0950 con 0x55f46b111970 2024-05-08T05:55:16.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.395+0000 7f9ee2800700 1 --2- 172.21.15.112:0/3405430119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b116c40 0x55f46b117020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:16.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.395+0000 7f9ee3200700 1 --2- 172.21.15.112:0/3405430119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b111970 0x55f46b111df0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0a6c60 tx=0x7f9edc0aa660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caf1daaaecd21717 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:16.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.395+0000 7f9ee1e00700 1 --2- 172.21.15.112:0/936163370 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9ec80b2880 0x7f9ec80b4d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00097a0 tx=0x7f9ed001e8f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:16.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.395+0000 7f9ecfe00700 1 -- 172.21.15.112:0/3405430119 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0074e0 con 0x55f46b111970 2024-05-08T05:55:16.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.395+0000 7f9ecfe00700 1 -- 172.21.15.112:0/3405430119 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9edc0ab2e0 con 0x55f46b111970 2024-05-08T05:55:16.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.395+0000 7f9ecfe00700 1 -- 172.21.15.112:0/3405430119 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0047d0 con 0x55f46b111970 2024-05-08T05:55:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.396+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3405430119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b111970 msgr2=0x55f46b111df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.396+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/3405430119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b111970 0x55f46b111df0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0a6c60 tx=0x7f9edc0aa660 comp rx=0 tx=0).stop 2024-05-08T05:55:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.396+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3405430119 shutdown_connections 2024-05-08T05:55:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.396+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/3405430119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b111970 0x55f46b111df0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.396+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/3405430119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b116c40 0x55f46b117020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.396+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/3405430119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b117560 0x55f46b111350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.396+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3405430119 >> 172.21.15.112:0/3405430119 conn(0x55f46b089750 msgr2=0x55f46b10bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.396+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3405430119 shutdown_connections 2024-05-08T05:55:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.396+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3405430119 wait complete. 2024-05-08T05:55:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.397+0000 7f9ef3d9db80 1 -- start start 2024-05-08T05:55:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.397+0000 7f9ef3d9db80 1 Processor -- start 2024-05-08T05:55:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.398+0000 7f9ef3d9db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b111970 0x55f46b11d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.398+0000 7f9ef3d9db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b116c40 0x55f46b11a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.398+0000 7f9ef3d9db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b117560 0x55f46b11ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.398+0000 7f9ef3d9db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f46b11dd60 con 0x55f46b111970 2024-05-08T05:55:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.398+0000 7f9ef3d9db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f46afc9ba0 con 0x55f46b117560 2024-05-08T05:55:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.398+0000 7f9ef3d9db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55f46b11dee0 con 0x55f46b116c40 2024-05-08T05:55:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.398+0000 7f9ef3d9db80 10 client.0 _ll_register_callbacks cb 0x55f46b00acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.398+0000 7f9ee1e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b116c40 0x55f46b11a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.398+0000 7f9ee2800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b111970 0x55f46b11d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.398+0000 7f9ee1e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b116c40 0x55f46b11a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50772/0 (socket says 172.21.15.112:50772) 2024-05-08T05:55:16.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.398+0000 7f9ee1e00700 1 -- 172.21.15.112:0/2126928135 learned_addr learned my addr 172.21.15.112:0/2126928135 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:16.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.398+0000 7f9ee3200700 1 --2- 172.21.15.112:0/2126928135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b117560 0x55f46b11ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.399+0000 7f9ee2800700 1 -- 172.21.15.112:0/2126928135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b116c40 msgr2=0x55f46b11a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:16.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.399+0000 7f9ee2800700 1 --2- 172.21.15.112:0/2126928135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b116c40 0x55f46b11a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.399+0000 7f9ee2800700 1 -- 172.21.15.112:0/2126928135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b117560 msgr2=0x55f46b11ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:16.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.399+0000 7f9ee2800700 1 --2- 172.21.15.112:0/2126928135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b117560 0x55f46b11ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.399+0000 7f9ee2800700 1 -- 172.21.15.112:0/2126928135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f46b0294a0 con 0x55f46b111970 2024-05-08T05:55:16.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.399+0000 7f9ee1e00700 1 --2- 172.21.15.112:0/2126928135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b116c40 0x55f46b11a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:16.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.399+0000 7f9ee3200700 1 --2- 172.21.15.112:0/2126928135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b117560 0x55f46b11ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:16.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.399+0000 7f9ee2800700 1 --2- 172.21.15.112:0/2126928135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b111970 0x55f46b11d230 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4012840 tx=0x7f9ed4012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b601af157bcf1d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:16.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.399+0000 7f9eccc00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:55:16.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.399+0000 7f9eccc00700 1 -- 172.21.15.112:0/2126928135 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4004020 con 0x55f46b111970 2024-05-08T05:55:16.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.399+0000 7f9eccc00700 1 -- 172.21.15.112:0/2126928135 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f9ed40041c0 con 0x55f46b111970 2024-05-08T05:55:16.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.399+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/2126928135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9edc0ab2e0 con 0x55f46b111970 2024-05-08T05:55:16.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.400+0000 7f9eccc00700 1 -- 172.21.15.112:0/2126928135 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9ed4022030 con 0x55f46b111970 2024-05-08T05:55:16.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.400+0000 7f9ef3d9db80 10 client.14991 fetch_fsmap learned FSMap version 25 2024-05-08T05:55:16.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.400+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/2126928135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f46b1285f0 con 0x55f46b111970 2024-05-08T05:55:16.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.400+0000 7f9eccc00700 1 -- 172.21.15.112:0/2126928135 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9edc0ab2e0 con 0x55f46b111970 2024-05-08T05:55:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.400+0000 7f9eccc00700 1 -- 172.21.15.112:0/2126928135 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9ed400f5f0 con 0x55f46b111970 2024-05-08T05:55:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.400+0000 7f9ef3d9db80 10 client.14991 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:55:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.400+0000 7f9ef3d9db80 10 client.14991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:55:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.400+0000 7f9ef3d9db80 20 client.14991 populate_metadata read hostname 'smithi112' 2024-05-08T05:55:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.400+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/2126928135 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f46b12ac30 0x55f46b12d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.400+0000 7f9ef3d9db80 4 client.14991 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:55:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.400+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/2126928135 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f46b0de1c0 con 0x55f46b12ac30 2024-05-08T05:55:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.401+0000 7f9ee1e00700 1 --2- 172.21.15.112:0/2126928135 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f46b12ac30 0x55f46b12d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.401+0000 7f9eccc00700 10 client.14991 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:55:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.401+0000 7f9ee1e00700 1 --2- 172.21.15.112:0/2126928135 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f46b12ac30 0x55f46b12d020 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4da3b230b581a9c1 server_cookie=9890ced633b5f58b in_seq=0 out_seq=0 2024-05-08T05:55:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.402+0000 7f9eccc00700 1 -- 172.21.15.112:0/2126928135 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f46b0de1c0 con 0x55f46b12ac30 2024-05-08T05:55:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.402+0000 7f9eccc00700 10 client.14991 handle_command_reply: tid=0 2024-05-08T05:55:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.402+0000 7f9ef3d9db80 1 client.14991 shutdown 2024-05-08T05:55:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.402+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/2126928135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b111970 msgr2=0x55f46b11d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:16.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.402+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/2126928135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b111970 0x55f46b11d230 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4012840 tx=0x7f9ed4012b50 comp rx=0 tx=0).stop 2024-05-08T05:55:16.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.402+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/2126928135 shutdown_connections 2024-05-08T05:55:16.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.402+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/2126928135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b117560 0x55f46b11ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.402+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/2126928135 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f46b12ac30 0x55f46b12d020 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.402+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/2126928135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b111970 0x55f46b11d230 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.402+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/2126928135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b116c40 0x55f46b11a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.402+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/2126928135 >> 172.21.15.112:0/2126928135 conn(0x55f46b089750 msgr2=0x55f46b10b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:16.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.402+0000 7f9eccc00700 0 client.14991 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:55:16.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.403+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/2126928135 shutdown_connections 2024-05-08T05:55:16.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.403+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/2126928135 wait complete. 2024-05-08T05:55:16.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.403+0000 7f9ef3d9db80 20 client.14991 trim_cache size 0 max 16384 2024-05-08T05:55:16.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.425+0000 7f9ef3d9db80 1 Processor -- start 2024-05-08T05:55:16.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.425+0000 7f9ef3d9db80 1 -- start start 2024-05-08T05:55:16.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.425+0000 7f9ef3d9db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b111970 0x55f46b111350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.425+0000 7f9ef3d9db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b116c40 0x55f46b126db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.426+0000 7f9ef3d9db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b117560 0x55f46b10b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.426+0000 7f9ef3d9db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41ae180 con 0x55f46b116c40 2024-05-08T05:55:16.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.426+0000 7f9ef3d9db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41aba10 con 0x55f46b111970 2024-05-08T05:55:16.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.426+0000 7f9ef3d9db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee407b890 con 0x55f46b117560 2024-05-08T05:55:16.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.426+0000 7f9ee1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b116c40 0x55f46b126db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.426+0000 7f9ee1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b116c40 0x55f46b126db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34556/0 (socket says 172.21.15.112:34556) 2024-05-08T05:55:16.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.426+0000 7f9ee1e00700 1 -- 172.21.15.112:0/3050924839 learned_addr learned my addr 172.21.15.112:0/3050924839 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:16.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.426+0000 7f9ee3200700 1 --2- 172.21.15.112:0/3050924839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b117560 0x55f46b10b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.426+0000 7f9ee2800700 1 --2- 172.21.15.112:0/3050924839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b111970 0x55f46b111350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.426+0000 7f9ee1e00700 1 -- 172.21.15.112:0/3050924839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b117560 msgr2=0x55f46b10b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:16.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.426+0000 7f9ee1e00700 1 --2- 172.21.15.112:0/3050924839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b117560 0x55f46b10b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.426+0000 7f9ee1e00700 1 -- 172.21.15.112:0/3050924839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b111970 msgr2=0x55f46b111350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:16.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.426+0000 7f9ee1e00700 1 --2- 172.21.15.112:0/3050924839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b111970 0x55f46b111350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.426+0000 7f9ee1e00700 1 -- 172.21.15.112:0/3050924839 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f46b12d680 con 0x55f46b116c40 2024-05-08T05:55:16.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.426+0000 7f9ee3200700 1 --2- 172.21.15.112:0/3050924839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b117560 0x55f46b10b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:16.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.427+0000 7f9ee1e00700 1 --2- 172.21.15.112:0/3050924839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b116c40 0x55f46b126db0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009e00 tx=0x7f9ed0029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d12fcb31ef12cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:16.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.427+0000 7f9ecf400700 1 -- 172.21.15.112:0/3050924839 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed003a070 con 0x55f46b116c40 2024-05-08T05:55:16.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.427+0000 7f9ecf400700 1 -- 172.21.15.112:0/3050924839 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed0035420 con 0x55f46b116c40 2024-05-08T05:55:16.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.427+0000 7f9ecf400700 1 -- 172.21.15.112:0/3050924839 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00038c0 con 0x55f46b116c40 2024-05-08T05:55:16.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.428+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3050924839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b116c40 msgr2=0x55f46b126db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:16.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.428+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/3050924839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b116c40 0x55f46b126db0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009e00 tx=0x7f9ed0029000 comp rx=0 tx=0).stop 2024-05-08T05:55:16.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.428+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3050924839 shutdown_connections 2024-05-08T05:55:16.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.428+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/3050924839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b117560 0x55f46b10b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.428+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/3050924839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b111970 0x55f46b111350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.428+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/3050924839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b116c40 0x55f46b126db0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.428+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3050924839 >> 172.21.15.112:0/3050924839 conn(0x55f46b089750 msgr2=0x55f46b115f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:16.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.428+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3050924839 shutdown_connections 2024-05-08T05:55:16.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.428+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3050924839 wait complete. 2024-05-08T05:55:16.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.429+0000 7f9ef3d9db80 1 -- start start 2024-05-08T05:55:16.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.429+0000 7f9ef3d9db80 1 Processor -- start 2024-05-08T05:55:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.429+0000 7f9ef3d9db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b111970 0x55f46b123410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.429+0000 7f9ef3d9db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b116c40 0x55f46b120100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.429+0000 7f9ef3d9db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b117560 0x55f46b120640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.430+0000 7f9ef3d9db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41a9ed0 con 0x55f46b116c40 2024-05-08T05:55:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.430+0000 7f9ef3d9db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41ab340 con 0x55f46b111970 2024-05-08T05:55:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.430+0000 7f9ef3d9db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41aa5a0 con 0x55f46b117560 2024-05-08T05:55:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.430+0000 7f9ef3d9db80 10 client.0 _ll_register_callbacks cb 0x55f46b00acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.430+0000 7f9ee1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b116c40 0x55f46b120100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.430+0000 7f9ee1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b116c40 0x55f46b120100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34560/0 (socket says 172.21.15.112:34560) 2024-05-08T05:55:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.430+0000 7f9ee1e00700 1 -- 172.21.15.112:0/3026602582 learned_addr learned my addr 172.21.15.112:0/3026602582 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.430+0000 7f9ee2800700 1 --2- 172.21.15.112:0/3026602582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b111970 0x55f46b123410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.430+0000 7f9ee3200700 1 --2- 172.21.15.112:0/3026602582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b117560 0x55f46b120640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.430+0000 7f9ee1e00700 1 -- 172.21.15.112:0/3026602582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b117560 msgr2=0x55f46b120640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:16.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.430+0000 7f9ee1e00700 1 --2- 172.21.15.112:0/3026602582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b117560 0x55f46b120640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.430+0000 7f9ee1e00700 1 -- 172.21.15.112:0/3026602582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b111970 msgr2=0x55f46b123410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:16.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.430+0000 7f9ee1e00700 1 --2- 172.21.15.112:0/3026602582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b111970 0x55f46b123410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.430+0000 7f9ee1e00700 1 -- 172.21.15.112:0/3026602582 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f46b120c30 con 0x55f46b116c40 2024-05-08T05:55:16.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.431+0000 7f9ee1e00700 1 --2- 172.21.15.112:0/3026602582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b116c40 0x55f46b120100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000c00 tx=0x7f9ed0011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e7c3ceb033f97b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:16.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.431+0000 7f9eccc00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:55:16.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.431+0000 7f9eccc00700 1 -- 172.21.15.112:0/3026602582 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0043060 con 0x55f46b116c40 2024-05-08T05:55:16.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.431+0000 7f9eccc00700 1 -- 172.21.15.112:0/3026602582 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f9ed00359e0 con 0x55f46b116c40 2024-05-08T05:55:16.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.431+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3026602582 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed0035420 con 0x55f46b116c40 2024-05-08T05:55:16.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.431+0000 7f9eccc00700 1 -- 172.21.15.112:0/3026602582 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9ed003a050 con 0x55f46b116c40 2024-05-08T05:55:16.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.431+0000 7f9ef3d9db80 10 client.14997 fetch_fsmap learned FSMap version 25 2024-05-08T05:55:16.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.431+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3026602582 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f46b196360 con 0x55f46b116c40 2024-05-08T05:55:16.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.432+0000 7f9eccc00700 1 -- 172.21.15.112:0/3026602582 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed0035420 con 0x55f46b116c40 2024-05-08T05:55:16.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.432+0000 7f9eccc00700 1 -- 172.21.15.112:0/3026602582 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9ed0003840 con 0x55f46b116c40 2024-05-08T05:55:16.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.432+0000 7f9ef3d9db80 10 client.14997 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:55:16.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.432+0000 7f9ef3d9db80 10 client.14997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:55:16.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.432+0000 7f9ef3d9db80 20 client.14997 populate_metadata read hostname 'smithi112' 2024-05-08T05:55:16.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.432+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/3026602582 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f46b196990 0x55f46b198db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:16.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.432+0000 7f9ef3d9db80 4 client.14997 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T05:55:16.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.432+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3026602582 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f46b071480 con 0x55f46b196990 2024-05-08T05:55:16.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.432+0000 7f9ee2800700 1 --2- 172.21.15.112:0/3026602582 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f46b196990 0x55f46b198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:16.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.433+0000 7f9ee2800700 1 --2- 172.21.15.112:0/3026602582 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f46b196990 0x55f46b198db0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bef05dd03197fd7 server_cookie=6ed756f83bc1c5a0 in_seq=0 out_seq=0 2024-05-08T05:55:16.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.433+0000 7f9eccc00700 10 client.14997 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:55:16.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.434+0000 7f9eccc00700 1 -- 172.21.15.112:0/3026602582 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f46b071480 con 0x55f46b196990 2024-05-08T05:55:16.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.434+0000 7f9eccc00700 10 client.14997 handle_command_reply: tid=0 2024-05-08T05:55:16.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.434+0000 7f9ef3d9db80 1 client.14997 shutdown 2024-05-08T05:55:16.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.435+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3026602582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b116c40 msgr2=0x55f46b120100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:16.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.435+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/3026602582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b116c40 0x55f46b120100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000c00 tx=0x7f9ed0011d50 comp rx=0 tx=0).stop 2024-05-08T05:55:16.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.435+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3026602582 shutdown_connections 2024-05-08T05:55:16.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.435+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/3026602582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f46b117560 0x55f46b120640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.435+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/3026602582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f46b111970 0x55f46b123410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.435+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/3026602582 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f46b196990 0x55f46b198db0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.435+0000 7f9ef3d9db80 1 --2- 172.21.15.112:0/3026602582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f46b116c40 0x55f46b120100 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000c00 tx=0x7f9ed0011d50 comp rx=0 tx=0).stop 2024-05-08T05:55:16.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.435+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3026602582 >> 172.21.15.112:0/3026602582 conn(0x55f46b089750 msgr2=0x55f46b11bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:16.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.435+0000 7f9eccc00700 0 client.14997 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:55:16.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.435+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3026602582 shutdown_connections 2024-05-08T05:55:16.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.436+0000 7f9ef3d9db80 1 -- 172.21.15.112:0/3026602582 wait complete. 2024-05-08T05:55:16.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.436+0000 7f9ef3d9db80 20 client.14997 trim_cache size 0 max 16384 2024-05-08T05:55:16.437 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:55:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-08T05:55:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "176762f1-6424-4934-b19e-bb1bb77e2f33", 2024-05-08T05:55:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-08T05:55:16.438 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:55:16.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.437+0000 7f9ee9400700 1 -- 172.21.15.112:0/936163370 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9ec80b2880 msgr2=0x7f9ec80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:16.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.437+0000 7f9ee9400700 1 --2- 172.21.15.112:0/936163370 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9ec80b2880 0x7f9ec80b4d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00097a0 tx=0x7f9ed001e8f0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.437+0000 7f9ee9400700 1 -- 172.21.15.112:0/936163370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee410b6c0 msgr2=0x7f9ee41a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:16.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.437+0000 7f9ee9400700 1 --2- 172.21.15.112:0/936163370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee410b6c0 0x7f9ee41a0630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00b070 tx=0x7f9edc009c90 comp rx=0 tx=0).stop 2024-05-08T05:55:16.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.437+0000 7f9ee9400700 1 -- 172.21.15.112:0/936163370 shutdown_connections 2024-05-08T05:55:16.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.437+0000 7f9ee9400700 1 --2- 172.21.15.112:0/936163370 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f9ec80b2880 0x7f9ec80b4d40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.437+0000 7f9ee9400700 1 --2- 172.21.15.112:0/936163370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ee410b6c0 0x7f9ee41a0630 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.437+0000 7f9ee9400700 1 --2- 172.21.15.112:0/936163370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ee41056c0 0x7f9ee419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.437+0000 7f9ee9400700 1 --2- 172.21.15.112:0/936163370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee4106080 0x7f9ee419c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:16.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.437+0000 7f9ee9400700 1 -- 172.21.15.112:0/936163370 >> 172.21.15.112:0/936163370 conn(0x7f9ee41011b0 msgr2=0x7f9ee4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:16.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.437+0000 7f9ee9400700 1 -- 172.21.15.112:0/936163370 shutdown_connections 2024-05-08T05:55:16.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:16.438+0000 7f9ee9400700 1 -- 172.21.15.112:0/936163370 wait complete. 2024-05-08T05:55:17.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:16 smithi112 ceph-mon[30003]: pgmap v440: 97 pgs: 97 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-08T05:55:17.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:16 smithi162 ceph-mon[34800]: pgmap v440: 97 pgs: 97 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-08T05:55:17.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:16 smithi150 ceph-mon[34315]: pgmap v440: 97 pgs: 97 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-08T05:55:17.250 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T05:55:17.697 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:55:18.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:17 smithi112 ceph-mon[30003]: scrub queued for path: / 2024-05-08T05:55:18.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:17 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:55:18.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:17 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T05:55:18.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:17 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:55:18.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:17 smithi162 ceph-mon[34800]: scrub queued for path: / 2024-05-08T05:55:18.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:17 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:55:18.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:17 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T05:55:18.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:17 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:55:18.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:17 smithi150 ceph-mon[34315]: scrub queued for path: / 2024-05-08T05:55:18.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:17 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:55:18.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:17 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T05:55:18.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:17 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:55:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:18 smithi112 ceph-mon[30003]: pgmap v441: 97 pgs: 97 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T05:55:19.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:18 smithi162 ceph-mon[34800]: pgmap v441: 97 pgs: 97 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T05:55:19.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:18 smithi150 ceph-mon[34315]: pgmap v441: 97 pgs: 97 active+clean; 652 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T05:55:19.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.964+0000 7f35d6e00700 1 -- 172.21.15.112:0/1911916768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35d0104100 msgr2=0x7f35d0108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:19.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.964+0000 7f35d6e00700 1 --2- 172.21.15.112:0/1911916768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35d0104100 0x7f35d0108120 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0098f0 tx=0x7f35bc009c00 comp rx=0 tx=0).stop 2024-05-08T05:55:19.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.964+0000 7f35d6e00700 1 -- 172.21.15.112:0/1911916768 shutdown_connections 2024-05-08T05:55:19.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.964+0000 7f35d6e00700 1 --2- 172.21.15.112:0/1911916768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35d0108850 0x7f35d01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.964+0000 7f35d6e00700 1 --2- 172.21.15.112:0/1911916768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d0103750 0x7f35d0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.964+0000 7f35d6e00700 1 --2- 172.21.15.112:0/1911916768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35d0104100 0x7f35d0108120 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.964+0000 7f35d6e00700 1 -- 172.21.15.112:0/1911916768 >> 172.21.15.112:0/1911916768 conn(0x7f35d007a600 msgr2=0x7f35d007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:19.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.965+0000 7f35d6e00700 1 -- 172.21.15.112:0/1911916768 shutdown_connections 2024-05-08T05:55:19.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.965+0000 7f35d6e00700 1 -- 172.21.15.112:0/1911916768 wait complete. 2024-05-08T05:55:19.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.966+0000 7f35d6e00700 1 Processor -- start 2024-05-08T05:55:19.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.966+0000 7f35d6e00700 1 -- start start 2024-05-08T05:55:19.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.967+0000 7f35d6e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35d0103750 0x7f35d019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:19.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.967+0000 7f35d6e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35d0104100 0x7f35d019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:19.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.967+0000 7f35d6e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d0108850 0x7f35d01999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:19.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.967+0000 7f35d6e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35d0113680 con 0x7f35d0108850 2024-05-08T05:55:19.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.967+0000 7f35d6e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f35d0113500 con 0x7f35d0103750 2024-05-08T05:55:19.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.967+0000 7f35d6e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f35d0113800 con 0x7f35d0104100 2024-05-08T05:55:19.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.967+0000 7f35d4e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d0108850 0x7f35d01999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:19.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.967+0000 7f35d4e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d0108850 0x7f35d01999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34580/0 (socket says 172.21.15.112:34580) 2024-05-08T05:55:19.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.967+0000 7f35d4e00700 1 -- 172.21.15.112:0/2588768654 learned_addr learned my addr 172.21.15.112:0/2588768654 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:19.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.967+0000 7f35cfe00700 1 --2- 172.21.15.112:0/2588768654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35d0103750 0x7f35d019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:19.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.967+0000 7f35cf400700 1 --2- 172.21.15.112:0/2588768654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35d0104100 0x7f35d019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:19.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.968+0000 7f35d4e00700 1 -- 172.21.15.112:0/2588768654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35d0104100 msgr2=0x7f35d019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:19.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.968+0000 7f35d4e00700 1 --2- 172.21.15.112:0/2588768654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35d0104100 0x7f35d019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.968+0000 7f35d4e00700 1 -- 172.21.15.112:0/2588768654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35d0103750 msgr2=0x7f35d019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:19.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.968+0000 7f35d4e00700 1 --2- 172.21.15.112:0/2588768654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35d0103750 0x7f35d019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.968+0000 7f35d4e00700 1 -- 172.21.15.112:0/2588768654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d019a120 con 0x7f35d0108850 2024-05-08T05:55:19.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.968+0000 7f35d4e00700 1 --2- 172.21.15.112:0/2588768654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d0108850 0x7f35d01999c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f35c800ef30 tx=0x7f35c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:19.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.968+0000 7f35ccc00700 1 -- 172.21.15.112:0/2588768654 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c80092e0 con 0x7f35d0108850 2024-05-08T05:55:19.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.968+0000 7f35ccc00700 1 -- 172.21.15.112:0/2588768654 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35c8004bb0 con 0x7f35d0108850 2024-05-08T05:55:19.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.968+0000 7f35d6e00700 1 -- 172.21.15.112:0/2588768654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d019a3b0 con 0x7f35d0108850 2024-05-08T05:55:19.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.968+0000 7f35ccc00700 1 -- 172.21.15.112:0/2588768654 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8004020 con 0x7f35d0108850 2024-05-08T05:55:19.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.969+0000 7f35d6e00700 1 -- 172.21.15.112:0/2588768654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d01a6680 con 0x7f35d0108850 2024-05-08T05:55:19.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.972+0000 7f35ccc00700 1 -- 172.21.15.112:0/2588768654 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f35c801b030 con 0x7f35d0108850 2024-05-08T05:55:19.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.973+0000 7f35e17f4b80 1 Processor -- start 2024-05-08T05:55:19.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.973+0000 7f35e17f4b80 1 -- start start 2024-05-08T05:55:19.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.974+0000 7f35ccc00700 1 --2- 172.21.15.112:0/2588768654 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f35b40b2880 0x7f35b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:19.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.974+0000 7f35e17f4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b0081440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:19.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.974+0000 7f35e17f4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 0x5624b0085490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:19.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.974+0000 7f35e17f4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b00859d0 0x5624b0085e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:19.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.974+0000 7f35e17f4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5624affd3a20 con 0x5624b0081980 2024-05-08T05:55:19.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.974+0000 7f35ccc00700 1 -- 172.21.15.112:0/2588768654 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f35c80e1d60 con 0x7f35d0108850 2024-05-08T05:55:19.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.974+0000 7f35e17f4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5624aff71760 con 0x5624b00859d0 2024-05-08T05:55:19.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.974+0000 7f35e17f4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5624affcf9e0 con 0x5624b0081060 2024-05-08T05:55:19.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.974+0000 7f35cf400700 1 --2- 172.21.15.112:0/2588768654 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f35b40b2880 0x7f35b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:19.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.974+0000 7f35cf400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 0x5624b0085490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:19.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.974+0000 7f35cfe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b0081440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:19.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.974+0000 7f35cfe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b0081440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50810/0 (socket says 172.21.15.112:50810) 2024-05-08T05:55:19.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.974+0000 7f35cf400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 0x5624b0085490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34596/0 (socket says 172.21.15.112:34596) 2024-05-08T05:55:19.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.974+0000 7f35cfe00700 1 -- 172.21.15.112:0/2526052961 learned_addr learned my addr 172.21.15.112:0/2526052961 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.975+0000 7f35d4e00700 1 --2- 172.21.15.112:0/2526052961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b00859d0 0x5624b0085e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.975+0000 7f35d4e00700 1 -- 172.21.15.112:0/2526052961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 msgr2=0x5624b0081440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.975+0000 7f35d4e00700 1 --2- 172.21.15.112:0/2526052961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b0081440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.975+0000 7f35d4e00700 1 -- 172.21.15.112:0/2526052961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 msgr2=0x5624b0085490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.975+0000 7f35d4e00700 1 --2- 172.21.15.112:0/2526052961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 0x5624b0085490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.975+0000 7f35d4e00700 1 -- 172.21.15.112:0/2526052961 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624aff208d0 con 0x5624b00859d0 2024-05-08T05:55:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.975+0000 7f35cfe00700 1 --2- 172.21.15.112:0/2526052961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b0081440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.975+0000 7f35cf400700 1 --2- 172.21.15.112:0/2526052961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 0x5624b0085490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.975+0000 7f35cf400700 1 --2- 172.21.15.112:0/2588768654 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f35b40b2880 0x7f35b40b4d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0098c0 tx=0x7f35bc009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.975+0000 7f35d4e00700 1 --2- 172.21.15.112:0/2526052961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b00859d0 0x5624b0085e30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f35c80df7d0 tx=0x7f35c800ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63de1cc786f32ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.976+0000 7f35c5600700 1 -- 172.21.15.112:0/2526052961 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c80a9380 con 0x5624b00859d0 2024-05-08T05:55:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.976+0000 7f35c5600700 1 -- 172.21.15.112:0/2526052961 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35c80ad730 con 0x5624b00859d0 2024-05-08T05:55:19.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.976+0000 7f35c5600700 1 -- 172.21.15.112:0/2526052961 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c80a4f50 con 0x5624b00859d0 2024-05-08T05:55:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.976+0000 7f35e17f4b80 1 -- 172.21.15.112:0/2526052961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b00859d0 msgr2=0x5624b0085e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.976+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/2526052961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b00859d0 0x5624b0085e30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f35c80df7d0 tx=0x7f35c800ca30 comp rx=0 tx=0).stop 2024-05-08T05:55:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.977+0000 7f35e17f4b80 1 -- 172.21.15.112:0/2526052961 shutdown_connections 2024-05-08T05:55:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.977+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/2526052961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b00859d0 0x5624b0085e30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.977+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/2526052961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b0081440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.977+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/2526052961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 0x5624b0085490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.977+0000 7f35e17f4b80 1 -- 172.21.15.112:0/2526052961 >> 172.21.15.112:0/2526052961 conn(0x5624affafb60 msgr2=0x5624b007b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.977+0000 7f35e17f4b80 1 -- 172.21.15.112:0/2526052961 shutdown_connections 2024-05-08T05:55:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.977+0000 7f35e17f4b80 1 -- 172.21.15.112:0/2526052961 wait complete. 2024-05-08T05:55:19.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.977+0000 7f35e17f4b80 1 -- start start 2024-05-08T05:55:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.978+0000 7f35e17f4b80 1 Processor -- start 2024-05-08T05:55:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.978+0000 7f35e17f4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b008ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.978+0000 7f35e17f4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 0x5624b008a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.978+0000 7f35e17f4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b00859d0 0x5624b008a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.978+0000 7f35e17f4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5624b008db80 con 0x5624b0081980 2024-05-08T05:55:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.978+0000 7f35e17f4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5624aff39b20 con 0x5624b00859d0 2024-05-08T05:55:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.978+0000 7f35e17f4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5624b008dd00 con 0x5624b0081060 2024-05-08T05:55:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.978+0000 7f35e17f4b80 10 client.0 _ll_register_callbacks cb 0x5624aff7ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.979+0000 7f35cfe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b008ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.979+0000 7f35cf400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 0x5624b008a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.979+0000 7f35d4e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b00859d0 0x5624b008a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.979+0000 7f35cfe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b008ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50820/0 (socket says 172.21.15.112:50820) 2024-05-08T05:55:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.979+0000 7f35cfe00700 1 -- 172.21.15.112:0/309329715 learned_addr learned my addr 172.21.15.112:0/309329715 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:19.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.979+0000 7f35cfe00700 1 -- 172.21.15.112:0/309329715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b00859d0 msgr2=0x5624b008a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:19.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.979+0000 7f35cfe00700 1 --2- 172.21.15.112:0/309329715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b00859d0 0x5624b008a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.979+0000 7f35cfe00700 1 -- 172.21.15.112:0/309329715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 msgr2=0x5624b008a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:19.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.979+0000 7f35cfe00700 1 --2- 172.21.15.112:0/309329715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 0x5624b008a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.979+0000 7f35cfe00700 1 -- 172.21.15.112:0/309329715 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624aff99420 con 0x5624b0081060 2024-05-08T05:55:19.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.979+0000 7f35cfe00700 1 --2- 172.21.15.112:0/309329715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b008ce40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009a20 tx=0x7f35c0012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf27dbe2c70e1d99 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:19.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.979+0000 7f35ba000700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:55:19.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.980+0000 7f35ba000700 1 -- 172.21.15.112:0/309329715 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c000f4e0 con 0x5624b0081060 2024-05-08T05:55:19.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.980+0000 7f35e17f4b80 1 -- 172.21.15.112:0/309329715 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35c80ad730 con 0x5624b0081060 2024-05-08T05:55:19.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.980+0000 7f35ba000700 1 -- 172.21.15.112:0/309329715 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f35c000f680 con 0x5624b0081060 2024-05-08T05:55:19.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.980+0000 7f35ba000700 1 -- 172.21.15.112:0/309329715 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f35c000f880 con 0x5624b0081060 2024-05-08T05:55:19.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.980+0000 7f35e17f4b80 10 client.25141 fetch_fsmap learned FSMap version 25 2024-05-08T05:55:19.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.980+0000 7f35e17f4b80 1 -- 172.21.15.112:0/309329715 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5624b0098440 con 0x5624b0081060 2024-05-08T05:55:19.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.980+0000 7f35ba000700 1 -- 172.21.15.112:0/309329715 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35c80ad730 con 0x5624b0081060 2024-05-08T05:55:19.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.980+0000 7f35ba000700 1 -- 172.21.15.112:0/309329715 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f35c0026a40 con 0x5624b0081060 2024-05-08T05:55:19.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.981+0000 7f35e17f4b80 10 client.25141 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:55:19.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.981+0000 7f35e17f4b80 10 client.25141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:55:19.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.981+0000 7f35e17f4b80 20 client.25141 populate_metadata read hostname 'smithi112' 2024-05-08T05:55:19.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.981+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/309329715 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5624b009aa20 0x5624b009ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:19.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.981+0000 7f35e17f4b80 4 client.25141 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:55:19.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.981+0000 7f35e17f4b80 1 -- 172.21.15.112:0/309329715 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624b004e0a0 con 0x5624b009aa20 2024-05-08T05:55:19.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.981+0000 7f35cf400700 1 --2- 172.21.15.112:0/309329715 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5624b009aa20 0x5624b009ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:19.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.982+0000 7f35cf400700 1 --2- 172.21.15.112:0/309329715 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5624b009aa20 0x5624b009ce70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0407cd75288ccb7 server_cookie=53cb171c25895ea4 in_seq=0 out_seq=0 2024-05-08T05:55:19.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.982+0000 7f35ba000700 10 client.25141 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:55:19.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.982+0000 7f35ba000700 1 -- 172.21.15.112:0/309329715 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5624b004e0a0 con 0x5624b009aa20 2024-05-08T05:55:19.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.982+0000 7f35ba000700 10 client.25141 handle_command_reply: tid=0 2024-05-08T05:55:19.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.982+0000 7f35e17f4b80 1 client.25141 shutdown 2024-05-08T05:55:19.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.983+0000 7f35e17f4b80 1 -- 172.21.15.112:0/309329715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 msgr2=0x5624b008ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:19.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.983+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/309329715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b008ce40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009a20 tx=0x7f35c0012b60 comp rx=0 tx=0).stop 2024-05-08T05:55:19.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.983+0000 7f35e17f4b80 1 -- 172.21.15.112:0/309329715 shutdown_connections 2024-05-08T05:55:19.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.983+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/309329715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b00859d0 0x5624b008a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.983+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/309329715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b008ce40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.983+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/309329715 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5624b009aa20 0x5624b009ce70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.983+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/309329715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 0x5624b008a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:19.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.983+0000 7f35ba000700 0 client.25141 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:55:19.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.983+0000 7f35e17f4b80 1 -- 172.21.15.112:0/309329715 >> 172.21.15.112:0/309329715 conn(0x5624affafb60 msgr2=0x5624b00762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:19.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.983+0000 7f35e17f4b80 1 -- 172.21.15.112:0/309329715 shutdown_connections 2024-05-08T05:55:19.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.983+0000 7f35e17f4b80 1 -- 172.21.15.112:0/309329715 wait complete. 2024-05-08T05:55:19.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:19.983+0000 7f35e17f4b80 20 client.25141 trim_cache size 0 max 16384 2024-05-08T05:55:20.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.005+0000 7f35e17f4b80 1 Processor -- start 2024-05-08T05:55:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.005+0000 7f35e17f4b80 1 -- start start 2024-05-08T05:55:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.005+0000 7f35e17f4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624affafb60 0x5624b0096c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.005+0000 7f35e17f4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b0076130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.005+0000 7f35e17f4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b0081980 0x5624b00a2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.005+0000 7f35e17f4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35d01b0490 con 0x5624affafb60 2024-05-08T05:55:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.005+0000 7f35e17f4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f35d01add20 con 0x5624b0081980 2024-05-08T05:55:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.005+0000 7f35e17f4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f35d01abb70 con 0x5624b0081060 2024-05-08T05:55:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.006+0000 7f35cfe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624affafb60 0x5624b0096c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.006+0000 7f35cfe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624affafb60 0x5624b0096c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34608/0 (socket says 172.21.15.112:34608) 2024-05-08T05:55:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.006+0000 7f35cfe00700 1 -- 172.21.15.112:0/3603124403 learned_addr learned my addr 172.21.15.112:0/3603124403 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:20.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.006+0000 7f35d4e00700 1 --2- 172.21.15.112:0/3603124403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b0081980 0x5624b00a2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:20.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.006+0000 7f35cf400700 1 --2- 172.21.15.112:0/3603124403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b0076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:20.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.006+0000 7f35cfe00700 1 -- 172.21.15.112:0/3603124403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 msgr2=0x5624b0076130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:20.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.006+0000 7f35cfe00700 1 --2- 172.21.15.112:0/3603124403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b0076130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.006+0000 7f35cfe00700 1 -- 172.21.15.112:0/3603124403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b0081980 msgr2=0x5624b00a2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:20.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.006+0000 7f35cfe00700 1 --2- 172.21.15.112:0/3603124403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b0081980 0x5624b00a2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.006+0000 7f35cfe00700 1 -- 172.21.15.112:0/3603124403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b007c9c0 con 0x5624affafb60 2024-05-08T05:55:20.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.006+0000 7f35cf400700 1 --2- 172.21.15.112:0/3603124403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b0076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:55:20.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.007+0000 7f35cfe00700 1 --2- 172.21.15.112:0/3603124403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624affafb60 0x5624b0096c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f35c0026ee0 tx=0x7f35c000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e370cd64f7b9c9a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:20.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.007+0000 7f35c4c00700 1 -- 172.21.15.112:0/3603124403 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c000fa20 con 0x5624affafb60 2024-05-08T05:55:20.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.007+0000 7f35c4c00700 1 -- 172.21.15.112:0/3603124403 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35c000fbc0 con 0x5624affafb60 2024-05-08T05:55:20.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.007+0000 7f35c4c00700 1 -- 172.21.15.112:0/3603124403 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0026760 con 0x5624affafb60 2024-05-08T05:55:20.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.008+0000 7f35e17f4b80 1 -- 172.21.15.112:0/3603124403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624affafb60 msgr2=0x5624b0096c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:20.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.008+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/3603124403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624affafb60 0x5624b0096c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f35c0026ee0 tx=0x7f35c000f770 comp rx=0 tx=0).stop 2024-05-08T05:55:20.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.008+0000 7f35e17f4b80 1 -- 172.21.15.112:0/3603124403 shutdown_connections 2024-05-08T05:55:20.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.008+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/3603124403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624b0081980 0x5624b00a2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.008+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/3603124403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624affafb60 0x5624b0096c10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.008+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/3603124403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b0076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.008+0000 7f35e17f4b80 1 -- 172.21.15.112:0/3603124403 >> 172.21.15.112:0/3603124403 conn(0x5624b008b0d0 msgr2=0x5624b0075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:20.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.008+0000 7f35e17f4b80 1 -- 172.21.15.112:0/3603124403 shutdown_connections 2024-05-08T05:55:20.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.008+0000 7f35e17f4b80 1 -- 172.21.15.112:0/3603124403 wait complete. 2024-05-08T05:55:20.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.008+0000 7f35e17f4b80 1 -- start start 2024-05-08T05:55:20.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.008+0000 7f35e17f4b80 1 Processor -- start 2024-05-08T05:55:20.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.009+0000 7f35e17f4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624affafb60 0x5624b0093360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:20.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.009+0000 7f35e17f4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b008df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:20.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.009+0000 7f35e17f4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 0x5624b008e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:20.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.009+0000 7f35e17f4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35d01ac1e0 con 0x5624b0081980 2024-05-08T05:55:20.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.009+0000 7f35e17f4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f35d01ad650 con 0x5624affafb60 2024-05-08T05:55:20.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.009+0000 7f35e17f4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f35d01ac8b0 con 0x5624b0081060 2024-05-08T05:55:20.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.009+0000 7f35e17f4b80 10 client.0 _ll_register_callbacks cb 0x5624aff7ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:20.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.009+0000 7f35cfe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624affafb60 0x5624b0093360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:20.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.009+0000 7f35cfe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624affafb60 0x5624b0093360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:56970/0 (socket says 172.21.15.112:56970) 2024-05-08T05:55:20.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.009+0000 7f35cfe00700 1 -- 172.21.15.112:0/366759675 learned_addr learned my addr 172.21.15.112:0/366759675 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:20.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.009+0000 7f35cf400700 1 --2- 172.21.15.112:0/366759675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b008df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:20.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.009+0000 7f35d4e00700 1 --2- 172.21.15.112:0/366759675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 0x5624b008e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:20.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.010+0000 7f35cfe00700 1 -- 172.21.15.112:0/366759675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 msgr2=0x5624b008df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:20.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.010+0000 7f35cfe00700 1 --2- 172.21.15.112:0/366759675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b008df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.010+0000 7f35cfe00700 1 -- 172.21.15.112:0/366759675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 msgr2=0x5624b008e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:20.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.010+0000 7f35cfe00700 1 --2- 172.21.15.112:0/366759675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 0x5624b008e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.010+0000 7f35cfe00700 1 -- 172.21.15.112:0/366759675 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b008ead0 con 0x5624affafb60 2024-05-08T05:55:20.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.010+0000 7f35cfe00700 1 --2- 172.21.15.112:0/366759675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624affafb60 0x5624b0093360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f35c0029040 tx=0x7f35c0012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cfd1c028682609 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:20.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.010+0000 7f35ba000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:55:20.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.010+0000 7f35ba000700 1 -- 172.21.15.112:0/366759675 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c000fa20 con 0x5624affafb60 2024-05-08T05:55:20.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.010+0000 7f35e17f4b80 1 -- 172.21.15.112:0/366759675 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35c000fbc0 con 0x5624affafb60 2024-05-08T05:55:20.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.010+0000 7f35ba000700 1 -- 172.21.15.112:0/366759675 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f35c0026630 con 0x5624affafb60 2024-05-08T05:55:20.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.010+0000 7f35ba000700 1 -- 172.21.15.112:0/366759675 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f35c003f050 con 0x5624affafb60 2024-05-08T05:55:20.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.010+0000 7f35e17f4b80 10 client.25127 fetch_fsmap learned FSMap version 25 2024-05-08T05:55:20.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.010+0000 7f35e17f4b80 1 -- 172.21.15.112:0/366759675 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5624b010e8f0 con 0x5624affafb60 2024-05-08T05:55:20.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.011+0000 7f35ba000700 1 -- 172.21.15.112:0/366759675 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35c000fbc0 con 0x5624affafb60 2024-05-08T05:55:20.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.011+0000 7f35ba000700 1 -- 172.21.15.112:0/366759675 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f35c00109e0 con 0x5624affafb60 2024-05-08T05:55:20.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.011+0000 7f35e17f4b80 10 client.25127 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:55:20.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.011+0000 7f35e17f4b80 10 client.25127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:55:20.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.011+0000 7f35e17f4b80 20 client.25127 populate_metadata read hostname 'smithi112' 2024-05-08T05:55:20.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.011+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/366759675 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5624b00859d0 0x5624b0110ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:20.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.011+0000 7f35e17f4b80 4 client.25127 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:55:20.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.011+0000 7f35e17f4b80 1 -- 172.21.15.112:0/366759675 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5624affe1400 con 0x5624b00859d0 2024-05-08T05:55:20.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.011+0000 7f35cf400700 1 --2- 172.21.15.112:0/366759675 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5624b00859d0 0x5624b0110ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:20.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.012+0000 7f35cf400700 1 --2- 172.21.15.112:0/366759675 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5624b00859d0 0x5624b0110ea0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ecb9512744b6ac server_cookie=80714f429c524004 in_seq=0 out_seq=0 2024-05-08T05:55:20.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.012+0000 7f35ba000700 10 client.25127 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:55:20.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.012+0000 7f35ba000700 1 -- 172.21.15.112:0/366759675 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5624affe1400 con 0x5624b00859d0 2024-05-08T05:55:20.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.012+0000 7f35ba000700 10 client.25127 handle_command_reply: tid=0 2024-05-08T05:55:20.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.012+0000 7f35e17f4b80 1 client.25127 shutdown 2024-05-08T05:55:20.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.012+0000 7f35e17f4b80 1 -- 172.21.15.112:0/366759675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624affafb60 msgr2=0x5624b0093360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:20.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.012+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/366759675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624affafb60 0x5624b0093360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f35c0029040 tx=0x7f35c0012b60 comp rx=0 tx=0).stop 2024-05-08T05:55:20.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.012+0000 7f35e17f4b80 1 -- 172.21.15.112:0/366759675 shutdown_connections 2024-05-08T05:55:20.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.012+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/366759675 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5624b00859d0 0x5624b0110ea0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.012+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/366759675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624b0081980 0x5624b008e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.012+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/366759675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624affafb60 0x5624b0093360 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.012+0000 7f35e17f4b80 1 --2- 172.21.15.112:0/366759675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5624b0081060 0x5624b008df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.012+0000 7f35ba000700 0 client.25127 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:55:20.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.012+0000 7f35e17f4b80 1 -- 172.21.15.112:0/366759675 >> 172.21.15.112:0/366759675 conn(0x5624b008b0d0 msgr2=0x5624b00762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:20.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.013+0000 7f35e17f4b80 1 -- 172.21.15.112:0/366759675 shutdown_connections 2024-05-08T05:55:20.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.013+0000 7f35e17f4b80 1 -- 172.21.15.112:0/366759675 wait complete. 2024-05-08T05:55:20.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.013+0000 7f35e17f4b80 20 client.25127 trim_cache size 0 max 16384 2024-05-08T05:55:20.016 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:55:20.016 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T05:55:20.016 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-05-08T05:55:20.016 INFO:teuthology.orchestra.run.smithi112.stdout: "176762f1-6424-4934-b19e-bb1bb77e2f33": { 2024-05-08T05:55:20.016 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-05-08T05:55:20.017 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "176762f1-6424-4934-b19e-bb1bb77e2f33", 2024-05-08T05:55:20.017 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-05-08T05:55:20.017 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T05:55:20.017 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T05:55:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:55:20.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.014+0000 7f35d6e00700 1 -- 172.21.15.112:0/2588768654 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f35b40b2880 msgr2=0x7f35b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:20.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.014+0000 7f35d6e00700 1 --2- 172.21.15.112:0/2588768654 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f35b40b2880 0x7f35b40b4d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0098c0 tx=0x7f35bc009580 comp rx=0 tx=0).stop 2024-05-08T05:55:20.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.014+0000 7f35d6e00700 1 -- 172.21.15.112:0/2588768654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d0108850 msgr2=0x7f35d01999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:20.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.014+0000 7f35d6e00700 1 --2- 172.21.15.112:0/2588768654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d0108850 0x7f35d01999c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f35c800ef30 tx=0x7f35c800c3a0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.014+0000 7f35d6e00700 1 -- 172.21.15.112:0/2588768654 shutdown_connections 2024-05-08T05:55:20.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.014+0000 7f35d6e00700 1 --2- 172.21.15.112:0/2588768654 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f35b40b2880 0x7f35b40b4d40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.014+0000 7f35d6e00700 1 --2- 172.21.15.112:0/2588768654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d0108850 0x7f35d01999c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.014+0000 7f35d6e00700 1 --2- 172.21.15.112:0/2588768654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35d0103750 0x7f35d019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.014+0000 7f35d6e00700 1 --2- 172.21.15.112:0/2588768654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35d0104100 0x7f35d019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:20.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.014+0000 7f35d6e00700 1 -- 172.21.15.112:0/2588768654 >> 172.21.15.112:0/2588768654 conn(0x7f35d007a600 msgr2=0x7f35d0101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:20.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.014+0000 7f35d6e00700 1 -- 172.21.15.112:0/2588768654 shutdown_connections 2024-05-08T05:55:20.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:20.014+0000 7f35d6e00700 1 -- 172.21.15.112:0/2588768654 wait complete. 2024-05-08T05:55:20.962 INFO:tasks.cephfs.filesystem:scrub status for tag:176762f1-6424-4934-b19e-bb1bb77e2f33 - {'path': '/', 'tag': '176762f1-6424-4934-b19e-bb1bb77e2f33', 'options': 'recursive,force'} 2024-05-08T05:55:21.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:20 smithi112 ceph-mon[30003]: pgmap v442: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-08T05:55:21.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:20 smithi162 ceph-mon[34800]: pgmap v442: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-08T05:55:21.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:20 smithi150 ceph-mon[34315]: pgmap v442: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-08T05:55:23.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:22 smithi112 ceph-mon[30003]: pgmap v443: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T05:55:23.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:22 smithi162 ceph-mon[34800]: pgmap v443: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T05:55:23.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:22 smithi150 ceph-mon[34315]: pgmap v443: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T05:55:25.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:24 smithi112 ceph-mon[30003]: pgmap v444: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-08T05:55:25.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:24 smithi162 ceph-mon[34800]: pgmap v444: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-08T05:55:25.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:24 smithi150 ceph-mon[34315]: pgmap v444: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-08T05:55:26.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:25 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:55:26.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:25 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:55:26.084 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:25 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:55:27.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:26 smithi112 ceph-mon[30003]: pgmap v445: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-08T05:55:27.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:55:27.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:55:27.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:55:27.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:26 smithi162 ceph-mon[34800]: pgmap v445: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-08T05:55:27.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:55:27.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:55:27.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:55:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:26 smithi150 ceph-mon[34315]: pgmap v445: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-08T05:55:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:55:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:55:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:55:29.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:28 smithi112 ceph-mon[30003]: pgmap v446: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-08T05:55:29.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:28 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T05:55:29.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:28 smithi162 ceph-mon[34800]: pgmap v446: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-08T05:55:29.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:28 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T05:55:29.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:28 smithi150 ceph-mon[34315]: pgmap v446: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-08T05:55:29.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:28 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T05:55:31.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:30 smithi112 ceph-mon[30003]: pgmap v447: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T05:55:31.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:30 smithi162 ceph-mon[34800]: pgmap v447: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T05:55:31.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:30 smithi150 ceph-mon[34315]: pgmap v447: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T05:55:33.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:32 smithi112 ceph-mon[30003]: pgmap v448: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T05:55:33.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:32 smithi162 ceph-mon[34800]: pgmap v448: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T05:55:33.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:32 smithi150 ceph-mon[34315]: pgmap v448: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T05:55:35.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:34 smithi112 ceph-mon[30003]: pgmap v449: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-08T05:55:35.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:34 smithi162 ceph-mon[34800]: pgmap v449: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-08T05:55:35.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:34 smithi150 ceph-mon[34315]: pgmap v449: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-08T05:55:37.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:36 smithi112 ceph-mon[30003]: pgmap v450: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T05:55:37.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:36 smithi162 ceph-mon[34800]: pgmap v450: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T05:55:37.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:36 smithi150 ceph-mon[34315]: pgmap v450: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T05:55:39.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:38 smithi112 ceph-mon[30003]: pgmap v451: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T05:55:39.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:38 smithi162 ceph-mon[34800]: pgmap v451: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T05:55:39.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:38 smithi150 ceph-mon[34315]: pgmap v451: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T05:55:41.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:40 smithi112 ceph-mon[30003]: pgmap v452: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:55:41.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:40 smithi162 ceph-mon[34800]: pgmap v452: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:55:41.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:40 smithi150 ceph-mon[34315]: pgmap v452: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:55:43.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:42 smithi150 ceph-mon[34315]: pgmap v453: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:55:43.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:42 smithi112 ceph-mon[30003]: pgmap v453: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:55:43.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:42 smithi162 ceph-mon[34800]: pgmap v453: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:55:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:44 smithi150 ceph-mon[34315]: pgmap v454: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:55:45.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:44 smithi112 ceph-mon[30003]: pgmap v454: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:55:45.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:44 smithi162 ceph-mon[34800]: pgmap v454: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:55:47.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:46 smithi150 ceph-mon[34315]: pgmap v455: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:55:47.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:46 smithi112 ceph-mon[30003]: pgmap v455: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:55:47.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:46 smithi162 ceph-mon[34800]: pgmap v455: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:55:49.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:48 smithi150 ceph-mon[34315]: pgmap v456: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:55:49.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:48 smithi112 ceph-mon[30003]: pgmap v456: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:55:49.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:48 smithi162 ceph-mon[34800]: pgmap v456: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:55:50.963 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T05:55:51.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:50 smithi150 ceph-mon[34315]: pgmap v457: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-08T05:55:51.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:50 smithi112 ceph-mon[30003]: pgmap v457: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-08T05:55:51.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:50 smithi162 ceph-mon[34800]: pgmap v457: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-08T05:55:51.408 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:55:53.118 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:52 smithi162 ceph-mon[34800]: pgmap v458: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T05:55:53.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:52 smithi150 ceph-mon[34315]: pgmap v458: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T05:55:53.306 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:52 smithi112 ceph-mon[30003]: pgmap v458: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T05:55:53.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.842+0000 7f41bd200700 1 -- 172.21.15.112:0/3319486890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b806bb40 msgr2=0x7f41b8110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:53.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.843+0000 7f41bd200700 1 --2- 172.21.15.112:0/3319486890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b806bb40 0x7f41b8110330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f41b000b0a0 tx=0x7f41b000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.843+0000 7f41bd200700 1 -- 172.21.15.112:0/3319486890 shutdown_connections 2024-05-08T05:55:53.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.843+0000 7f41bd200700 1 --2- 172.21.15.112:0/3319486890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b806bb40 0x7f41b8110330 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.843+0000 7f41bd200700 1 --2- 172.21.15.112:0/3319486890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41b8109510 0x7f41b81098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.843+0000 7f41bd200700 1 --2- 172.21.15.112:0/3319486890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41b806b180 0x7f41b806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.843+0000 7f41bd200700 1 -- 172.21.15.112:0/3319486890 >> 172.21.15.112:0/3319486890 conn(0x7f41b8079350 msgr2=0x7f41b8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:53.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.843+0000 7f41bd200700 1 -- 172.21.15.112:0/3319486890 shutdown_connections 2024-05-08T05:55:53.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.843+0000 7f41bd200700 1 -- 172.21.15.112:0/3319486890 wait complete. 2024-05-08T05:55:53.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.844+0000 7f41bd200700 1 Processor -- start 2024-05-08T05:55:53.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.844+0000 7f41bd200700 1 -- start start 2024-05-08T05:55:53.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.845+0000 7f41bd200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b806b180 0x7f41b819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.845+0000 7f41bd200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41b806bb40 0x7f41b819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.845+0000 7f41bd200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41b8109510 0x7f41b81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.845+0000 7f41bd200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8113690 con 0x7f41b806b180 2024-05-08T05:55:53.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.845+0000 7f41bd200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8113510 con 0x7f41b806bb40 2024-05-08T05:55:53.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.845+0000 7f41bd200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8113810 con 0x7f41b8109510 2024-05-08T05:55:53.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.845+0000 7f41b5e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b806b180 0x7f41b819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.845+0000 7f41b5400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41b806bb40 0x7f41b819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.845+0000 7f41b6800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41b8109510 0x7f41b81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.845+0000 7f41b5400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41b806bb40 0x7f41b819c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:50106/0 (socket says 172.21.15.112:50106) 2024-05-08T05:55:53.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.845+0000 7f41b5e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b806b180 0x7f41b819bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58034/0 (socket says 172.21.15.112:58034) 2024-05-08T05:55:53.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.845+0000 7f41b5400700 1 -- 172.21.15.112:0/3664807527 learned_addr learned my addr 172.21.15.112:0/3664807527 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:53.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.846+0000 7f41b5400700 1 -- 172.21.15.112:0/3664807527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41b8109510 msgr2=0x7f41b81a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:53.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.846+0000 7f41b5400700 1 --2- 172.21.15.112:0/3664807527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41b8109510 0x7f41b81a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.846+0000 7f41b5400700 1 -- 172.21.15.112:0/3664807527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b806b180 msgr2=0x7f41b819bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:53.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.846+0000 7f41b5400700 1 --2- 172.21.15.112:0/3664807527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b806b180 0x7f41b819bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.846+0000 7f41b5400700 1 -- 172.21.15.112:0/3664807527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b81a0d60 con 0x7f41b806bb40 2024-05-08T05:55:53.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.846+0000 7f41b5e00700 1 --2- 172.21.15.112:0/3664807527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b806b180 0x7f41b819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:53.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.846+0000 7f41b6800700 1 --2- 172.21.15.112:0/3664807527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41b8109510 0x7f41b81a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:53.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.846+0000 7f41b5400700 1 --2- 172.21.15.112:0/3664807527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41b806bb40 0x7f41b819c290 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f41a400e9c0 tx=0x7f41a400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:53.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.849+0000 7f41aea00700 1 -- 172.21.15.112:0/3664807527 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a400cbd0 con 0x7f41b806bb40 2024-05-08T05:55:53.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.849+0000 7f41aea00700 1 -- 172.21.15.112:0/3664807527 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a400ee60 con 0x7f41b806bb40 2024-05-08T05:55:53.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.849+0000 7f41bd200700 1 -- 172.21.15.112:0/3664807527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b81a1050 con 0x7f41b806bb40 2024-05-08T05:55:53.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.849+0000 7f41aea00700 1 -- 172.21.15.112:0/3664807527 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40055c0 con 0x7f41b806bb40 2024-05-08T05:55:53.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.849+0000 7f41bd200700 1 -- 172.21.15.112:0/3664807527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b81a88d0 con 0x7f41b806bb40 2024-05-08T05:55:53.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.853+0000 7f41aea00700 1 -- 172.21.15.112:0/3664807527 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f41a4004750 con 0x7f41b806bb40 2024-05-08T05:55:53.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.854+0000 7f41c7c50b80 1 Processor -- start 2024-05-08T05:55:53.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.854+0000 7f41c7c50b80 1 -- start start 2024-05-08T05:55:53.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.854+0000 7f41c7c50b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e47cbc0 0x56321e47cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.854+0000 7f41c7c50b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e4772d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.854+0000 7f41c7c50b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e4778f0 0x56321e477d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.854+0000 7f41c7c50b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56321e3c9a20 con 0x56321e4778f0 2024-05-08T05:55:53.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.854+0000 7f41c7c50b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56321e367760 con 0x56321e47d4e0 2024-05-08T05:55:53.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.854+0000 7f41c7c50b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56321e3c59e0 con 0x56321e47cbc0 2024-05-08T05:55:53.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.855+0000 7f41b6800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e47cbc0 0x56321e47cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.855+0000 7f41b5e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e4772d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.855+0000 7f41b5e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e4772d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:50114/0 (socket says 172.21.15.112:50114) 2024-05-08T05:55:53.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.855+0000 7f41b6800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e47cbc0 0x56321e47cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:33742/0 (socket says 172.21.15.112:33742) 2024-05-08T05:55:53.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.855+0000 7f41b5e00700 1 -- 172.21.15.112:0/440879872 learned_addr learned my addr 172.21.15.112:0/440879872 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:53.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.855+0000 7f41b5400700 1 --2- 172.21.15.112:0/440879872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e4778f0 0x56321e477d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.855+0000 7f41aea00700 1 --2- 172.21.15.112:0/3664807527 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f419c0b2900 0x7f419c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.855+0000 7f41b5e00700 1 -- 172.21.15.112:0/440879872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e47cbc0 msgr2=0x56321e47cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:53.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.855+0000 7f41b5e00700 1 --2- 172.21.15.112:0/440879872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e47cbc0 0x56321e47cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.855+0000 7f41b5e00700 1 -- 172.21.15.112:0/440879872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e4778f0 msgr2=0x56321e477d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.855+0000 7f41b5e00700 1 --2- 172.21.15.112:0/440879872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e4778f0 0x56321e477d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.855+0000 7f41b5e00700 1 -- 172.21.15.112:0/440879872 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321e3168d0 con 0x56321e47d4e0 2024-05-08T05:55:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.855+0000 7f41aea00700 1 -- 172.21.15.112:0/3664807527 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f41a40e24a0 con 0x7f41b806bb40 2024-05-08T05:55:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.856+0000 7f41b6800700 1 --2- 172.21.15.112:0/3664807527 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f419c0b2900 0x7f419c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.856+0000 7f41b5e00700 1 --2- 172.21.15.112:0/440879872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e4772d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f41a80115c0 tx=0x7f41a8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a174e1fd7b74141 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.856+0000 7f41a3400700 1 -- 172.21.15.112:0/440879872 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a801e070 con 0x56321e47d4e0 2024-05-08T05:55:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.856+0000 7f41a3400700 1 -- 172.21.15.112:0/440879872 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a8019ae0 con 0x56321e47d4e0 2024-05-08T05:55:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.856+0000 7f41a3400700 1 -- 172.21.15.112:0/440879872 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a800ebd0 con 0x56321e47d4e0 2024-05-08T05:55:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.857+0000 7f41c7c50b80 1 -- 172.21.15.112:0/440879872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 msgr2=0x56321e4772d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.857+0000 7f41b6800700 1 --2- 172.21.15.112:0/3664807527 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f419c0b2900 0x7f419c0b4dc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f41b000b780 tx=0x7f41b001f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.857+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/440879872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e4772d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f41a80115c0 tx=0x7f41a8009f90 comp rx=0 tx=0).stop 2024-05-08T05:55:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.857+0000 7f41c7c50b80 1 -- 172.21.15.112:0/440879872 shutdown_connections 2024-05-08T05:55:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.857+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/440879872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e4778f0 0x56321e477d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.857+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/440879872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e47cbc0 0x56321e47cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.857+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/440879872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e4772d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.857+0000 7f41c7c50b80 1 -- 172.21.15.112:0/440879872 >> 172.21.15.112:0/440879872 conn(0x56321e3ef6d0 msgr2=0x56321e471b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.857+0000 7f41c7c50b80 1 -- 172.21.15.112:0/440879872 shutdown_connections 2024-05-08T05:55:53.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.857+0000 7f41c7c50b80 1 -- 172.21.15.112:0/440879872 wait complete. 2024-05-08T05:55:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.858+0000 7f41c7c50b80 1 -- start start 2024-05-08T05:55:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.858+0000 7f41c7c50b80 1 Processor -- start 2024-05-08T05:55:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.859+0000 7f41c7c50b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e4778f0 0x56321e483160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.859+0000 7f41c7c50b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e47cbc0 0x56321e480500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.859+0000 7f41c7c50b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e480a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.859+0000 7f41c7c50b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56321e483f20 con 0x56321e47cbc0 2024-05-08T05:55:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.859+0000 7f41c7c50b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56321e32fb20 con 0x56321e47d4e0 2024-05-08T05:55:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.859+0000 7f41c7c50b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56321e4840a0 con 0x56321e4778f0 2024-05-08T05:55:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.859+0000 7f41c7c50b80 10 client.0 _ll_register_callbacks cb 0x56321e370c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.859+0000 7f41b5400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e47cbc0 0x56321e480500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.859+0000 7f41b6800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e480a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.859+0000 7f41b6800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e480a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:50120/0 (socket says 172.21.15.112:50120) 2024-05-08T05:55:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.859+0000 7f41b6800700 1 -- 172.21.15.112:0/4195836023 learned_addr learned my addr 172.21.15.112:0/4195836023 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.859+0000 7f41b5e00700 1 --2- 172.21.15.112:0/4195836023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e4778f0 0x56321e483160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.860+0000 7f41b5400700 1 -- 172.21.15.112:0/4195836023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e4778f0 msgr2=0x56321e483160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.860+0000 7f41b5400700 1 --2- 172.21.15.112:0/4195836023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e4778f0 0x56321e483160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.860+0000 7f41b5400700 1 -- 172.21.15.112:0/4195836023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 msgr2=0x56321e480a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:53.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.860+0000 7f41b5400700 1 --2- 172.21.15.112:0/4195836023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e480a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.860+0000 7f41b5400700 1 -- 172.21.15.112:0/4195836023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321e38f420 con 0x56321e47cbc0 2024-05-08T05:55:53.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.860+0000 7f41b6800700 1 --2- 172.21.15.112:0/4195836023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e480a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:53.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.860+0000 7f41b5400700 1 --2- 172.21.15.112:0/4195836023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e47cbc0 0x56321e480500 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f41a40a6ca0 tx=0x7f41a40a9f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe652f70d0b1dea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:53.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.860+0000 7f4197e00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:55:53.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.860+0000 7f4197e00700 1 -- 172.21.15.112:0/4195836023 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40a9800 con 0x56321e47cbc0 2024-05-08T05:55:53.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.860+0000 7f4197e00700 1 -- 172.21.15.112:0/4195836023 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f41a40cc0c0 con 0x56321e47cbc0 2024-05-08T05:55:53.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.860+0000 7f41c7c50b80 1 -- 172.21.15.112:0/4195836023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a8019ae0 con 0x56321e47cbc0 2024-05-08T05:55:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.861+0000 7f4197e00700 1 -- 172.21.15.112:0/4195836023 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f41a40a51c0 con 0x56321e47cbc0 2024-05-08T05:55:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.861+0000 7f41c7c50b80 10 client.15027 fetch_fsmap learned FSMap version 25 2024-05-08T05:55:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.861+0000 7f41c7c50b80 1 -- 172.21.15.112:0/4195836023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56321e48e750 con 0x56321e47cbc0 2024-05-08T05:55:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.861+0000 7f4197e00700 1 -- 172.21.15.112:0/4195836023 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a8019ae0 con 0x56321e47cbc0 2024-05-08T05:55:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.861+0000 7f4197e00700 1 -- 172.21.15.112:0/4195836023 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f41a40cace0 con 0x56321e47cbc0 2024-05-08T05:55:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.861+0000 7f41c7c50b80 10 client.15027 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:55:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.861+0000 7f41c7c50b80 10 client.15027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:55:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.861+0000 7f41c7c50b80 20 client.15027 populate_metadata read hostname 'smithi112' 2024-05-08T05:55:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.861+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/4195836023 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56321e490d90 0x56321e493180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.861+0000 7f41c7c50b80 4 client.15027 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:55:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.861+0000 7f41c7c50b80 1 -- 172.21.15.112:0/4195836023 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56321e444140 con 0x56321e490d90 2024-05-08T05:55:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.862+0000 7f41b5e00700 1 --2- 172.21.15.112:0/4195836023 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56321e490d90 0x56321e493180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.862+0000 7f4197e00700 10 client.15027 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:55:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.862+0000 7f41b5e00700 1 --2- 172.21.15.112:0/4195836023 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56321e490d90 0x56321e493180 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb559cd77628d44 server_cookie=d66853bd5d086be in_seq=0 out_seq=0 2024-05-08T05:55:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.863+0000 7f4197e00700 1 -- 172.21.15.112:0/4195836023 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56321e444140 con 0x56321e490d90 2024-05-08T05:55:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.863+0000 7f4197e00700 10 client.15027 handle_command_reply: tid=0 2024-05-08T05:55:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.863+0000 7f41c7c50b80 1 client.15027 shutdown 2024-05-08T05:55:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.863+0000 7f41c7c50b80 1 -- 172.21.15.112:0/4195836023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e47cbc0 msgr2=0x56321e480500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.863+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/4195836023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e47cbc0 0x56321e480500 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f41a40a6ca0 tx=0x7f41a40a9f10 comp rx=0 tx=0).stop 2024-05-08T05:55:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.863+0000 7f41c7c50b80 1 -- 172.21.15.112:0/4195836023 shutdown_connections 2024-05-08T05:55:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.863+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/4195836023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e480a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.863+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/4195836023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e4778f0 0x56321e483160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.863+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/4195836023 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56321e490d90 0x56321e493180 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.863+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/4195836023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e47cbc0 0x56321e480500 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.863+0000 7f41c7c50b80 1 -- 172.21.15.112:0/4195836023 >> 172.21.15.112:0/4195836023 conn(0x56321e3ef6d0 msgr2=0x56321e471190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:53.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.863+0000 7f4197e00700 0 client.15027 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:55:53.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.864+0000 7f41c7c50b80 1 -- 172.21.15.112:0/4195836023 shutdown_connections 2024-05-08T05:55:53.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.864+0000 7f41c7c50b80 1 -- 172.21.15.112:0/4195836023 wait complete. 2024-05-08T05:55:53.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.864+0000 7f41c7c50b80 20 client.15027 trim_cache size 0 max 16384 2024-05-08T05:55:53.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.886+0000 7f41c7c50b80 1 Processor -- start 2024-05-08T05:55:53.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.886+0000 7f41c7c50b80 1 -- start start 2024-05-08T05:55:53.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.886+0000 7f41c7c50b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e4778f0 0x56321e47e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.886+0000 7f41c7c50b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e47cbc0 0x56321e48cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.886+0000 7f41c7c50b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e471910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.886+0000 7f41c7c50b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81b26f0 con 0x56321e4778f0 2024-05-08T05:55:53.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.886+0000 7f41c7c50b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81aff80 con 0x56321e47d4e0 2024-05-08T05:55:53.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.886+0000 7f41c7c50b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81addd0 con 0x56321e47cbc0 2024-05-08T05:55:53.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.887+0000 7f41b5e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e4778f0 0x56321e47e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.887+0000 7f41b5400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e47cbc0 0x56321e48cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.887+0000 7f41b5e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e4778f0 0x56321e47e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58062/0 (socket says 172.21.15.112:58062) 2024-05-08T05:55:53.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.887+0000 7f41b5e00700 1 -- 172.21.15.112:0/2033144523 learned_addr learned my addr 172.21.15.112:0/2033144523 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.887+0000 7f41b6800700 1 --2- 172.21.15.112:0/2033144523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e471910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.887+0000 7f41b5400700 1 -- 172.21.15.112:0/2033144523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 msgr2=0x56321e471910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.887+0000 7f41b5400700 1 --2- 172.21.15.112:0/2033144523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e471910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.887+0000 7f41b5400700 1 -- 172.21.15.112:0/2033144523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e4778f0 msgr2=0x56321e47e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.887+0000 7f41b5400700 1 --2- 172.21.15.112:0/2033144523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e4778f0 0x56321e47e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.887+0000 7f41b5400700 1 -- 172.21.15.112:0/2033144523 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321e477610 con 0x56321e47cbc0 2024-05-08T05:55:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.887+0000 7f41b5e00700 1 --2- 172.21.15.112:0/2033144523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e4778f0 0x56321e47e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.888+0000 7f41b5400700 1 --2- 172.21.15.112:0/2033144523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e47cbc0 0x56321e48cfe0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009860 tx=0x7f41a400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba2028d642345d9d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.888+0000 7f41a2a00700 1 -- 172.21.15.112:0/2033144523 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40a91a0 con 0x56321e47cbc0 2024-05-08T05:55:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.888+0000 7f41a2a00700 1 -- 172.21.15.112:0/2033144523 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a40c9070 con 0x56321e47cbc0 2024-05-08T05:55:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.888+0000 7f41a2a00700 1 -- 172.21.15.112:0/2033144523 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40a9380 con 0x56321e47cbc0 2024-05-08T05:55:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.889+0000 7f41c7c50b80 1 -- 172.21.15.112:0/2033144523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e47cbc0 msgr2=0x56321e48cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.889+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/2033144523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e47cbc0 0x56321e48cfe0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009860 tx=0x7f41a400ce30 comp rx=0 tx=0).stop 2024-05-08T05:55:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.889+0000 7f41c7c50b80 1 -- 172.21.15.112:0/2033144523 shutdown_connections 2024-05-08T05:55:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.889+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/2033144523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47d4e0 0x56321e471910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.889+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/2033144523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e4778f0 0x56321e47e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.889+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/2033144523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e47cbc0 0x56321e48cfe0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.889+0000 7f41c7c50b80 1 -- 172.21.15.112:0/2033144523 >> 172.21.15.112:0/2033144523 conn(0x56321e3ef6d0 msgr2=0x56321e481bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.889+0000 7f41c7c50b80 1 -- 172.21.15.112:0/2033144523 shutdown_connections 2024-05-08T05:55:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.889+0000 7f41c7c50b80 1 -- 172.21.15.112:0/2033144523 wait complete. 2024-05-08T05:55:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.889+0000 7f41c7c50b80 1 -- start start 2024-05-08T05:55:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.890+0000 7f41c7c50b80 1 Processor -- start 2024-05-08T05:55:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.890+0000 7f41c7c50b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e4778f0 0x56321e498410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.890+0000 7f41c7c50b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47cbc0 0x56321e498950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.890+0000 7f41c7c50b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e47d4e0 0x56321e4fa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.890+0000 7f41c7c50b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81ae440 con 0x56321e47d4e0 2024-05-08T05:55:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.890+0000 7f41c7c50b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81af8b0 con 0x56321e47cbc0 2024-05-08T05:55:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.890+0000 7f41c7c50b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81aeb10 con 0x56321e4778f0 2024-05-08T05:55:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.890+0000 7f41c7c50b80 10 client.0 _ll_register_callbacks cb 0x56321e370c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.890+0000 7f41b6800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e47d4e0 0x56321e4fa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.890+0000 7f41b5e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e4778f0 0x56321e498410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.890+0000 7f41b5e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e4778f0 0x56321e498410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:33768/0 (socket says 172.21.15.112:33768) 2024-05-08T05:55:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.890+0000 7f41b5e00700 1 -- 172.21.15.112:0/126322727 learned_addr learned my addr 172.21.15.112:0/126322727 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.890+0000 7f41b6800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e47d4e0 0x56321e4fa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58074/0 (socket says 172.21.15.112:58074) 2024-05-08T05:55:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.891+0000 7f41b5400700 1 --2- 172.21.15.112:0/126322727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47cbc0 0x56321e498950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.891+0000 7f41b5e00700 1 -- 172.21.15.112:0/126322727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47cbc0 msgr2=0x56321e498950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.891+0000 7f41b5e00700 1 --2- 172.21.15.112:0/126322727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47cbc0 0x56321e498950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.891+0000 7f41b5e00700 1 -- 172.21.15.112:0/126322727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e47d4e0 msgr2=0x56321e4fa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.891+0000 7f41b5e00700 1 --2- 172.21.15.112:0/126322727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e47d4e0 0x56321e4fa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.891+0000 7f41b5e00700 1 -- 172.21.15.112:0/126322727 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321e499220 con 0x56321e4778f0 2024-05-08T05:55:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.891+0000 7f41b5e00700 1 --2- 172.21.15.112:0/126322727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e4778f0 0x56321e498410 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f41a8000c00 tx=0x7f41a800e6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bd426495aec7502 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.891+0000 7f4197e00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:55:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.892+0000 7f4197e00700 1 -- 172.21.15.112:0/126322727 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a802f070 con 0x56321e4778f0 2024-05-08T05:55:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.892+0000 7f4197e00700 1 -- 172.21.15.112:0/126322727 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f41a80054c0 con 0x56321e4778f0 2024-05-08T05:55:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.892+0000 7f41c7c50b80 1 -- 172.21.15.112:0/126322727 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a40c9070 con 0x56321e4778f0 2024-05-08T05:55:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.892+0000 7f4197e00700 1 -- 172.21.15.112:0/126322727 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f41a803b050 con 0x56321e4778f0 2024-05-08T05:55:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.892+0000 7f41c7c50b80 10 client.25162 fetch_fsmap learned FSMap version 25 2024-05-08T05:55:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.892+0000 7f41c7c50b80 1 -- 172.21.15.112:0/126322727 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56321e4fbac0 con 0x56321e4778f0 2024-05-08T05:55:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.892+0000 7f4197e00700 1 -- 172.21.15.112:0/126322727 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a40c9070 con 0x56321e4778f0 2024-05-08T05:55:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.892+0000 7f4197e00700 1 -- 172.21.15.112:0/126322727 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f41a8003680 con 0x56321e4778f0 2024-05-08T05:55:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.893+0000 7f41c7c50b80 10 client.25162 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:55:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.893+0000 7f41c7c50b80 10 client.25162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:55:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.893+0000 7f41c7c50b80 20 client.25162 populate_metadata read hostname 'smithi112' 2024-05-08T05:55:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.893+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/126322727 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56321e4fbf50 0x56321e4fe370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.893+0000 7f41c7c50b80 4 client.25162 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:55:53.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.893+0000 7f41c7c50b80 1 -- 172.21.15.112:0/126322727 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56321e3d7400 con 0x56321e4fbf50 2024-05-08T05:55:53.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.893+0000 7f41b5400700 1 --2- 172.21.15.112:0/126322727 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56321e4fbf50 0x56321e4fe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:53.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.893+0000 7f41b5400700 1 --2- 172.21.15.112:0/126322727 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56321e4fbf50 0x56321e4fe370 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc9fc46ec7b935f server_cookie=35cb34df88cc6195 in_seq=0 out_seq=0 2024-05-08T05:55:53.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.893+0000 7f4197e00700 10 client.25162 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:55:53.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.894+0000 7f4197e00700 1 -- 172.21.15.112:0/126322727 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56321e3d7400 con 0x56321e4fbf50 2024-05-08T05:55:53.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.894+0000 7f4197e00700 10 client.25162 handle_command_reply: tid=0 2024-05-08T05:55:53.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.894+0000 7f41c7c50b80 1 client.25162 shutdown 2024-05-08T05:55:53.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.894+0000 7f41c7c50b80 1 -- 172.21.15.112:0/126322727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e4778f0 msgr2=0x56321e498410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:53.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.894+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/126322727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e4778f0 0x56321e498410 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f41a8000c00 tx=0x7f41a800e6b0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.894+0000 7f41c7c50b80 1 -- 172.21.15.112:0/126322727 shutdown_connections 2024-05-08T05:55:53.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.894+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/126322727 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56321e4fbf50 0x56321e4fe370 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.894+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/126322727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56321e47d4e0 0x56321e4fa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.894+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/126322727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56321e4778f0 0x56321e498410 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.894+0000 7f41c7c50b80 1 --2- 172.21.15.112:0/126322727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321e47cbc0 0x56321e498950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.894+0000 7f41c7c50b80 1 -- 172.21.15.112:0/126322727 >> 172.21.15.112:0/126322727 conn(0x56321e3ef6d0 msgr2=0x56321e47bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:53.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.894+0000 7f4197e00700 0 client.25162 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:55:53.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.894+0000 7f41c7c50b80 1 -- 172.21.15.112:0/126322727 shutdown_connections 2024-05-08T05:55:53.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.894+0000 7f41c7c50b80 1 -- 172.21.15.112:0/126322727 wait complete. 2024-05-08T05:55:53.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.894+0000 7f41c7c50b80 20 client.25162 trim_cache size 0 max 16384 2024-05-08T05:55:53.902 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:55:53.902 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-08T05:55:53.903 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-08T05:55:53.903 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:55:53.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.895+0000 7f41bd200700 1 -- 172.21.15.112:0/3664807527 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f419c0b2900 msgr2=0x7f419c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:53.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.895+0000 7f41bd200700 1 --2- 172.21.15.112:0/3664807527 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f419c0b2900 0x7f419c0b4dc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f41b000b780 tx=0x7f41b001f040 comp rx=0 tx=0).stop 2024-05-08T05:55:53.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.895+0000 7f41bd200700 1 -- 172.21.15.112:0/3664807527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41b806bb40 msgr2=0x7f41b819c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:53.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.895+0000 7f41bd200700 1 --2- 172.21.15.112:0/3664807527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41b806bb40 0x7f41b819c290 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f41a400e9c0 tx=0x7f41a400ecd0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.895+0000 7f41bd200700 1 -- 172.21.15.112:0/3664807527 shutdown_connections 2024-05-08T05:55:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.895+0000 7f41bd200700 1 --2- 172.21.15.112:0/3664807527 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f419c0b2900 0x7f419c0b4dc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.895+0000 7f41bd200700 1 --2- 172.21.15.112:0/3664807527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41b8109510 0x7f41b81a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.895+0000 7f41bd200700 1 --2- 172.21.15.112:0/3664807527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41b806b180 0x7f41b819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.895+0000 7f41bd200700 1 --2- 172.21.15.112:0/3664807527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41b806bb40 0x7f41b819c290 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.895+0000 7f41bd200700 1 -- 172.21.15.112:0/3664807527 >> 172.21.15.112:0/3664807527 conn(0x7f41b8079350 msgr2=0x7f41b810e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.896+0000 7f41bd200700 1 -- 172.21.15.112:0/3664807527 shutdown_connections 2024-05-08T05:55:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:53.896+0000 7f41bd200700 1 -- 172.21.15.112:0/3664807527 wait complete. 2024-05-08T05:55:54.853 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:55:54.854 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:55:55.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:54 smithi112 ceph-mon[30003]: pgmap v459: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T05:55:55.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:54 smithi150 ceph-mon[34315]: pgmap v459: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T05:55:55.294 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:55:55.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:54 smithi162 ceph-mon[34800]: pgmap v459: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T05:55:57.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:56 smithi112 ceph-mon[30003]: pgmap v460: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-08T05:55:57.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:56 smithi150 ceph-mon[34315]: pgmap v460: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-08T05:55:57.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:56 smithi162 ceph-mon[34800]: pgmap v460: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-08T05:55:57.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.604+0000 7f2f95e00700 1 -- 172.21.15.112:0/130117530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9010b680 msgr2=0x7f2f9010ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:57.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.604+0000 7f2f95e00700 1 --2- 172.21.15.112:0/130117530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9010b680 0x7f2f9010ba60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2f840099c0 tx=0x7f2f84009cd0 comp rx=0 tx=0).stop 2024-05-08T05:55:57.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.604+0000 7f2f95e00700 1 -- 172.21.15.112:0/130117530 shutdown_connections 2024-05-08T05:55:57.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.604+0000 7f2f95e00700 1 --2- 172.21.15.112:0/130117530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f90105f90 0x7f2f90112450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:57.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.604+0000 7f2f95e00700 1 --2- 172.21.15.112:0/130117530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9010b680 0x7f2f9010ba60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:57.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.604+0000 7f2f95e00700 1 --2- 172.21.15.112:0/130117530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f901055d0 0x7f2f90105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:57.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.605+0000 7f2f95e00700 1 -- 172.21.15.112:0/130117530 >> 172.21.15.112:0/130117530 conn(0x7f2f90101080 msgr2=0x7f2f901034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:57.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.605+0000 7f2f95e00700 1 -- 172.21.15.112:0/130117530 shutdown_connections 2024-05-08T05:55:57.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.605+0000 7f2f95e00700 1 -- 172.21.15.112:0/130117530 wait complete. 2024-05-08T05:55:57.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.606+0000 7f2f95e00700 1 Processor -- start 2024-05-08T05:55:57.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.607+0000 7f2f95e00700 1 -- start start 2024-05-08T05:55:57.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.607+0000 7f2f95e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f901055d0 0x7f2f901a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:57.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.607+0000 7f2f95e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f90105f90 0x7f2f901a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:57.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.607+0000 7f2f95e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9010b680 0x7f2f9019bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:57.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.607+0000 7f2f95e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901157d0 con 0x7f2f9010b680 2024-05-08T05:55:57.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.607+0000 7f2f95e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90115650 con 0x7f2f901055d0 2024-05-08T05:55:57.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.607+0000 7f2f95e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90115950 con 0x7f2f90105f90 2024-05-08T05:55:57.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.607+0000 7f2f8fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f90105f90 0x7f2f901a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:57.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.607+0000 7f2f8fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f90105f90 0x7f2f901a1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:33784/0 (socket says 172.21.15.112:33784) 2024-05-08T05:55:57.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.607+0000 7f2f8fe00700 1 -- 172.21.15.112:0/3581275294 learned_addr learned my addr 172.21.15.112:0/3581275294 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:55:57.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.607+0000 7f2f94a00700 1 --2- 172.21.15.112:0/3581275294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f901055d0 0x7f2f901a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:57.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.608+0000 7f2f94a00700 1 -- 172.21.15.112:0/3581275294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f90105f90 msgr2=0x7f2f901a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:57.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.608+0000 7f2f94a00700 1 --2- 172.21.15.112:0/3581275294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f90105f90 0x7f2f901a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:57.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.608+0000 7f2f94a00700 1 -- 172.21.15.112:0/3581275294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9010b680 msgr2=0x7f2f9019bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:57.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.608+0000 7f2f94a00700 1 --2- 172.21.15.112:0/3581275294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9010b680 0x7f2f9019bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:57.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.608+0000 7f2f94a00700 1 -- 172.21.15.112:0/3581275294 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9019c210 con 0x7f2f901055d0 2024-05-08T05:55:57.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.608+0000 7f2f94a00700 1 --2- 172.21.15.112:0/3581275294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f901055d0 0x7f2f901a18a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2f840098e0 tx=0x7f2f84004690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:57.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.609+0000 7f2f8d600700 1 -- 172.21.15.112:0/3581275294 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f84009e60 con 0x7f2f901055d0 2024-05-08T05:55:57.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.609+0000 7f2f95e00700 1 -- 172.21.15.112:0/3581275294 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9019c4a0 con 0x7f2f901055d0 2024-05-08T05:55:57.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.609+0000 7f2f8d600700 1 -- 172.21.15.112:0/3581275294 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f840047f0 con 0x7f2f901055d0 2024-05-08T05:55:57.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.609+0000 7f2f95e00700 1 -- 172.21.15.112:0/3581275294 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f901a8770 con 0x7f2f901055d0 2024-05-08T05:55:57.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.609+0000 7f2f8d600700 1 -- 172.21.15.112:0/3581275294 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f84005290 con 0x7f2f901055d0 2024-05-08T05:55:57.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.610+0000 7f2f95e00700 1 -- 172.21.15.112:0/3581275294 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f9010db10 con 0x7f2f901055d0 2024-05-08T05:55:57.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.612+0000 7f2f8d600700 1 -- 172.21.15.112:0/3581275294 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2f84028060 con 0x7f2f901055d0 2024-05-08T05:55:57.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.614+0000 7f2f8d600700 1 --2- 172.21.15.112:0/3581275294 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2f780b2900 0x7f2f780b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:57.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.615+0000 7f2f8d600700 1 -- 172.21.15.112:0/3581275294 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f2f840e2b20 con 0x7f2f901055d0 2024-05-08T05:55:57.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.615+0000 7f2f8fe00700 1 --2- 172.21.15.112:0/3581275294 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2f780b2900 0x7f2f780b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:57.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.616+0000 7f2f8fe00700 1 --2- 172.21.15.112:0/3581275294 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2f780b2900 0x7f2f780b4dc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2f9010ad30 tx=0x7f2f7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:57.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.616+0000 7f2f8d600700 1 -- 172.21.15.112:0/3581275294 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f84013070 con 0x7f2f901055d0 2024-05-08T05:55:57.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.908+0000 7f2f95e00700 1 -- 172.21.15.112:0/3581275294 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f9006a190 con 0x7f2f901055d0 2024-05-08T05:55:57.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.909+0000 7f2f8d600700 1 -- 172.21.15.112:0/3581275294 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11891 (secure 0 0 0) 0x7f2f9006a190 con 0x7f2f901055d0 2024-05-08T05:55:57.909 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:55:57.909 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T05:54:58.248303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24518,24511]},"id":1}]} 2024-05-08T05:55:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.912+0000 7f2f95e00700 1 -- 172.21.15.112:0/3581275294 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2f780b2900 msgr2=0x7f2f780b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.912+0000 7f2f95e00700 1 --2- 172.21.15.112:0/3581275294 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2f780b2900 0x7f2f780b4dc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2f9010ad30 tx=0x7f2f7c009260 comp rx=0 tx=0).stop 2024-05-08T05:55:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.912+0000 7f2f95e00700 1 -- 172.21.15.112:0/3581275294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f901055d0 msgr2=0x7f2f901a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.912+0000 7f2f95e00700 1 --2- 172.21.15.112:0/3581275294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f901055d0 0x7f2f901a18a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2f840098e0 tx=0x7f2f84004690 comp rx=0 tx=0).stop 2024-05-08T05:55:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.912+0000 7f2f95e00700 1 -- 172.21.15.112:0/3581275294 shutdown_connections 2024-05-08T05:55:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.912+0000 7f2f95e00700 1 --2- 172.21.15.112:0/3581275294 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2f780b2900 0x7f2f780b4dc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.912+0000 7f2f95e00700 1 --2- 172.21.15.112:0/3581275294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f9010b680 0x7f2f9019bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.912+0000 7f2f95e00700 1 --2- 172.21.15.112:0/3581275294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f901055d0 0x7f2f901a18a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.912+0000 7f2f95e00700 1 --2- 172.21.15.112:0/3581275294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f90105f90 0x7f2f901a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.912+0000 7f2f95e00700 1 -- 172.21.15.112:0/3581275294 >> 172.21.15.112:0/3581275294 conn(0x7f2f90101080 msgr2=0x7f2f90102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.913+0000 7f2f95e00700 1 -- 172.21.15.112:0/3581275294 shutdown_connections 2024-05-08T05:55:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:55:57.913+0000 7f2f95e00700 1 -- 172.21.15.112:0/3581275294 wait complete. 2024-05-08T05:55:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-05-08T05:55:58.903 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-08T05:55:59.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:58 smithi150 ceph-mon[34315]: pgmap v461: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T05:55:59.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:55:58 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3581275294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:59.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:58 smithi112 ceph-mon[30003]: pgmap v461: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T05:55:59.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:55:58 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3581275294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:59.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:58 smithi162 ceph-mon[34800]: pgmap v461: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T05:55:59.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:55:58 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3581275294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:59.340 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:01.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:00 smithi112 ceph-mon[30003]: pgmap v462: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-08T05:56:01.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:00 smithi150 ceph-mon[34315]: pgmap v462: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-08T05:56:01.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:00 smithi162 ceph-mon[34800]: pgmap v462: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-08T05:56:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.652+0000 7f9603200700 1 -- 172.21.15.112:0/2907145569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96041033d0 msgr2=0x7f9604103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.652+0000 7f9603200700 1 --2- 172.21.15.112:0/2907145569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96041033d0 0x7f9604103850 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009990 tx=0x7f95f0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.653+0000 7f9603200700 1 -- 172.21.15.112:0/2907145569 shutdown_connections 2024-05-08T05:56:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.653+0000 7f9603200700 1 --2- 172.21.15.112:0/2907145569 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9604103d90 0x7f96041102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.653+0000 7f9603200700 1 --2- 172.21.15.112:0/2907145569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9604109460 0x7f9604109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.653+0000 7f9603200700 1 --2- 172.21.15.112:0/2907145569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96041033d0 0x7f9604103850 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.653+0000 7f9603200700 1 -- 172.21.15.112:0/2907145569 >> 172.21.15.112:0/2907145569 conn(0x7f96040fef20 msgr2=0x7f9604101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.653+0000 7f9603200700 1 -- 172.21.15.112:0/2907145569 shutdown_connections 2024-05-08T05:56:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.653+0000 7f9603200700 1 -- 172.21.15.112:0/2907145569 wait complete. 2024-05-08T05:56:01.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.654+0000 7f9603200700 1 Processor -- start 2024-05-08T05:56:01.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.654+0000 7f9603200700 1 -- start start 2024-05-08T05:56:01.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.654+0000 7f9603200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96041033d0 0x7f96041a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.654+0000 7f9603200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9604103d90 0x7f96041a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.654+0000 7f9603200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9604109460 0x7f960419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.654+0000 7f9603200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96041135f0 con 0x7f9604109460 2024-05-08T05:56:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.654+0000 7f9603200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9604113470 con 0x7f9604103d90 2024-05-08T05:56:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.654+0000 7f9603200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9604113770 con 0x7f96041033d0 2024-05-08T05:56:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.655+0000 7f9601400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9604103d90 0x7f96041a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.655+0000 7f9601400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9604103d90 0x7f96041a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:50186/0 (socket says 172.21.15.112:50186) 2024-05-08T05:56:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.655+0000 7f9601400700 1 -- 172.21.15.112:0/2239548311 learned_addr learned my addr 172.21.15.112:0/2239548311 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.655+0000 7f9601e00700 1 --2- 172.21.15.112:0/2239548311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96041033d0 0x7f96041a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.655+0000 7f9602800700 1 --2- 172.21.15.112:0/2239548311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9604109460 0x7f960419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.655+0000 7f9601400700 1 -- 172.21.15.112:0/2239548311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96041033d0 msgr2=0x7f96041a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.655+0000 7f9601400700 1 --2- 172.21.15.112:0/2239548311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96041033d0 0x7f96041a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.655+0000 7f9601400700 1 -- 172.21.15.112:0/2239548311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9604109460 msgr2=0x7f960419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.655+0000 7f9601400700 1 --2- 172.21.15.112:0/2239548311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9604109460 0x7f960419baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.655+0000 7f9601400700 1 -- 172.21.15.112:0/2239548311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960419c250 con 0x7f9604103d90 2024-05-08T05:56:01.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.655+0000 7f9601e00700 1 --2- 172.21.15.112:0/2239548311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96041033d0 0x7f96041a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:01.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.656+0000 7f9601400700 1 --2- 172.21.15.112:0/2239548311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9604103d90 0x7f96041a1e70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f95f0005910 tx=0x7f95f000b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:01.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.656+0000 7f95faa00700 1 -- 172.21.15.112:0/2239548311 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0027070 con 0x7f9604103d90 2024-05-08T05:56:01.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.656+0000 7f95faa00700 1 -- 172.21.15.112:0/2239548311 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95f0022470 con 0x7f9604103d90 2024-05-08T05:56:01.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.656+0000 7f95faa00700 1 -- 172.21.15.112:0/2239548311 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0017650 con 0x7f9604103d90 2024-05-08T05:56:01.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.656+0000 7f9603200700 1 -- 172.21.15.112:0/2239548311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960419c4e0 con 0x7f9604103d90 2024-05-08T05:56:01.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.656+0000 7f9603200700 1 -- 172.21.15.112:0/2239548311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96041a8800 con 0x7f9604103d90 2024-05-08T05:56:01.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.662+0000 7f95faa00700 1 -- 172.21.15.112:0/2239548311 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f95f002a070 con 0x7f9604103d90 2024-05-08T05:56:01.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.662+0000 7f961330cb80 1 Processor -- start 2024-05-08T05:56:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.662+0000 7f961330cb80 1 -- start start 2024-05-08T05:56:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.662+0000 7f961330cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a5250fda0 0x558a52510180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.663+0000 7f961330cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a525106c0 0x558a5250a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.663+0000 7f961330cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250aa50 0x558a5250aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.663+0000 7f961330cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558a5245ca20 con 0x558a5250fda0 2024-05-08T05:56:01.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.663+0000 7f961330cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558a523fa760 con 0x558a5250aa50 2024-05-08T05:56:01.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.663+0000 7f961330cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x558a524589e0 con 0x558a525106c0 2024-05-08T05:56:01.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.663+0000 7f9601e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a525106c0 0x558a5250a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.663+0000 7f9601e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a525106c0 0x558a5250a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:33802/0 (socket says 172.21.15.112:33802) 2024-05-08T05:56:01.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.663+0000 7f9601e00700 1 -- 172.21.15.112:0/2532285819 learned_addr learned my addr 172.21.15.112:0/2532285819 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:01.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.663+0000 7f9602800700 1 --2- 172.21.15.112:0/2532285819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a5250fda0 0x558a52510180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.663+0000 7f9601400700 1 --2- 172.21.15.112:0/2532285819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250aa50 0x558a5250aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.664+0000 7f9602800700 1 -- 172.21.15.112:0/2532285819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a525106c0 msgr2=0x558a5250a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.664+0000 7f9602800700 1 --2- 172.21.15.112:0/2532285819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a525106c0 0x558a5250a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.664+0000 7f9602800700 1 -- 172.21.15.112:0/2532285819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250aa50 msgr2=0x558a5250aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.664+0000 7f9602800700 1 --2- 172.21.15.112:0/2532285819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250aa50 0x558a5250aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.664+0000 7f9602800700 1 -- 172.21.15.112:0/2532285819 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a523a98d0 con 0x558a5250fda0 2024-05-08T05:56:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.664+0000 7f95faa00700 1 --2- 172.21.15.112:0/2239548311 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f95e80b2900 0x7f95e80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.664+0000 7f95faa00700 1 -- 172.21.15.112:0/2239548311 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f95f00e2ed0 con 0x7f9604103d90 2024-05-08T05:56:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.664+0000 7f9601e00700 1 --2- 172.21.15.112:0/2239548311 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f95e80b2900 0x7f95e80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.664+0000 7f9602800700 1 --2- 172.21.15.112:0/2532285819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a5250fda0 0x558a52510180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009310 tx=0x7f95fc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dc283c283e0e21 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.665+0000 7f95ef400700 1 -- 172.21.15.112:0/2532285819 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc019020 con 0x558a5250fda0 2024-05-08T05:56:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.665+0000 7f95ef400700 1 -- 172.21.15.112:0/2532285819 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95fc015af0 con 0x558a5250fda0 2024-05-08T05:56:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.665+0000 7f95ef400700 1 -- 172.21.15.112:0/2532285819 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc014a80 con 0x558a5250fda0 2024-05-08T05:56:01.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.666+0000 7f961330cb80 1 -- 172.21.15.112:0/2532285819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a5250fda0 msgr2=0x558a52510180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.666+0000 7f9601e00700 1 --2- 172.21.15.112:0/2239548311 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f95e80b2900 0x7f95e80b4dc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009fd0 tx=0x7f95f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:01.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.666+0000 7f961330cb80 1 --2- 172.21.15.112:0/2532285819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a5250fda0 0x558a52510180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009310 tx=0x7f95fc009920 comp rx=0 tx=0).stop 2024-05-08T05:56:01.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.666+0000 7f961330cb80 1 -- 172.21.15.112:0/2532285819 shutdown_connections 2024-05-08T05:56:01.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.666+0000 7f961330cb80 1 --2- 172.21.15.112:0/2532285819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250aa50 0x558a5250aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.666+0000 7f961330cb80 1 --2- 172.21.15.112:0/2532285819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a5250fda0 0x558a52510180 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.666+0000 7f961330cb80 1 --2- 172.21.15.112:0/2532285819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a525106c0 0x558a5250a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.666+0000 7f961330cb80 1 -- 172.21.15.112:0/2532285819 >> 172.21.15.112:0/2532285819 conn(0x558a524826d0 msgr2=0x558a52504ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:01.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.666+0000 7f961330cb80 1 -- 172.21.15.112:0/2532285819 shutdown_connections 2024-05-08T05:56:01.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.666+0000 7f961330cb80 1 -- 172.21.15.112:0/2532285819 wait complete. 2024-05-08T05:56:01.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.666+0000 7f961330cb80 1 -- start start 2024-05-08T05:56:01.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.667+0000 7f961330cb80 1 Processor -- start 2024-05-08T05:56:01.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.667+0000 7f961330cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250aa50 0x558a52516450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.667+0000 7f961330cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250fda0 0x558a525137f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.667+0000 7f961330cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 0x558a52513d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.667+0000 7f961330cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558a52517150 con 0x558a525106c0 2024-05-08T05:56:01.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.667+0000 7f961330cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558a523c2b20 con 0x558a5250fda0 2024-05-08T05:56:01.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.667+0000 7f961330cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x558a525172d0 con 0x558a5250aa50 2024-05-08T05:56:01.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.667+0000 7f961330cb80 10 client.0 _ll_register_callbacks cb 0x558a52403c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:01.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.668+0000 7f9601e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250aa50 0x558a52516450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.668+0000 7f9601e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250aa50 0x558a52516450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:33806/0 (socket says 172.21.15.112:33806) 2024-05-08T05:56:01.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.668+0000 7f9601e00700 1 -- 172.21.15.112:0/3282086597 learned_addr learned my addr 172.21.15.112:0/3282086597 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:01.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.668+0000 7f9602800700 1 --2- 172.21.15.112:0/3282086597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 0x558a52513d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.668+0000 7f9601400700 1 --2- 172.21.15.112:0/3282086597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250fda0 0x558a525137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.668+0000 7f9601e00700 1 -- 172.21.15.112:0/3282086597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250fda0 msgr2=0x558a525137f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.668+0000 7f9601e00700 1 --2- 172.21.15.112:0/3282086597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250fda0 0x558a525137f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.668+0000 7f9601e00700 1 -- 172.21.15.112:0/3282086597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 msgr2=0x558a52513d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.668+0000 7f9601e00700 1 --2- 172.21.15.112:0/3282086597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 0x558a52513d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.668+0000 7f9601e00700 1 -- 172.21.15.112:0/3282086597 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a52422420 con 0x558a5250aa50 2024-05-08T05:56:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.668+0000 7f9602800700 1 --2- 172.21.15.112:0/3282086597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 0x558a52513d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.669+0000 7f9601e00700 1 --2- 172.21.15.112:0/3282086597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250aa50 0x558a52516450 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f95f400f6b0 tx=0x7f95f40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57cef99a4b8f96d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.669+0000 7f95e3e00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:56:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.669+0000 7f95e3e00700 1 -- 172.21.15.112:0/3282086597 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f400fe80 con 0x558a5250aa50 2024-05-08T05:56:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.669+0000 7f95e3e00700 1 -- 172.21.15.112:0/3282086597 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f95f4004260 con 0x558a5250aa50 2024-05-08T05:56:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.669+0000 7f961330cb80 1 -- 172.21.15.112:0/3282086597 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95fc015af0 con 0x558a5250aa50 2024-05-08T05:56:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.669+0000 7f95e3e00700 1 -- 172.21.15.112:0/3282086597 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f95f402e030 con 0x558a5250aa50 2024-05-08T05:56:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.669+0000 7f961330cb80 10 client.25180 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:01.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.669+0000 7f961330cb80 1 -- 172.21.15.112:0/3282086597 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558a52521ab0 con 0x558a5250aa50 2024-05-08T05:56:01.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.670+0000 7f95e3e00700 1 -- 172.21.15.112:0/3282086597 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95fc015af0 con 0x558a5250aa50 2024-05-08T05:56:01.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.670+0000 7f95e3e00700 1 -- 172.21.15.112:0/3282086597 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f95f4026020 con 0x558a5250aa50 2024-05-08T05:56:01.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.670+0000 7f961330cb80 10 client.25180 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:01.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.670+0000 7f961330cb80 10 client.25180 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T05:56:01.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.670+0000 7f961330cb80 20 client.25180 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:01.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.670+0000 7f961330cb80 1 --2- 172.21.15.112:0/3282086597 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x558a525240f0 0x558a525264e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.670+0000 7f961330cb80 4 client.25180 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.670+0000 7f961330cb80 1 -- 172.21.15.112:0/3282086597 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a524d7140 con 0x558a525240f0 2024-05-08T05:56:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.670+0000 7f9601400700 1 --2- 172.21.15.112:0/3282086597 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x558a525240f0 0x558a525264e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.671+0000 7f9601400700 1 --2- 172.21.15.112:0/3282086597 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x558a525240f0 0x558a525264e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9278e071b1cd5718 server_cookie=2a154e08111754c7 in_seq=0 out_seq=0 2024-05-08T05:56:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.671+0000 7f95e3e00700 10 client.25180 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:56:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.672+0000 7f95e3e00700 1 -- 172.21.15.112:0/3282086597 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a524d7140 con 0x558a525240f0 2024-05-08T05:56:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.672+0000 7f95e3e00700 10 client.25180 handle_command_reply: tid=0 2024-05-08T05:56:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.672+0000 7f961330cb80 1 client.25180 shutdown 2024-05-08T05:56:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.672+0000 7f961330cb80 1 -- 172.21.15.112:0/3282086597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250aa50 msgr2=0x558a52516450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.672+0000 7f961330cb80 1 --2- 172.21.15.112:0/3282086597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250aa50 0x558a52516450 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f95f400f6b0 tx=0x7f95f40047c0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.672+0000 7f961330cb80 1 -- 172.21.15.112:0/3282086597 shutdown_connections 2024-05-08T05:56:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.672+0000 7f961330cb80 1 --2- 172.21.15.112:0/3282086597 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x558a525240f0 0x558a525264e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.672+0000 7f961330cb80 1 --2- 172.21.15.112:0/3282086597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 0x558a52513d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.672+0000 7f961330cb80 1 --2- 172.21.15.112:0/3282086597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250aa50 0x558a52516450 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.672+0000 7f961330cb80 1 --2- 172.21.15.112:0/3282086597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250fda0 0x558a525137f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.672+0000 7f961330cb80 1 -- 172.21.15.112:0/3282086597 >> 172.21.15.112:0/3282086597 conn(0x558a524826d0 msgr2=0x558a524ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:01.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.672+0000 7f95e3e00700 0 client.25180 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:56:01.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.672+0000 7f961330cb80 1 -- 172.21.15.112:0/3282086597 shutdown_connections 2024-05-08T05:56:01.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.673+0000 7f961330cb80 1 -- 172.21.15.112:0/3282086597 wait complete. 2024-05-08T05:56:01.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.673+0000 7f961330cb80 20 client.25180 trim_cache size 0 max 16384 2024-05-08T05:56:01.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.690+0000 7f961330cb80 1 Processor -- start 2024-05-08T05:56:01.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.690+0000 7f961330cb80 1 -- start start 2024-05-08T05:56:01.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.690+0000 7f961330cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250aa50 0x558a52513d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.690+0000 7f961330cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250fda0 0x558a52520210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.690+0000 7f961330cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 0x558a524ff860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.690+0000 7f961330cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96041b2480 con 0x558a525106c0 2024-05-08T05:56:01.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.690+0000 7f961330cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96041afd10 con 0x558a5250aa50 2024-05-08T05:56:01.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.690+0000 7f961330cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f96041adb60 con 0x558a5250fda0 2024-05-08T05:56:01.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.690+0000 7f9602800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 0x558a524ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.690+0000 7f9601e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250aa50 0x558a52513d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.691+0000 7f9601e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250aa50 0x558a52513d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:50216/0 (socket says 172.21.15.112:50216) 2024-05-08T05:56:01.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.691+0000 7f9601e00700 1 -- 172.21.15.112:0/3239479135 learned_addr learned my addr 172.21.15.112:0/3239479135 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:01.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.691+0000 7f9601400700 1 --2- 172.21.15.112:0/3239479135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250fda0 0x558a52520210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.691+0000 7f9601e00700 1 -- 172.21.15.112:0/3239479135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250fda0 msgr2=0x558a52520210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.691+0000 7f9601e00700 1 --2- 172.21.15.112:0/3239479135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250fda0 0x558a52520210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.691+0000 7f9601e00700 1 -- 172.21.15.112:0/3239479135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 msgr2=0x558a524ff860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.691+0000 7f9601e00700 1 --2- 172.21.15.112:0/3239479135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 0x558a524ff860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.691+0000 7f9601e00700 1 -- 172.21.15.112:0/3239479135 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a524ffda0 con 0x558a5250aa50 2024-05-08T05:56:01.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.691+0000 7f9601400700 1 --2- 172.21.15.112:0/3239479135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250fda0 0x558a52520210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:01.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.691+0000 7f9601e00700 1 --2- 172.21.15.112:0/3239479135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250aa50 0x558a52513d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f95f4028660 tx=0x7f95f40284c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=652741a4d3f1d70d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:01.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.691+0000 7f95eea00700 1 -- 172.21.15.112:0/3239479135 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f400fd50 con 0x558a5250aa50 2024-05-08T05:56:01.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.692+0000 7f95eea00700 1 -- 172.21.15.112:0/3239479135 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95f4037020 con 0x558a5250aa50 2024-05-08T05:56:01.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.692+0000 7f95eea00700 1 -- 172.21.15.112:0/3239479135 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4003ab0 con 0x558a5250aa50 2024-05-08T05:56:01.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.692+0000 7f961330cb80 1 -- 172.21.15.112:0/3239479135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250aa50 msgr2=0x558a52513d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.692+0000 7f961330cb80 1 --2- 172.21.15.112:0/3239479135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250aa50 0x558a52513d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f95f4028660 tx=0x7f95f40284c0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.693+0000 7f961330cb80 1 -- 172.21.15.112:0/3239479135 shutdown_connections 2024-05-08T05:56:01.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.693+0000 7f961330cb80 1 --2- 172.21.15.112:0/3239479135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 0x558a524ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.693+0000 7f961330cb80 1 --2- 172.21.15.112:0/3239479135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250aa50 0x558a52513d80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.693+0000 7f961330cb80 1 --2- 172.21.15.112:0/3239479135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250fda0 0x558a52520210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.693+0000 7f961330cb80 1 -- 172.21.15.112:0/3239479135 >> 172.21.15.112:0/3239479135 conn(0x558a524826d0 msgr2=0x558a52510aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:01.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.693+0000 7f961330cb80 1 -- 172.21.15.112:0/3239479135 shutdown_connections 2024-05-08T05:56:01.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.693+0000 7f961330cb80 1 -- 172.21.15.112:0/3239479135 wait complete. 2024-05-08T05:56:01.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.693+0000 7f961330cb80 1 -- start start 2024-05-08T05:56:01.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.694+0000 7f961330cb80 1 Processor -- start 2024-05-08T05:56:01.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.694+0000 7f961330cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250aa50 0x558a5251c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.694+0000 7f961330cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250fda0 0x558a5251ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.694+0000 7f961330cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 0x558a5252b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.694+0000 7f961330cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96041ae1d0 con 0x558a525106c0 2024-05-08T05:56:01.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.694+0000 7f961330cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96041af640 con 0x558a5250fda0 2024-05-08T05:56:01.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.694+0000 7f961330cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f96041ae8a0 con 0x558a5250aa50 2024-05-08T05:56:01.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.694+0000 7f961330cb80 10 client.0 _ll_register_callbacks cb 0x558a52403c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:01.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.695+0000 7f9601e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250aa50 0x558a5251c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.695+0000 7f9601e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250aa50 0x558a5251c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:33826/0 (socket says 172.21.15.112:33826) 2024-05-08T05:56:01.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.695+0000 7f9601e00700 1 -- 172.21.15.112:0/502734508 learned_addr learned my addr 172.21.15.112:0/502734508 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:01.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.695+0000 7f9601400700 1 --2- 172.21.15.112:0/502734508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250fda0 0x558a5251ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.695+0000 7f9602800700 1 --2- 172.21.15.112:0/502734508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 0x558a5252b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.695+0000 7f9601e00700 1 -- 172.21.15.112:0/502734508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250fda0 msgr2=0x558a5251ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.695+0000 7f9601e00700 1 --2- 172.21.15.112:0/502734508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250fda0 0x558a5251ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.695+0000 7f9601e00700 1 -- 172.21.15.112:0/502734508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 msgr2=0x558a5252b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.695+0000 7f9601e00700 1 --2- 172.21.15.112:0/502734508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 0x558a5252b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.695+0000 7f9601e00700 1 -- 172.21.15.112:0/502734508 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a5252b960 con 0x558a5250aa50 2024-05-08T05:56:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.695+0000 7f9601e00700 1 --2- 172.21.15.112:0/502734508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250aa50 0x558a5251c770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f95f4028700 tx=0x7f95f400ff80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=575062ecdfdd3fab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:01.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.695+0000 7f95e3e00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:56:01.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.696+0000 7f95e3e00700 1 -- 172.21.15.112:0/502734508 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f40055a0 con 0x558a5250aa50 2024-05-08T05:56:01.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.696+0000 7f961330cb80 1 -- 172.21.15.112:0/502734508 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95f4037020 con 0x558a5250aa50 2024-05-08T05:56:01.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.696+0000 7f95e3e00700 1 -- 172.21.15.112:0/502734508 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f95f4003930 con 0x558a5250aa50 2024-05-08T05:56:01.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.696+0000 7f95e3e00700 1 -- 172.21.15.112:0/502734508 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f95f4005740 con 0x558a5250aa50 2024-05-08T05:56:01.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.696+0000 7f961330cb80 10 client.25186 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:01.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.696+0000 7f961330cb80 1 -- 172.21.15.112:0/502734508 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558a5258e5b0 con 0x558a5250aa50 2024-05-08T05:56:01.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.697+0000 7f95e3e00700 1 -- 172.21.15.112:0/502734508 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95f4037020 con 0x558a5250aa50 2024-05-08T05:56:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.697+0000 7f95e3e00700 1 -- 172.21.15.112:0/502734508 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f95f4026070 con 0x558a5250aa50 2024-05-08T05:56:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.697+0000 7f961330cb80 10 client.25186 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.697+0000 7f961330cb80 10 client.25186 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T05:56:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.697+0000 7f961330cb80 20 client.25186 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.697+0000 7f961330cb80 1 --2- 172.21.15.112:0/502734508 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x558a5258eb70 0x558a52590f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.697+0000 7f961330cb80 4 client.25186 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:56:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.697+0000 7f961330cb80 1 -- 172.21.15.112:0/502734508 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a5246a400 con 0x558a5258eb70 2024-05-08T05:56:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.697+0000 7f9601400700 1 --2- 172.21.15.112:0/502734508 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x558a5258eb70 0x558a52590f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.698+0000 7f9601400700 1 --2- 172.21.15.112:0/502734508 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x558a5258eb70 0x558a52590f90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4766b7fb33d7ef64 server_cookie=4b88f714ec09028 in_seq=0 out_seq=0 2024-05-08T05:56:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.698+0000 7f95e3e00700 10 client.25186 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:56:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.698+0000 7f95e3e00700 1 -- 172.21.15.112:0/502734508 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a5246a400 con 0x558a5258eb70 2024-05-08T05:56:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.698+0000 7f95e3e00700 10 client.25186 handle_command_reply: tid=0 2024-05-08T05:56:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.698+0000 7f961330cb80 1 client.25186 shutdown 2024-05-08T05:56:01.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.698+0000 7f961330cb80 1 -- 172.21.15.112:0/502734508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250aa50 msgr2=0x558a5251c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.698+0000 7f961330cb80 1 --2- 172.21.15.112:0/502734508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250aa50 0x558a5251c770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f95f4028700 tx=0x7f95f400ff80 comp rx=0 tx=0).stop 2024-05-08T05:56:01.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.698+0000 7f961330cb80 1 -- 172.21.15.112:0/502734508 shutdown_connections 2024-05-08T05:56:01.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.698+0000 7f961330cb80 1 --2- 172.21.15.112:0/502734508 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x558a5258eb70 0x558a52590f90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.698+0000 7f961330cb80 1 --2- 172.21.15.112:0/502734508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558a525106c0 0x558a5252b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.698+0000 7f961330cb80 1 --2- 172.21.15.112:0/502734508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558a5250aa50 0x558a5251c770 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.698+0000 7f961330cb80 1 --2- 172.21.15.112:0/502734508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a5250fda0 0x558a5251ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.698+0000 7f961330cb80 1 -- 172.21.15.112:0/502734508 >> 172.21.15.112:0/502734508 conn(0x558a524826d0 msgr2=0x558a52511330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:01.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.698+0000 7f95e3e00700 0 client.25186 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:56:01.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.699+0000 7f961330cb80 1 -- 172.21.15.112:0/502734508 shutdown_connections 2024-05-08T05:56:01.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.699+0000 7f961330cb80 1 -- 172.21.15.112:0/502734508 wait complete. 2024-05-08T05:56:01.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.699+0000 7f961330cb80 20 client.25186 trim_cache size 0 max 16384 2024-05-08T05:56:01.701 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:56:01.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.699+0000 7f9603200700 1 -- 172.21.15.112:0/2239548311 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f95e80b2900 msgr2=0x7f95e80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.699+0000 7f9603200700 1 --2- 172.21.15.112:0/2239548311 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f95e80b2900 0x7f95e80b4dc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009fd0 tx=0x7f95f4009260 comp rx=0 tx=0).stop 2024-05-08T05:56:01.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.700+0000 7f9603200700 1 -- 172.21.15.112:0/2239548311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9604103d90 msgr2=0x7f96041a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.700+0000 7f9603200700 1 --2- 172.21.15.112:0/2239548311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9604103d90 0x7f96041a1e70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f95f0005910 tx=0x7f95f000b670 comp rx=0 tx=0).stop 2024-05-08T05:56:01.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.700+0000 7f9603200700 1 -- 172.21.15.112:0/2239548311 shutdown_connections 2024-05-08T05:56:01.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.700+0000 7f9603200700 1 --2- 172.21.15.112:0/2239548311 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f95e80b2900 0x7f95e80b4dc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.700+0000 7f9603200700 1 --2- 172.21.15.112:0/2239548311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9604109460 0x7f960419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.700+0000 7f9603200700 1 --2- 172.21.15.112:0/2239548311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96041033d0 0x7f96041a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.700+0000 7f9603200700 1 --2- 172.21.15.112:0/2239548311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9604103d90 0x7f96041a1e70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.700+0000 7f9603200700 1 -- 172.21.15.112:0/2239548311 >> 172.21.15.112:0/2239548311 conn(0x7f96040fef20 msgr2=0x7f96040ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:01.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.700+0000 7f9603200700 1 -- 172.21.15.112:0/2239548311 shutdown_connections 2024-05-08T05:56:01.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:01.700+0000 7f9603200700 1 -- 172.21.15.112:0/2239548311 wait complete. 2024-05-08T05:56:02.816 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-08T05:56:03.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:02 smithi112 ceph-mon[30003]: pgmap v463: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:03.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:02 smithi150 ceph-mon[34315]: pgmap v463: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:03.256 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:03.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:02 smithi162 ceph-mon[34800]: pgmap v463: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:05.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:04 smithi112 ceph-mon[30003]: pgmap v464: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:56:05.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:04 smithi150 ceph-mon[34315]: pgmap v464: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:56:05.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:04 smithi162 ceph-mon[34800]: pgmap v464: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:56:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.715+0000 7f2896c00700 1 -- 172.21.15.112:0/3557971709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28900756d0 msgr2=0x7f2890075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.715+0000 7f2896c00700 1 --2- 172.21.15.112:0/3557971709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28900756d0 0x7f2890075ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2880009990 tx=0x7f2880009ca0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.715+0000 7f2896c00700 1 -- 172.21.15.112:0/3557971709 shutdown_connections 2024-05-08T05:56:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.715+0000 7f2896c00700 1 --2- 172.21.15.112:0/3557971709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2890114670 0x7f2890116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.715+0000 7f2896c00700 1 --2- 172.21.15.112:0/3557971709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28900756d0 0x7f2890075ab0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.715+0000 7f2896c00700 1 --2- 172.21.15.112:0/3557971709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2890075ff0 0x7f2890114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.715+0000 7f2896c00700 1 -- 172.21.15.112:0/3557971709 >> 172.21.15.112:0/3557971709 conn(0x7f289007a600 msgr2=0x7f289007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.715+0000 7f2896c00700 1 -- 172.21.15.112:0/3557971709 shutdown_connections 2024-05-08T05:56:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.715+0000 7f2896c00700 1 -- 172.21.15.112:0/3557971709 wait complete. 2024-05-08T05:56:05.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.716+0000 7f2896c00700 1 Processor -- start 2024-05-08T05:56:05.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.716+0000 7f2896c00700 1 -- start start 2024-05-08T05:56:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.717+0000 7f2896c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28900756d0 0x7f28901a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.717+0000 7f2896c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2890075ff0 0x7f28901a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.717+0000 7f2896c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2890114670 0x7f28901a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.717+0000 7f2896c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2890119c60 con 0x7f2890114670 2024-05-08T05:56:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.717+0000 7f2896c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2890119ae0 con 0x7f2890075ff0 2024-05-08T05:56:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.717+0000 7f2896c00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2890119de0 con 0x7f28900756d0 2024-05-08T05:56:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.717+0000 7f2894c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2890114670 0x7f28901a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.717+0000 7f288fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28900756d0 0x7f28901a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.717+0000 7f288f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2890075ff0 0x7f28901a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.717+0000 7f288fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28900756d0 0x7f28901a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:58236/0 (socket says 172.21.15.112:58236) 2024-05-08T05:56:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.717+0000 7f2894c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2890114670 0x7f28901a4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60170/0 (socket says 172.21.15.112:60170) 2024-05-08T05:56:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.717+0000 7f2894c00700 1 -- 172.21.15.112:0/2637323520 learned_addr learned my addr 172.21.15.112:0/2637323520 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.717+0000 7f288f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2890075ff0 0x7f28901a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:41762/0 (socket says 172.21.15.112:41762) 2024-05-08T05:56:05.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.718+0000 7f288f400700 1 -- 172.21.15.112:0/2637323520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28900756d0 msgr2=0x7f28901a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:05.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.718+0000 7f288f400700 1 --2- 172.21.15.112:0/2637323520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28900756d0 0x7f28901a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.718+0000 7f288f400700 1 -- 172.21.15.112:0/2637323520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2890114670 msgr2=0x7f28901a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:05.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.718+0000 7f288f400700 1 --2- 172.21.15.112:0/2637323520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2890114670 0x7f28901a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.718+0000 7f288f400700 1 -- 172.21.15.112:0/2637323520 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28901a5220 con 0x7f2890075ff0 2024-05-08T05:56:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.718+0000 7f288f400700 1 --2- 172.21.15.112:0/2637323520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2890075ff0 0x7f28901a0750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f287c00e9f0 tx=0x7f287c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.718+0000 7f288cc00700 1 -- 172.21.15.112:0/2637323520 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c00cb70 con 0x7f2890075ff0 2024-05-08T05:56:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.718+0000 7f2896c00700 1 -- 172.21.15.112:0/2637323520 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28901a5510 con 0x7f2890075ff0 2024-05-08T05:56:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.718+0000 7f288cc00700 1 -- 172.21.15.112:0/2637323520 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f287c00ee90 con 0x7f2890075ff0 2024-05-08T05:56:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.718+0000 7f288cc00700 1 -- 172.21.15.112:0/2637323520 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c003d40 con 0x7f2890075ff0 2024-05-08T05:56:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.718+0000 7f2896c00700 1 -- 172.21.15.112:0/2637323520 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28901acd90 con 0x7f2890075ff0 2024-05-08T05:56:05.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.723+0000 7f288cc00700 1 -- 172.21.15.112:0/2637323520 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f287c003ee0 con 0x7f2890075ff0 2024-05-08T05:56:05.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.724+0000 7f28a15c6b80 1 Processor -- start 2024-05-08T05:56:05.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.724+0000 7f28a15c6b80 1 -- start start 2024-05-08T05:56:05.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.725+0000 7f28a15c6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e75fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.725+0000 7f28a15c6b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e764e0 0x563170e702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.725+0000 7f288cc00700 1 --2- 172.21.15.112:0/2637323520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f28740b2880 0x7f28740b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.725+0000 7f28a15c6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e708f0 0x563170e70d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.725+0000 7f28a15c6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563170dc2a20 con 0x563170e708f0 2024-05-08T05:56:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.725+0000 7f28a15c6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563170d60760 con 0x563170e75bc0 2024-05-08T05:56:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.725+0000 7f28a15c6b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563170dbe9e0 con 0x563170e764e0 2024-05-08T05:56:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.725+0000 7f288cc00700 1 -- 172.21.15.112:0/2637323520 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f287c0e2380 con 0x7f2890075ff0 2024-05-08T05:56:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.725+0000 7f288fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e764e0 0x563170e702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.725+0000 7f288fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e764e0 0x563170e702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:58244/0 (socket says 172.21.15.112:58244) 2024-05-08T05:56:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.725+0000 7f288fe00700 1 -- 172.21.15.112:0/2177774028 learned_addr learned my addr 172.21.15.112:0/2177774028 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.725+0000 7f288f400700 1 --2- 172.21.15.112:0/2637323520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f28740b2880 0x7f28740b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.725+0000 7f2894c00700 1 --2- 172.21.15.112:0/2177774028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e708f0 0x563170e70d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.726+0000 7f288fe00700 1 -- 172.21.15.112:0/2177774028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 msgr2=0x563170e75fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:56:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.726+0000 7f288fe00700 1 --2- 172.21.15.112:0/2177774028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e75fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.726+0000 7f288fe00700 1 -- 172.21.15.112:0/2177774028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e708f0 msgr2=0x563170e70d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.726+0000 7f288fe00700 1 --2- 172.21.15.112:0/2177774028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e708f0 0x563170e70d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.726+0000 7f288fe00700 1 -- 172.21.15.112:0/2177774028 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563170d0f8d0 con 0x563170e764e0 2024-05-08T05:56:05.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.726+0000 7f2894c00700 1 --2- 172.21.15.112:0/2177774028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e708f0 0x563170e70d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:05.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.726+0000 7f288fe00700 1 --2- 172.21.15.112:0/2177774028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e764e0 0x563170e702d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2880009990 tx=0x7f28800096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10d3dd5947831b49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:05.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.726+0000 7f288f400700 1 --2- 172.21.15.112:0/2637323520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f28740b2880 0x7f28740b4d40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f287c0037a0 tx=0x7f287c00bf10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:05.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.726+0000 7f2885600700 1 -- 172.21.15.112:0/2177774028 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880027070 con 0x563170e764e0 2024-05-08T05:56:05.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.726+0000 7f2885600700 1 -- 172.21.15.112:0/2177774028 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28800056a0 con 0x563170e764e0 2024-05-08T05:56:05.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.726+0000 7f2885600700 1 -- 172.21.15.112:0/2177774028 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880003b40 con 0x563170e764e0 2024-05-08T05:56:05.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.727+0000 7f28a15c6b80 1 -- 172.21.15.112:0/2177774028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e764e0 msgr2=0x563170e702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:05.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.727+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/2177774028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e764e0 0x563170e702d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2880009990 tx=0x7f28800096a0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.727+0000 7f28a15c6b80 1 -- 172.21.15.112:0/2177774028 shutdown_connections 2024-05-08T05:56:05.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.727+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/2177774028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e708f0 0x563170e70d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.727+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/2177774028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e75fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.727+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/2177774028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e764e0 0x563170e702d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.727+0000 7f28a15c6b80 1 -- 172.21.15.112:0/2177774028 >> 172.21.15.112:0/2177774028 conn(0x563170de86d0 msgr2=0x563170e6ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:05.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.727+0000 7f28a15c6b80 1 -- 172.21.15.112:0/2177774028 shutdown_connections 2024-05-08T05:56:05.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.728+0000 7f28a15c6b80 1 -- 172.21.15.112:0/2177774028 wait complete. 2024-05-08T05:56:05.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.728+0000 7f28a15c6b80 1 -- start start 2024-05-08T05:56:05.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.728+0000 7f28a15c6b80 1 Processor -- start 2024-05-08T05:56:05.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.729+0000 7f28a15c6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e708f0 0x563170e7c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.729+0000 7f28a15c6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e795d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.729+0000 7f28a15c6b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e764e0 0x563170e79b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.729+0000 7f28a15c6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563170e7cec0 con 0x563170e708f0 2024-05-08T05:56:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.729+0000 7f28a15c6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563170d28b20 con 0x563170e75bc0 2024-05-08T05:56:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.729+0000 7f28a15c6b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563170e7d040 con 0x563170e764e0 2024-05-08T05:56:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.729+0000 7f28a15c6b80 10 client.0 _ll_register_callbacks cb 0x563170d69c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.729+0000 7f2894c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e708f0 0x563170e7c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.729+0000 7f2894c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e708f0 0x563170e7c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60194/0 (socket says 172.21.15.112:60194) 2024-05-08T05:56:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.729+0000 7f2894c00700 1 -- 172.21.15.112:0/3925813457 learned_addr learned my addr 172.21.15.112:0/3925813457 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.729+0000 7f2894c00700 1 --2- 172.21.15.112:0/3925813457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e764e0 0x563170e79b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.730+0000 7f288fe00700 1 --2- 172.21.15.112:0/3925813457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e795d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.730+0000 7f2894c00700 1 -- 172.21.15.112:0/3925813457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e764e0 msgr2=0x563170e79b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.730+0000 7f2894c00700 1 --2- 172.21.15.112:0/3925813457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e764e0 0x563170e79b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.730+0000 7f2894c00700 1 -- 172.21.15.112:0/3925813457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 msgr2=0x563170e795d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.730+0000 7f2894c00700 1 --2- 172.21.15.112:0/3925813457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e795d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.730+0000 7f2894c00700 1 -- 172.21.15.112:0/3925813457 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563170d88420 con 0x563170e708f0 2024-05-08T05:56:05.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.730+0000 7f2894c00700 1 --2- 172.21.15.112:0/3925813457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e708f0 0x563170e7c230 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f288801a460 tx=0x7f2888009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fb362e9d5adfe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:05.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.730+0000 7f287a000700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:56:05.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.730+0000 7f287a000700 1 -- 172.21.15.112:0/3925813457 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288801ac90 con 0x563170e708f0 2024-05-08T05:56:05.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.731+0000 7f287a000700 1 -- 172.21.15.112:0/3925813457 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f2888012040 con 0x563170e708f0 2024-05-08T05:56:05.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.731+0000 7f28a15c6b80 1 -- 172.21.15.112:0/3925813457 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28800056a0 con 0x563170e708f0 2024-05-08T05:56:05.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.731+0000 7f287a000700 1 -- 172.21.15.112:0/3925813457 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f288801ae30 con 0x563170e708f0 2024-05-08T05:56:05.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.731+0000 7f28a15c6b80 10 client.15063 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:05.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.731+0000 7f28a15c6b80 1 -- 172.21.15.112:0/3925813457 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563170e87760 con 0x563170e708f0 2024-05-08T05:56:05.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.731+0000 7f287a000700 1 -- 172.21.15.112:0/3925813457 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28800056a0 con 0x563170e708f0 2024-05-08T05:56:05.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.731+0000 7f287a000700 1 -- 172.21.15.112:0/3925813457 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2888016500 con 0x563170e708f0 2024-05-08T05:56:05.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.731+0000 7f28a15c6b80 10 client.15063 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.731+0000 7f28a15c6b80 10 client.15063 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T05:56:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.732+0000 7f28a15c6b80 20 client.15063 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.732+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/3925813457 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x563170e89da0 0x563170e8c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.732+0000 7f28a15c6b80 4 client.15063 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.732+0000 7f28a15c6b80 1 -- 172.21.15.112:0/3925813457 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563170e3d140 con 0x563170e89da0 2024-05-08T05:56:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.732+0000 7f288fe00700 1 --2- 172.21.15.112:0/3925813457 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x563170e89da0 0x563170e8c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.732+0000 7f288fe00700 1 --2- 172.21.15.112:0/3925813457 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x563170e89da0 0x563170e8c190 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5bced16a5ef36d0e server_cookie=1ef245ceebc1bdf in_seq=0 out_seq=0 2024-05-08T05:56:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.732+0000 7f287a000700 10 client.15063 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T05:56:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.733+0000 7f287a000700 1 -- 172.21.15.112:0/3925813457 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563170e3d140 con 0x563170e89da0 2024-05-08T05:56:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.733+0000 7f287a000700 10 client.15063 handle_command_reply: tid=0 2024-05-08T05:56:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.733+0000 7f28a15c6b80 1 client.15063 shutdown 2024-05-08T05:56:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.733+0000 7f28a15c6b80 1 -- 172.21.15.112:0/3925813457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e708f0 msgr2=0x563170e7c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.733+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/3925813457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e708f0 0x563170e7c230 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f288801a460 tx=0x7f2888009920 comp rx=0 tx=0).stop 2024-05-08T05:56:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.734+0000 7f28a15c6b80 1 -- 172.21.15.112:0/3925813457 shutdown_connections 2024-05-08T05:56:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.734+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/3925813457 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x563170e89da0 0x563170e8c190 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.734+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/3925813457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e764e0 0x563170e79b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.734+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/3925813457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e708f0 0x563170e7c230 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.734+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/3925813457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e795d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.734+0000 7f28a15c6b80 1 -- 172.21.15.112:0/3925813457 >> 172.21.15.112:0/3925813457 conn(0x563170de86d0 msgr2=0x563170e6a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.734+0000 7f287a000700 0 client.15063 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T05:56:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.734+0000 7f28a15c6b80 1 -- 172.21.15.112:0/3925813457 shutdown_connections 2024-05-08T05:56:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.734+0000 7f28a15c6b80 1 -- 172.21.15.112:0/3925813457 wait complete. 2024-05-08T05:56:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.734+0000 7f28a15c6b80 20 client.15063 trim_cache size 0 max 16384 2024-05-08T05:56:05.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.750+0000 7f28a15c6b80 1 Processor -- start 2024-05-08T05:56:05.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.750+0000 7f28a15c6b80 1 -- start start 2024-05-08T05:56:05.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.751+0000 7f28a15c6b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e708f0 0x563170e78550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.751+0000 7f28a15c6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e771a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.751+0000 7f28a15c6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e764e0 0x563170e85f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.751+0000 7f28a15c6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28901b6b60 con 0x563170e764e0 2024-05-08T05:56:05.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.751+0000 7f28a15c6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f28901b43f0 con 0x563170e75bc0 2024-05-08T05:56:05.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.751+0000 7f28a15c6b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f28901b2240 con 0x563170e708f0 2024-05-08T05:56:05.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.751+0000 7f2894c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e708f0 0x563170e78550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.751+0000 7f2894c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e708f0 0x563170e78550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:58268/0 (socket says 172.21.15.112:58268) 2024-05-08T05:56:05.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.751+0000 7f288fe00700 1 --2- 172.21.15.112:0/1117658498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.751+0000 7f2894c00700 1 -- 172.21.15.112:0/1117658498 learned_addr learned my addr 172.21.15.112:0/1117658498 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:05.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.751+0000 7f2894c00700 1 --2- 172.21.15.112:0/1117658498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e764e0 0x563170e85f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.752+0000 7f288fe00700 1 -- 172.21.15.112:0/1117658498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e708f0 msgr2=0x563170e78550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:05.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.752+0000 7f288fe00700 1 --2- 172.21.15.112:0/1117658498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e708f0 0x563170e78550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.752+0000 7f288fe00700 1 -- 172.21.15.112:0/1117658498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e764e0 msgr2=0x563170e85f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:05.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.752+0000 7f288fe00700 1 --2- 172.21.15.112:0/1117658498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e764e0 0x563170e85f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.752+0000 7f288fe00700 1 -- 172.21.15.112:0/1117658498 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563170e7c5a0 con 0x563170e75bc0 2024-05-08T05:56:05.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.752+0000 7f2894c00700 1 --2- 172.21.15.112:0/1117658498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e764e0 0x563170e85f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:05.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.752+0000 7f288fe00700 1 --2- 172.21.15.112:0/1117658498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e771a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2880025040 tx=0x7f2880003720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=728f2a2ddff9790d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:05.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.752+0000 7f2884c00700 1 -- 172.21.15.112:0/1117658498 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880039070 con 0x563170e75bc0 2024-05-08T05:56:05.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.752+0000 7f2884c00700 1 -- 172.21.15.112:0/1117658498 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2880028040 con 0x563170e75bc0 2024-05-08T05:56:05.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.752+0000 7f2884c00700 1 -- 172.21.15.112:0/1117658498 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880032070 con 0x563170e75bc0 2024-05-08T05:56:05.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.753+0000 7f28a15c6b80 1 -- 172.21.15.112:0/1117658498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 msgr2=0x563170e771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:05.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.753+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/1117658498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e771a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2880025040 tx=0x7f2880003720 comp rx=0 tx=0).stop 2024-05-08T05:56:05.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.753+0000 7f28a15c6b80 1 -- 172.21.15.112:0/1117658498 shutdown_connections 2024-05-08T05:56:05.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.753+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/1117658498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e764e0 0x563170e85f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.753+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/1117658498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e708f0 0x563170e78550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.753+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/1117658498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e771a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.753+0000 7f28a15c6b80 1 -- 172.21.15.112:0/1117658498 >> 172.21.15.112:0/1117658498 conn(0x563170de86d0 msgr2=0x563170e74700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:05.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.753+0000 7f28a15c6b80 1 -- 172.21.15.112:0/1117658498 shutdown_connections 2024-05-08T05:56:05.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.753+0000 7f28a15c6b80 1 -- 172.21.15.112:0/1117658498 wait complete. 2024-05-08T05:56:05.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.754+0000 7f28a15c6b80 1 -- start start 2024-05-08T05:56:05.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.754+0000 7f28a15c6b80 1 Processor -- start 2024-05-08T05:56:05.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.754+0000 7f28a15c6b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e708f0 0x563170e82560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f28a15c6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e90df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f28a15c6b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e764e0 0x563170e91330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f28a15c6b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28901b28b0 con 0x563170e764e0 2024-05-08T05:56:05.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f28a15c6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f28901b3d20 con 0x563170e75bc0 2024-05-08T05:56:05.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f28a15c6b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f28901b2f80 con 0x563170e708f0 2024-05-08T05:56:05.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f28a15c6b80 10 client.0 _ll_register_callbacks cb 0x563170d69c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:05.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f2894c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e764e0 0x563170e91330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f2894c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e708f0 0x563170e82560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f2894c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e708f0 0x563170e82560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:58278/0 (socket says 172.21.15.112:58278) 2024-05-08T05:56:05.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f2894c00700 1 -- 172.21.15.112:0/4094891380 learned_addr learned my addr 172.21.15.112:0/4094891380 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:05.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f288fe00700 1 --2- 172.21.15.112:0/4094891380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e90df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f2894c00700 1 -- 172.21.15.112:0/4094891380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 msgr2=0x563170e90df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:05.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f2894c00700 1 --2- 172.21.15.112:0/4094891380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e90df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f2894c00700 1 -- 172.21.15.112:0/4094891380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e764e0 msgr2=0x563170e91330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:05.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f2894c00700 1 --2- 172.21.15.112:0/4094891380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e764e0 0x563170e91330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.755+0000 7f2894c00700 1 -- 172.21.15.112:0/4094891380 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563170e91870 con 0x563170e708f0 2024-05-08T05:56:05.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.756+0000 7f2894c00700 1 --2- 172.21.15.112:0/4094891380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e708f0 0x563170e82560 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2888008a10 tx=0x7f28880093d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=603ae379736e56ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:05.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.756+0000 7f287a000700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:56:05.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.756+0000 7f287a000700 1 -- 172.21.15.112:0/4094891380 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888003a00 con 0x563170e708f0 2024-05-08T05:56:05.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.756+0000 7f28a15c6b80 1 -- 172.21.15.112:0/4094891380 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2880028040 con 0x563170e708f0 2024-05-08T05:56:05.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.756+0000 7f287a000700 1 -- 172.21.15.112:0/4094891380 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f2888003ba0 con 0x563170e708f0 2024-05-08T05:56:05.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.756+0000 7f287a000700 1 -- 172.21.15.112:0/4094891380 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2888003da0 con 0x563170e708f0 2024-05-08T05:56:05.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.757+0000 7f28a15c6b80 10 client.25204 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.757+0000 7f28a15c6b80 1 -- 172.21.15.112:0/4094891380 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563170ef42e0 con 0x563170e708f0 2024-05-08T05:56:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.757+0000 7f287a000700 1 -- 172.21.15.112:0/4094891380 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2880028040 con 0x563170e708f0 2024-05-08T05:56:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.757+0000 7f287a000700 1 -- 172.21.15.112:0/4094891380 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2888004020 con 0x563170e708f0 2024-05-08T05:56:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.757+0000 7f28a15c6b80 10 client.25204 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.757+0000 7f28a15c6b80 10 client.25204 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T05:56:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.757+0000 7f28a15c6b80 20 client.25204 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.757+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/4094891380 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x563170ef4910 0x563170ef6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.757+0000 7f28a15c6b80 4 client.25204 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:56:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.757+0000 7f28a15c6b80 1 -- 172.21.15.112:0/4094891380 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563170dd0400 con 0x563170ef4910 2024-05-08T05:56:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.758+0000 7f288fe00700 1 --2- 172.21.15.112:0/4094891380 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x563170ef4910 0x563170ef6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.758+0000 7f288fe00700 1 --2- 172.21.15.112:0/4094891380 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x563170ef4910 0x563170ef6d30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48af3e08af67991c server_cookie=5943bdda77bd12d2 in_seq=0 out_seq=0 2024-05-08T05:56:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.758+0000 7f287a000700 10 client.25204 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T05:56:05.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.758+0000 7f287a000700 1 -- 172.21.15.112:0/4094891380 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563170dd0400 con 0x563170ef4910 2024-05-08T05:56:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.758+0000 7f287a000700 10 client.25204 handle_command_reply: tid=0 2024-05-08T05:56:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.758+0000 7f28a15c6b80 1 client.25204 shutdown 2024-05-08T05:56:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.759+0000 7f28a15c6b80 1 -- 172.21.15.112:0/4094891380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e708f0 msgr2=0x563170e82560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.759+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/4094891380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e708f0 0x563170e82560 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2888008a10 tx=0x7f28880093d0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.759+0000 7f28a15c6b80 1 -- 172.21.15.112:0/4094891380 shutdown_connections 2024-05-08T05:56:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.759+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/4094891380 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x563170ef4910 0x563170ef6d30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.759+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/4094891380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563170e764e0 0x563170e91330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.759+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/4094891380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563170e708f0 0x563170e82560 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.759+0000 7f28a15c6b80 1 --2- 172.21.15.112:0/4094891380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563170e75bc0 0x563170e90df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.759+0000 7f287a000700 0 client.25204 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T05:56:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.759+0000 7f28a15c6b80 1 -- 172.21.15.112:0/4094891380 >> 172.21.15.112:0/4094891380 conn(0x563170de86d0 msgr2=0x563170e74f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.759+0000 7f28a15c6b80 1 -- 172.21.15.112:0/4094891380 shutdown_connections 2024-05-08T05:56:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.759+0000 7f28a15c6b80 1 -- 172.21.15.112:0/4094891380 wait complete. 2024-05-08T05:56:05.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.759+0000 7f28a15c6b80 20 client.25204 trim_cache size 0 max 16384 2024-05-08T05:56:05.763 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:56:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.760+0000 7f2896c00700 1 -- 172.21.15.112:0/2637323520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f28740b2880 msgr2=0x7f28740b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.760+0000 7f2896c00700 1 --2- 172.21.15.112:0/2637323520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f28740b2880 0x7f28740b4d40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f287c0037a0 tx=0x7f287c00bf10 comp rx=0 tx=0).stop 2024-05-08T05:56:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.760+0000 7f2896c00700 1 -- 172.21.15.112:0/2637323520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2890075ff0 msgr2=0x7f28901a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.760+0000 7f2896c00700 1 --2- 172.21.15.112:0/2637323520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2890075ff0 0x7f28901a0750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f287c00e9f0 tx=0x7f287c00ed00 comp rx=0 tx=0).stop 2024-05-08T05:56:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.760+0000 7f2896c00700 1 -- 172.21.15.112:0/2637323520 shutdown_connections 2024-05-08T05:56:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.760+0000 7f2896c00700 1 --2- 172.21.15.112:0/2637323520 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f28740b2880 0x7f28740b4d40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.760+0000 7f2896c00700 1 --2- 172.21.15.112:0/2637323520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2890114670 0x7f28901a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.760+0000 7f2896c00700 1 --2- 172.21.15.112:0/2637323520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28900756d0 0x7f28901a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.760+0000 7f2896c00700 1 --2- 172.21.15.112:0/2637323520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2890075ff0 0x7f28901a0750 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.760+0000 7f2896c00700 1 -- 172.21.15.112:0/2637323520 >> 172.21.15.112:0/2637323520 conn(0x7f289007a600 msgr2=0x7f28901134a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.760+0000 7f2896c00700 1 -- 172.21.15.112:0/2637323520 shutdown_connections 2024-05-08T05:56:05.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:05.760+0000 7f2896c00700 1 -- 172.21.15.112:0/2637323520 wait complete. 2024-05-08T05:56:06.782 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-08T05:56:07.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:06 smithi112 ceph-mon[30003]: pgmap v465: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:56:07.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:06 smithi150 ceph-mon[34315]: pgmap v465: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:56:07.208 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:07.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:06 smithi162 ceph-mon[34800]: pgmap v465: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:56:08.956 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:08 smithi112 ceph-mon[30003]: pgmap v466: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:09.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:08 smithi150 ceph-mon[34315]: pgmap v466: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:09.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:08 smithi162 ceph-mon[34800]: pgmap v466: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:09.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.473+0000 7f0957e00700 1 -- 172.21.15.112:0/1201395094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0950103750 msgr2=0x7f0950103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:09.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.473+0000 7f0957e00700 1 --2- 172.21.15.112:0/1201395094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0950103750 0x7f0950103b30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f09440098f0 tx=0x7f0944009c00 comp rx=0 tx=0).stop 2024-05-08T05:56:09.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.473+0000 7f0957e00700 1 -- 172.21.15.112:0/1201395094 shutdown_connections 2024-05-08T05:56:09.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.473+0000 7f0957e00700 1 --2- 172.21.15.112:0/1201395094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0950108850 0x7f09501103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.473+0000 7f0957e00700 1 --2- 172.21.15.112:0/1201395094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0950103750 0x7f0950103b30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.473+0000 7f0957e00700 1 --2- 172.21.15.112:0/1201395094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0950104100 0x7f0950108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.473+0000 7f0957e00700 1 -- 172.21.15.112:0/1201395094 >> 172.21.15.112:0/1201395094 conn(0x7f095007a600 msgr2=0x7f095007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:09.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.473+0000 7f0957e00700 1 -- 172.21.15.112:0/1201395094 shutdown_connections 2024-05-08T05:56:09.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.473+0000 7f0957e00700 1 -- 172.21.15.112:0/1201395094 wait complete. 2024-05-08T05:56:09.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.474+0000 7f0957e00700 1 Processor -- start 2024-05-08T05:56:09.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.474+0000 7f0957e00700 1 -- start start 2024-05-08T05:56:09.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.475+0000 7f0957e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0950103750 0x7f095019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.475+0000 7f0957e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0950104100 0x7f095019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.475+0000 7f0957e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0950108850 0x7f09501999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.475+0000 7f0957e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0950113670 con 0x7f0950104100 2024-05-08T05:56:09.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.475+0000 7f0957e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f09501134f0 con 0x7f0950103750 2024-05-08T05:56:09.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.475+0000 7f0957e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f09501137f0 con 0x7f0950108850 2024-05-08T05:56:09.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.475+0000 7f0954a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0950104100 0x7f095019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.475+0000 7f0954a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0950104100 0x7f095019fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60228/0 (socket says 172.21.15.112:60228) 2024-05-08T05:56:09.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.475+0000 7f0954a00700 1 -- 172.21.15.112:0/1136768348 learned_addr learned my addr 172.21.15.112:0/1136768348 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:09.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.475+0000 7f0955400700 1 --2- 172.21.15.112:0/1136768348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0950103750 0x7f095019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.475+0000 7f0955e00700 1 --2- 172.21.15.112:0/1136768348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0950108850 0x7f09501999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.475+0000 7f0955400700 1 -- 172.21.15.112:0/1136768348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0950108850 msgr2=0x7f09501999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:09.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.475+0000 7f0955400700 1 --2- 172.21.15.112:0/1136768348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0950108850 0x7f09501999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.476+0000 7f0955400700 1 -- 172.21.15.112:0/1136768348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0950104100 msgr2=0x7f095019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:09.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.476+0000 7f0955400700 1 --2- 172.21.15.112:0/1136768348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0950104100 0x7f095019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.476+0000 7f0955400700 1 -- 172.21.15.112:0/1136768348 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095019a130 con 0x7f0950103750 2024-05-08T05:56:09.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.476+0000 7f0954a00700 1 --2- 172.21.15.112:0/1136768348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0950104100 0x7f095019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:09.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.476+0000 7f0955400700 1 --2- 172.21.15.112:0/1136768348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0950103750 0x7f095019f810 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f094400c010 tx=0x7f094400fa20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:09.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.476+0000 7f094e000700 1 -- 172.21.15.112:0/1136768348 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09440050c0 con 0x7f0950103750 2024-05-08T05:56:09.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.476+0000 7f094e000700 1 -- 172.21.15.112:0/1136768348 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f094400b560 con 0x7f0950103750 2024-05-08T05:56:09.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.476+0000 7f094e000700 1 -- 172.21.15.112:0/1136768348 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09440039a0 con 0x7f0950103750 2024-05-08T05:56:09.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.476+0000 7f0957e00700 1 -- 172.21.15.112:0/1136768348 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095019a3c0 con 0x7f0950103750 2024-05-08T05:56:09.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.476+0000 7f0957e00700 1 -- 172.21.15.112:0/1136768348 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09501a66e0 con 0x7f0950103750 2024-05-08T05:56:09.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.480+0000 7f094e000700 1 -- 172.21.15.112:0/1136768348 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f094400b710 con 0x7f0950103750 2024-05-08T05:56:09.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.481+0000 7f096272fb80 1 Processor -- start 2024-05-08T05:56:09.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.481+0000 7f096272fb80 1 -- start start 2024-05-08T05:56:09.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.481+0000 7f096272fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e2117abc0 0x562e2117afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.481+0000 7f096272fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117b4e0 0x562e211752d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.481+0000 7f096272fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e211758f0 0x562e21175d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.481+0000 7f096272fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562e210c7a20 con 0x562e2117abc0 2024-05-08T05:56:09.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.481+0000 7f096272fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562e21065760 con 0x562e211758f0 2024-05-08T05:56:09.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.481+0000 7f096272fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562e210c39e0 con 0x562e2117b4e0 2024-05-08T05:56:09.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.482+0000 7f0955e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e2117abc0 0x562e2117afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.482+0000 7f0955e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e2117abc0 0x562e2117afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60234/0 (socket says 172.21.15.112:60234) 2024-05-08T05:56:09.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.482+0000 7f0954a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e211758f0 0x562e21175d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.482+0000 7f0955e00700 1 -- 172.21.15.112:0/2171291972 learned_addr learned my addr 172.21.15.112:0/2171291972 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:09.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.482+0000 7f0955400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117b4e0 0x562e211752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.482+0000 7f094e000700 1 --2- 172.21.15.112:0/1136768348 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f09380b2900 0x7f09380b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.482+0000 7f0954a00700 1 -- 172.21.15.112:0/2171291972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117b4e0 msgr2=0x562e211752d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:09.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.482+0000 7f0954a00700 1 --2- 172.21.15.112:0/2171291972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117b4e0 0x562e211752d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.482+0000 7f0954a00700 1 -- 172.21.15.112:0/2171291972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e2117abc0 msgr2=0x562e2117afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:09.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.482+0000 7f0954a00700 1 --2- 172.21.15.112:0/2171291972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e2117abc0 0x562e2117afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.482+0000 7f0954a00700 1 -- 172.21.15.112:0/2171291972 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e210148d0 con 0x562e211758f0 2024-05-08T05:56:09.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.482+0000 7f0955e00700 1 --2- 172.21.15.112:0/2171291972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e2117abc0 0x562e2117afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:09.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.482+0000 7f0955e00700 1 --2- 172.21.15.112:0/1136768348 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f09380b2900 0x7f09380b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.482+0000 7f094e000700 1 -- 172.21.15.112:0/1136768348 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f09440e2fd0 con 0x7f0950103750 2024-05-08T05:56:09.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.482+0000 7f0954a00700 1 --2- 172.21.15.112:0/2171291972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e211758f0 0x562e21175d70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f093c011540 tx=0x7f093c011850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b736860adc5d294d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:09.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.483+0000 7f0942a00700 1 -- 172.21.15.112:0/2171291972 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c004230 con 0x562e211758f0 2024-05-08T05:56:09.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.483+0000 7f0942a00700 1 -- 172.21.15.112:0/2171291972 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f093c0043d0 con 0x562e211758f0 2024-05-08T05:56:09.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.483+0000 7f0942a00700 1 -- 172.21.15.112:0/2171291972 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c019920 con 0x562e211758f0 2024-05-08T05:56:09.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.484+0000 7f096272fb80 1 -- 172.21.15.112:0/2171291972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e211758f0 msgr2=0x562e21175d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:09.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.484+0000 7f0955e00700 1 --2- 172.21.15.112:0/1136768348 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f09380b2900 0x7f09380b4dc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f094800e1d0 tx=0x7f0948009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:09.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.484+0000 7f096272fb80 1 --2- 172.21.15.112:0/2171291972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e211758f0 0x562e21175d70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f093c011540 tx=0x7f093c011850 comp rx=0 tx=0).stop 2024-05-08T05:56:09.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.484+0000 7f096272fb80 1 -- 172.21.15.112:0/2171291972 shutdown_connections 2024-05-08T05:56:09.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.484+0000 7f096272fb80 1 --2- 172.21.15.112:0/2171291972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e211758f0 0x562e21175d70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.484+0000 7f096272fb80 1 --2- 172.21.15.112:0/2171291972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e2117abc0 0x562e2117afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.484+0000 7f096272fb80 1 --2- 172.21.15.112:0/2171291972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117b4e0 0x562e211752d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.484+0000 7f096272fb80 1 -- 172.21.15.112:0/2171291972 >> 172.21.15.112:0/2171291972 conn(0x562e210ed6d0 msgr2=0x562e2116fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:09.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.484+0000 7f096272fb80 1 -- 172.21.15.112:0/2171291972 shutdown_connections 2024-05-08T05:56:09.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.484+0000 7f096272fb80 1 -- 172.21.15.112:0/2171291972 wait complete. 2024-05-08T05:56:09.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.485+0000 7f096272fb80 1 -- start start 2024-05-08T05:56:09.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.485+0000 7f096272fb80 1 Processor -- start 2024-05-08T05:56:09.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.485+0000 7f096272fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e211758f0 0x562e21188e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.485+0000 7f096272fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e2117abc0 0x562e21189340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.485+0000 7f096272fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e2117b4e0 0x562e21183220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.485+0000 7f096272fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562e21181f20 con 0x562e2117abc0 2024-05-08T05:56:09.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.485+0000 7f096272fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562e2102db20 con 0x562e2117b4e0 2024-05-08T05:56:09.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.485+0000 7f096272fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562e211820a0 con 0x562e211758f0 2024-05-08T05:56:09.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.486+0000 7f096272fb80 10 client.0 _ll_register_callbacks cb 0x562e2106ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:09.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.486+0000 7f0954a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e2117abc0 0x562e21189340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.486+0000 7f0955400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e211758f0 0x562e21188e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.486+0000 7f0955e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e2117b4e0 0x562e21183220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.486+0000 7f0955400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e211758f0 0x562e21188e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:58308/0 (socket says 172.21.15.112:58308) 2024-05-08T05:56:09.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.486+0000 7f0955400700 1 -- 172.21.15.112:0/3899257147 learned_addr learned my addr 172.21.15.112:0/3899257147 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:09.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.486+0000 7f0955400700 1 -- 172.21.15.112:0/3899257147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e2117b4e0 msgr2=0x562e21183220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:09.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.486+0000 7f0955400700 1 --2- 172.21.15.112:0/3899257147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e2117b4e0 0x562e21183220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.486+0000 7f0955400700 1 -- 172.21.15.112:0/3899257147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e2117abc0 msgr2=0x562e21189340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:09.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.486+0000 7f0955400700 1 --2- 172.21.15.112:0/3899257147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e2117abc0 0x562e21189340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.486+0000 7f0955400700 1 -- 172.21.15.112:0/3899257147 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e2108d420 con 0x562e211758f0 2024-05-08T05:56:09.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.487+0000 7f0955400700 1 --2- 172.21.15.112:0/3899257147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e211758f0 0x562e21188e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0944009810 tx=0x7f09440a6b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfc079748bcd1ec4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:09.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.487+0000 7f0937400700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:56:09.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.487+0000 7f0937400700 1 -- 172.21.15.112:0/3899257147 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09440a9d60 con 0x562e211758f0 2024-05-08T05:56:09.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.487+0000 7f0937400700 1 -- 172.21.15.112:0/3899257147 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f09440aa850 con 0x562e211758f0 2024-05-08T05:56:09.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.487+0000 7f096272fb80 1 -- 172.21.15.112:0/3899257147 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f093c0043d0 con 0x562e211758f0 2024-05-08T05:56:09.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.487+0000 7f0937400700 1 -- 172.21.15.112:0/3899257147 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f09440aa100 con 0x562e211758f0 2024-05-08T05:56:09.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.487+0000 7f096272fb80 10 client.25216 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:09.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.487+0000 7f096272fb80 1 -- 172.21.15.112:0/3899257147 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562e21181f80 con 0x562e211758f0 2024-05-08T05:56:09.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.487+0000 7f0937400700 1 -- 172.21.15.112:0/3899257147 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f093c0043d0 con 0x562e211758f0 2024-05-08T05:56:09.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.488+0000 7f0937400700 1 -- 172.21.15.112:0/3899257147 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f09440b2f30 con 0x562e211758f0 2024-05-08T05:56:09.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.488+0000 7f096272fb80 10 client.25216 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:09.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.488+0000 7f096272fb80 10 client.25216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:56:09.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.488+0000 7f096272fb80 20 client.25216 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:09.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.488+0000 7f096272fb80 1 --2- 172.21.15.112:0/3899257147 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562e21192ff0 0x562e21195420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.488+0000 7f096272fb80 4 client.25216 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:09.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.488+0000 7f096272fb80 1 -- 172.21.15.112:0/3899257147 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e21142140 con 0x562e21192ff0 2024-05-08T05:56:09.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.488+0000 7f0954a00700 1 --2- 172.21.15.112:0/3899257147 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562e21192ff0 0x562e21195420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.489+0000 7f0954a00700 1 --2- 172.21.15.112:0/3899257147 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562e21192ff0 0x562e21195420 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1393ec0a5a8e80d server_cookie=c5e7c118ccd0b612 in_seq=0 out_seq=0 2024-05-08T05:56:09.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.489+0000 7f0937400700 10 client.25216 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:09.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.489+0000 7f0937400700 1 -- 172.21.15.112:0/3899257147 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e21142140 con 0x562e21192ff0 2024-05-08T05:56:09.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.489+0000 7f0937400700 10 client.25216 handle_command_reply: tid=0 2024-05-08T05:56:09.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.490+0000 7f096272fb80 1 client.25216 shutdown 2024-05-08T05:56:09.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.490+0000 7f096272fb80 1 -- 172.21.15.112:0/3899257147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e211758f0 msgr2=0x562e21188e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:09.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.490+0000 7f096272fb80 1 --2- 172.21.15.112:0/3899257147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e211758f0 0x562e21188e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0944009810 tx=0x7f09440a6b30 comp rx=0 tx=0).stop 2024-05-08T05:56:09.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.490+0000 7f096272fb80 1 -- 172.21.15.112:0/3899257147 shutdown_connections 2024-05-08T05:56:09.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.490+0000 7f096272fb80 1 --2- 172.21.15.112:0/3899257147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e2117b4e0 0x562e21183220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.490+0000 7f096272fb80 1 --2- 172.21.15.112:0/3899257147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e211758f0 0x562e21188e00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.490+0000 7f096272fb80 1 --2- 172.21.15.112:0/3899257147 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562e21192ff0 0x562e21195420 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.490+0000 7f096272fb80 1 --2- 172.21.15.112:0/3899257147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e2117abc0 0x562e21189340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.490+0000 7f0937400700 0 client.25216 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:09.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.490+0000 7f096272fb80 1 -- 172.21.15.112:0/3899257147 >> 172.21.15.112:0/3899257147 conn(0x562e210ed6d0 msgr2=0x562e2116f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:09.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.490+0000 7f096272fb80 1 -- 172.21.15.112:0/3899257147 shutdown_connections 2024-05-08T05:56:09.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.490+0000 7f096272fb80 1 -- 172.21.15.112:0/3899257147 wait complete. 2024-05-08T05:56:09.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.490+0000 7f096272fb80 20 client.25216 trim_cache size 0 max 16384 2024-05-08T05:56:09.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.507+0000 7f096272fb80 1 Processor -- start 2024-05-08T05:56:09.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.507+0000 7f096272fb80 1 -- start start 2024-05-08T05:56:09.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.507+0000 7f096272fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e211758f0 0x562e211752d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.507+0000 7f096272fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e2117abc0 0x562e2119acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.507+0000 7f096272fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117b4e0 0x562e2119b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.507+0000 7f096272fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f09501ae1b0 con 0x562e211758f0 2024-05-08T05:56:09.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.507+0000 7f096272fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f09501aba40 con 0x562e2117abc0 2024-05-08T05:56:09.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.507+0000 7f096272fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f09501a9890 con 0x562e2117b4e0 2024-05-08T05:56:09.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.508+0000 7f0955400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e211758f0 0x562e211752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.508+0000 7f0955e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117b4e0 0x562e2119b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.508+0000 7f0955e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117b4e0 0x562e2119b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:58324/0 (socket says 172.21.15.112:58324) 2024-05-08T05:56:09.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.508+0000 7f0955400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e211758f0 0x562e211752d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60248/0 (socket says 172.21.15.112:60248) 2024-05-08T05:56:09.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.508+0000 7f0955e00700 1 -- 172.21.15.112:0/1745812237 learned_addr learned my addr 172.21.15.112:0/1745812237 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:09.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.508+0000 7f0954a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e2117abc0 0x562e2119acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.508+0000 7f0955e00700 1 -- 172.21.15.112:0/1745812237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e2117abc0 msgr2=0x562e2119acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:09.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.508+0000 7f0955e00700 1 --2- 172.21.15.112:0/1745812237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e2117abc0 0x562e2119acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.508+0000 7f0955e00700 1 -- 172.21.15.112:0/1745812237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e211758f0 msgr2=0x562e211752d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:09.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.508+0000 7f0955e00700 1 --2- 172.21.15.112:0/1745812237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e211758f0 0x562e211752d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.508+0000 7f0955e00700 1 -- 172.21.15.112:0/1745812237 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e2116aab0 con 0x562e2117b4e0 2024-05-08T05:56:09.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.508+0000 7f0955400700 1 --2- 172.21.15.112:0/1745812237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e211758f0 0x562e211752d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:09.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.508+0000 7f0955e00700 1 --2- 172.21.15.112:0/1745812237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117b4e0 0x562e2119b230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0948016460 tx=0x7f0948004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=942e1d8cee76154c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:09.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.509+0000 7f0942000700 1 -- 172.21.15.112:0/1745812237 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948016df0 con 0x562e2117b4e0 2024-05-08T05:56:09.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.509+0000 7f0942000700 1 -- 172.21.15.112:0/1745812237 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0948004020 con 0x562e2117b4e0 2024-05-08T05:56:09.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.509+0000 7f0942000700 1 -- 172.21.15.112:0/1745812237 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948024670 con 0x562e2117b4e0 2024-05-08T05:56:09.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.510+0000 7f096272fb80 1 -- 172.21.15.112:0/1745812237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117b4e0 msgr2=0x562e2119b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:09.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.510+0000 7f096272fb80 1 --2- 172.21.15.112:0/1745812237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117b4e0 0x562e2119b230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0948016460 tx=0x7f0948004b40 comp rx=0 tx=0).stop 2024-05-08T05:56:09.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.510+0000 7f096272fb80 1 -- 172.21.15.112:0/1745812237 shutdown_connections 2024-05-08T05:56:09.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.510+0000 7f096272fb80 1 --2- 172.21.15.112:0/1745812237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117b4e0 0x562e2119b230 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.510+0000 7f096272fb80 1 --2- 172.21.15.112:0/1745812237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e211758f0 0x562e211752d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.510+0000 7f096272fb80 1 --2- 172.21.15.112:0/1745812237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e2117abc0 0x562e2119acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.510+0000 7f096272fb80 1 -- 172.21.15.112:0/1745812237 >> 172.21.15.112:0/1745812237 conn(0x562e210ed6d0 msgr2=0x562e21188e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:09.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.510+0000 7f096272fb80 1 -- 172.21.15.112:0/1745812237 shutdown_connections 2024-05-08T05:56:09.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.510+0000 7f096272fb80 1 -- 172.21.15.112:0/1745812237 wait complete. 2024-05-08T05:56:09.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.510+0000 7f096272fb80 1 -- start start 2024-05-08T05:56:09.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.510+0000 7f096272fb80 1 Processor -- start 2024-05-08T05:56:09.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.511+0000 7f096272fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e211758f0 0x562e2118f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.511+0000 7f096272fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117abc0 0x562e2118fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.511+0000 7f096272fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e2117b4e0 0x562e2118aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.511+0000 7f096272fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f09501a9f00 con 0x562e211758f0 2024-05-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.511+0000 7f096272fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f09501ab370 con 0x562e2117b4e0 2024-05-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.511+0000 7f096272fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f09501aa5d0 con 0x562e2117abc0 2024-05-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.511+0000 7f096272fb80 10 client.0 _ll_register_callbacks cb 0x562e2106ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.511+0000 7f0955400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e211758f0 0x562e2118f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.511+0000 7f0954a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117abc0 0x562e2118fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.511+0000 7f0955400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e211758f0 0x562e2118f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60264/0 (socket says 172.21.15.112:60264) 2024-05-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.511+0000 7f0955400700 1 -- 172.21.15.112:0/2036990306 learned_addr learned my addr 172.21.15.112:0/2036990306 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.511+0000 7f0955e00700 1 --2- 172.21.15.112:0/2036990306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e2117b4e0 0x562e2118aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.512+0000 7f0954a00700 1 -- 172.21.15.112:0/2036990306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e2117b4e0 msgr2=0x562e2118aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.512+0000 7f0954a00700 1 --2- 172.21.15.112:0/2036990306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e2117b4e0 0x562e2118aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.512+0000 7f0954a00700 1 -- 172.21.15.112:0/2036990306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e211758f0 msgr2=0x562e2118f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.512+0000 7f0954a00700 1 --2- 172.21.15.112:0/2036990306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e211758f0 0x562e2118f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.512+0000 7f0954a00700 1 -- 172.21.15.112:0/2036990306 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e2118b320 con 0x562e2117abc0 2024-05-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.512+0000 7f0954a00700 1 --2- 172.21.15.112:0/2036990306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117abc0 0x562e2118fa50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f093c009850 tx=0x7f093c005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd4dfbe3c0739d57 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.512+0000 7f0937400700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.512+0000 7f0937400700 1 -- 172.21.15.112:0/2036990306 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c003880 con 0x562e2117abc0 2024-05-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.512+0000 7f096272fb80 1 -- 172.21.15.112:0/2036990306 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0948004020 con 0x562e2117abc0 2024-05-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.513+0000 7f0937400700 1 -- 172.21.15.112:0/2036990306 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f093c003a20 con 0x562e2117abc0 2024-05-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.513+0000 7f0937400700 1 -- 172.21.15.112:0/2036990306 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f093c03b050 con 0x562e2117abc0 2024-05-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.513+0000 7f096272fb80 10 client.25222 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.513+0000 7f096272fb80 1 -- 172.21.15.112:0/2036990306 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562e211fd530 con 0x562e2117abc0 2024-05-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.513+0000 7f0937400700 1 -- 172.21.15.112:0/2036990306 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0948004020 con 0x562e2117abc0 2024-05-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.513+0000 7f0937400700 1 -- 172.21.15.112:0/2036990306 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f093c02bce0 con 0x562e2117abc0 2024-05-08T05:56:09.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.513+0000 7f096272fb80 10 client.25222 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:09.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.513+0000 7f096272fb80 10 client.25222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:56:09.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.513+0000 7f096272fb80 20 client.25222 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:09.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.513+0000 7f096272fb80 1 --2- 172.21.15.112:0/2036990306 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562e211fdab0 0x562e211ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.513+0000 7f096272fb80 4 client.25222 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:56:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.513+0000 7f096272fb80 1 -- 172.21.15.112:0/2036990306 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e210d5400 con 0x562e211fdab0 2024-05-08T05:56:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.514+0000 7f0955400700 1 --2- 172.21.15.112:0/2036990306 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562e211fdab0 0x562e211ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.514+0000 7f0937400700 10 client.25222 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.514+0000 7f0955400700 1 --2- 172.21.15.112:0/2036990306 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562e211fdab0 0x562e211ffed0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2219f001218b017 server_cookie=e8d2d166ae1bbc15 in_seq=0 out_seq=0 2024-05-08T05:56:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.515+0000 7f0937400700 1 -- 172.21.15.112:0/2036990306 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e210d5400 con 0x562e211fdab0 2024-05-08T05:56:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.515+0000 7f0937400700 10 client.25222 handle_command_reply: tid=0 2024-05-08T05:56:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.515+0000 7f096272fb80 1 client.25222 shutdown 2024-05-08T05:56:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.515+0000 7f096272fb80 1 -- 172.21.15.112:0/2036990306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117abc0 msgr2=0x562e2118fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.515+0000 7f096272fb80 1 --2- 172.21.15.112:0/2036990306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117abc0 0x562e2118fa50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f093c009850 tx=0x7f093c005e40 comp rx=0 tx=0).stop 2024-05-08T05:56:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.515+0000 7f096272fb80 1 -- 172.21.15.112:0/2036990306 shutdown_connections 2024-05-08T05:56:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.515+0000 7f096272fb80 1 --2- 172.21.15.112:0/2036990306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e2117b4e0 0x562e2118aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.515+0000 7f096272fb80 1 --2- 172.21.15.112:0/2036990306 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562e211fdab0 0x562e211ffed0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.515+0000 7f096272fb80 1 --2- 172.21.15.112:0/2036990306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e211758f0 0x562e2118f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.515+0000 7f096272fb80 1 --2- 172.21.15.112:0/2036990306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562e2117abc0 0x562e2118fa50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.515+0000 7f096272fb80 1 -- 172.21.15.112:0/2036990306 >> 172.21.15.112:0/2036990306 conn(0x562e210ed6d0 msgr2=0x562e211829f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:09.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.515+0000 7f0937400700 0 client.25222 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:09.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.515+0000 7f096272fb80 1 -- 172.21.15.112:0/2036990306 shutdown_connections 2024-05-08T05:56:09.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.515+0000 7f096272fb80 1 -- 172.21.15.112:0/2036990306 wait complete. 2024-05-08T05:56:09.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.515+0000 7f096272fb80 20 client.25222 trim_cache size 0 max 16384 2024-05-08T05:56:09.518 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.516+0000 7f0957e00700 1 -- 172.21.15.112:0/1136768348 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f09380b2900 msgr2=0x7f09380b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.516+0000 7f0957e00700 1 --2- 172.21.15.112:0/1136768348 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f09380b2900 0x7f09380b4dc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f094800e1d0 tx=0x7f0948009040 comp rx=0 tx=0).stop 2024-05-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.516+0000 7f0957e00700 1 -- 172.21.15.112:0/1136768348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0950103750 msgr2=0x7f095019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.516+0000 7f0957e00700 1 --2- 172.21.15.112:0/1136768348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0950103750 0x7f095019f810 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f094400c010 tx=0x7f094400fa20 comp rx=0 tx=0).stop 2024-05-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.516+0000 7f0957e00700 1 -- 172.21.15.112:0/1136768348 shutdown_connections 2024-05-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.516+0000 7f0957e00700 1 --2- 172.21.15.112:0/1136768348 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f09380b2900 0x7f09380b4dc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.516+0000 7f0957e00700 1 --2- 172.21.15.112:0/1136768348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0950108850 0x7f09501999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.516+0000 7f0957e00700 1 --2- 172.21.15.112:0/1136768348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0950103750 0x7f095019f810 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.516+0000 7f0957e00700 1 --2- 172.21.15.112:0/1136768348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0950104100 0x7f095019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.516+0000 7f0957e00700 1 -- 172.21.15.112:0/1136768348 >> 172.21.15.112:0/1136768348 conn(0x7f095007a600 msgr2=0x7f09501018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.516+0000 7f0957e00700 1 -- 172.21.15.112:0/1136768348 shutdown_connections 2024-05-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:09.517+0000 7f0957e00700 1 -- 172.21.15.112:0/1136768348 wait complete. 2024-05-08T05:56:11.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:10 smithi150 ceph-mon[34315]: pgmap v467: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:56:11.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:10 smithi112 ceph-mon[30003]: pgmap v467: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:56:11.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:10 smithi162 ceph-mon[34800]: pgmap v467: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:56:11.541 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T05:56:11.542 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T05:56:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:13.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:12 smithi150 ceph-mon[34315]: pgmap v468: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:13.294 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:12 smithi112 ceph-mon[30003]: pgmap v468: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:13.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:12 smithi162 ceph-mon[34800]: pgmap v468: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:14.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.539+0000 7fe692600700 1 -- 172.21.15.112:0/402434924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c103d90 msgr2=0x7fe68c1102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:14.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.539+0000 7fe692600700 1 --2- 172.21.15.112:0/402434924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c103d90 0x7fe68c1102a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe684009f90 tx=0x7fe68400b330 comp rx=0 tx=0).stop 2024-05-08T05:56:14.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.539+0000 7fe692600700 1 -- 172.21.15.112:0/402434924 shutdown_connections 2024-05-08T05:56:14.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.539+0000 7fe692600700 1 --2- 172.21.15.112:0/402434924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c103d90 0x7fe68c1102a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.539+0000 7fe692600700 1 --2- 172.21.15.112:0/402434924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe68c109460 0x7fe68c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.539+0000 7fe692600700 1 --2- 172.21.15.112:0/402434924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe68c1033d0 0x7fe68c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.539+0000 7fe692600700 1 -- 172.21.15.112:0/402434924 >> 172.21.15.112:0/402434924 conn(0x7fe68c0fef20 msgr2=0x7fe68c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:14.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.539+0000 7fe692600700 1 -- 172.21.15.112:0/402434924 shutdown_connections 2024-05-08T05:56:14.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.539+0000 7fe692600700 1 -- 172.21.15.112:0/402434924 wait complete. 2024-05-08T05:56:14.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.541+0000 7fe692600700 1 Processor -- start 2024-05-08T05:56:14.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.541+0000 7fe692600700 1 -- start start 2024-05-08T05:56:14.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.541+0000 7fe692600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c1033d0 0x7fe68c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.541+0000 7fe692600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe68c103d90 0x7fe68c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.541+0000 7fe692600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe68c109460 0x7fe68c199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.541+0000 7fe692600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1135c0 con 0x7fe68c1033d0 2024-05-08T05:56:14.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.541+0000 7fe692600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c113440 con 0x7fe68c109460 2024-05-08T05:56:14.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.541+0000 7fe692600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c113740 con 0x7fe68c103d90 2024-05-08T05:56:14.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.542+0000 7fe68be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe68c103d90 0x7fe68c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.542+0000 7fe691c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe68c109460 0x7fe68c199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.542+0000 7fe691200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c1033d0 0x7fe68c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.542+0000 7fe68be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe68c103d90 0x7fe68c19fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:32826/0 (socket says 172.21.15.112:32826) 2024-05-08T05:56:14.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.542+0000 7fe68be00700 1 -- 172.21.15.112:0/1885322670 learned_addr learned my addr 172.21.15.112:0/1885322670 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:14.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.542+0000 7fe68be00700 1 -- 172.21.15.112:0/1885322670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe68c109460 msgr2=0x7fe68c199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:14.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.542+0000 7fe68be00700 1 --2- 172.21.15.112:0/1885322670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe68c109460 0x7fe68c199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.542+0000 7fe68be00700 1 -- 172.21.15.112:0/1885322670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c1033d0 msgr2=0x7fe68c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:14.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.543+0000 7fe68be00700 1 --2- 172.21.15.112:0/1885322670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c1033d0 0x7fe68c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.543+0000 7fe68be00700 1 -- 172.21.15.112:0/1885322670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c199fe0 con 0x7fe68c103d90 2024-05-08T05:56:14.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.543+0000 7fe691200700 1 --2- 172.21.15.112:0/1885322670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c1033d0 0x7fe68c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:14.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.543+0000 7fe68be00700 1 --2- 172.21.15.112:0/1885322670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe68c103d90 0x7fe68c19fc00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe67800e9a0 tx=0x7fe67800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:14.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.543+0000 7fe689600700 1 -- 172.21.15.112:0/1885322670 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67800c9c0 con 0x7fe68c103d90 2024-05-08T05:56:14.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.543+0000 7fe692600700 1 -- 172.21.15.112:0/1885322670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68c19a2d0 con 0x7fe68c103d90 2024-05-08T05:56:14.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.543+0000 7fe692600700 1 -- 172.21.15.112:0/1885322670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68c1a6590 con 0x7fe68c103d90 2024-05-08T05:56:14.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.544+0000 7fe689600700 1 -- 172.21.15.112:0/1885322670 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe67800cb60 con 0x7fe68c103d90 2024-05-08T05:56:14.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.544+0000 7fe689600700 1 -- 172.21.15.112:0/1885322670 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678018680 con 0x7fe68c103d90 2024-05-08T05:56:14.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.547+0000 7fe689600700 1 -- 172.21.15.112:0/1885322670 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe678010430 con 0x7fe68c103d90 2024-05-08T05:56:14.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.549+0000 7fe69e558b80 1 Processor -- start 2024-05-08T05:56:14.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.549+0000 7fe69e558b80 1 -- start start 2024-05-08T05:56:14.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.549+0000 7fe69e558b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e4e70 0x55a1b22e5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.549+0000 7fe69e558b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e5790 0x55a1b22df470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.549+0000 7fe69e558b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 0x55a1b22dff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.549+0000 7fe689600700 1 --2- 172.21.15.112:0/1885322670 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6740b2900 0x7fe6740b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.549+0000 7fe69e558b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a1b222daa0 con 0x55a1b22e4e70 2024-05-08T05:56:14.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.549+0000 7fe69e558b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a1b21cb7e0 con 0x55a1b22dfa90 2024-05-08T05:56:14.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.549+0000 7fe69e558b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a1b2229a60 con 0x55a1b22e5790 2024-05-08T05:56:14.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.550+0000 7fe689600700 1 -- 172.21.15.112:0/1885322670 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fe6780e23c0 con 0x7fe68c103d90 2024-05-08T05:56:14.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.550+0000 7fe68be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 0x55a1b22dff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.550+0000 7fe68be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 0x55a1b22dff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:36424/0 (socket says 172.21.15.112:36424) 2024-05-08T05:56:14.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.550+0000 7fe68be00700 1 -- 172.21.15.112:0/1659559347 learned_addr learned my addr 172.21.15.112:0/1659559347 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:14.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.550+0000 7fe691200700 1 --2- 172.21.15.112:0/1659559347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e5790 0x55a1b22df470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.550+0000 7fe68be00700 1 -- 172.21.15.112:0/1659559347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e5790 msgr2=0x55a1b22df470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:14.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.550+0000 7fe68be00700 1 --2- 172.21.15.112:0/1659559347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e5790 0x55a1b22df470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.550+0000 7fe68be00700 1 -- 172.21.15.112:0/1659559347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e4e70 msgr2=0x55a1b22e5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:14.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.550+0000 7fe68be00700 1 --2- 172.21.15.112:0/1659559347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e4e70 0x55a1b22e5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.550+0000 7fe68be00700 1 -- 172.21.15.112:0/1659559347 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b217a950 con 0x55a1b22dfa90 2024-05-08T05:56:14.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.550+0000 7fe68be00700 1 --2- 172.21.15.112:0/1659559347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 0x55a1b22dff10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe678010c50 tx=0x7fe6780a9850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a7c16112729b261 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:14.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.551+0000 7fe691c00700 1 --2- 172.21.15.112:0/1885322670 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6740b2900 0x7fe6740b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.551+0000 7fe67e000700 1 -- 172.21.15.112:0/1659559347 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6780a9460 con 0x55a1b22dfa90 2024-05-08T05:56:14.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.551+0000 7fe67e000700 1 -- 172.21.15.112:0/1659559347 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6780aa050 con 0x55a1b22dfa90 2024-05-08T05:56:14.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.551+0000 7fe67e000700 1 -- 172.21.15.112:0/1659559347 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6780b2320 con 0x55a1b22dfa90 2024-05-08T05:56:14.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.552+0000 7fe69e558b80 1 -- 172.21.15.112:0/1659559347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 msgr2=0x55a1b22dff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:14.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.552+0000 7fe69e558b80 1 --2- 172.21.15.112:0/1659559347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 0x55a1b22dff10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe678010c50 tx=0x7fe6780a9850 comp rx=0 tx=0).stop 2024-05-08T05:56:14.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.552+0000 7fe691c00700 1 --2- 172.21.15.112:0/1885322670 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6740b2900 0x7fe6740b4dc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe684009790 tx=0x7fe684009d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:14.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.552+0000 7fe69e558b80 1 -- 172.21.15.112:0/1659559347 shutdown_connections 2024-05-08T05:56:14.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.552+0000 7fe69e558b80 1 --2- 172.21.15.112:0/1659559347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 0x55a1b22dff10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.552+0000 7fe69e558b80 1 --2- 172.21.15.112:0/1659559347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e4e70 0x55a1b22e5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.552+0000 7fe69e558b80 1 --2- 172.21.15.112:0/1659559347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e5790 0x55a1b22df470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.552+0000 7fe69e558b80 1 -- 172.21.15.112:0/1659559347 >> 172.21.15.112:0/1659559347 conn(0x55a1b2209be0 msgr2=0x55a1b22d5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:14.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.552+0000 7fe69e558b80 1 -- 172.21.15.112:0/1659559347 shutdown_connections 2024-05-08T05:56:14.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.552+0000 7fe69e558b80 1 -- 172.21.15.112:0/1659559347 wait complete. 2024-05-08T05:56:14.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.553+0000 7fe69e558b80 1 -- start start 2024-05-08T05:56:14.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.553+0000 7fe69e558b80 1 Processor -- start 2024-05-08T05:56:14.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.553+0000 7fe69e558b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 0x55a1b22eb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.553+0000 7fe69e558b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e4e70 0x55a1b22e86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.553+0000 7fe69e558b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e5790 0x55a1b22e8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.553+0000 7fe69e558b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a1b22ec180 con 0x55a1b22e4e70 2024-05-08T05:56:14.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.554+0000 7fe69e558b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a1b2193ba0 con 0x55a1b22dfa90 2024-05-08T05:56:14.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.554+0000 7fe69e558b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a1b22ec300 con 0x55a1b22e5790 2024-05-08T05:56:14.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.554+0000 7fe69e558b80 10 client.0 _ll_register_callbacks cb 0x55a1b21d4cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:14.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.554+0000 7fe691200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 0x55a1b22eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.554+0000 7fe691200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 0x55a1b22eb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:36434/0 (socket says 172.21.15.112:36434) 2024-05-08T05:56:14.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.554+0000 7fe691200700 1 -- 172.21.15.112:0/1246621369 learned_addr learned my addr 172.21.15.112:0/1246621369 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:14.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.554+0000 7fe691c00700 1 --2- 172.21.15.112:0/1246621369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e5790 0x55a1b22e8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.554+0000 7fe68be00700 1 --2- 172.21.15.112:0/1246621369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e4e70 0x55a1b22e86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.555+0000 7fe691200700 1 -- 172.21.15.112:0/1246621369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e5790 msgr2=0x55a1b22e8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:14.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.555+0000 7fe691200700 1 --2- 172.21.15.112:0/1246621369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e5790 0x55a1b22e8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.555+0000 7fe691200700 1 -- 172.21.15.112:0/1246621369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e4e70 msgr2=0x55a1b22e86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:14.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.555+0000 7fe691200700 1 --2- 172.21.15.112:0/1246621369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e4e70 0x55a1b22e86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.555+0000 7fe691200700 1 -- 172.21.15.112:0/1246621369 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b21f34a0 con 0x55a1b22dfa90 2024-05-08T05:56:14.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.555+0000 7fe691c00700 1 --2- 172.21.15.112:0/1246621369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e5790 0x55a1b22e8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:14.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.555+0000 7fe68be00700 1 --2- 172.21.15.112:0/1246621369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e4e70 0x55a1b22e86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:14.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.555+0000 7fe691200700 1 --2- 172.21.15.112:0/1246621369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 0x55a1b22eb340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe6800129d0 tx=0x7fe680012ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73533cc4ee2a1e7c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:14.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.555+0000 7fe672a00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:56:14.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.555+0000 7fe672a00700 1 -- 172.21.15.112:0/1246621369 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680004050 con 0x55a1b22dfa90 2024-05-08T05:56:14.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.555+0000 7fe672a00700 1 -- 172.21.15.112:0/1246621369 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fe6800041f0 con 0x55a1b22dfa90 2024-05-08T05:56:14.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.556+0000 7fe69e558b80 1 -- 172.21.15.112:0/1246621369 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6780aa050 con 0x55a1b22dfa90 2024-05-08T05:56:14.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.556+0000 7fe672a00700 1 -- 172.21.15.112:0/1246621369 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe6800043f0 con 0x55a1b22dfa90 2024-05-08T05:56:14.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.556+0000 7fe69e558b80 10 client.25214 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:14.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.556+0000 7fe69e558b80 1 -- 172.21.15.112:0/1246621369 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a1b22f6920 con 0x55a1b22dfa90 2024-05-08T05:56:14.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.557+0000 7fe672a00700 1 -- 172.21.15.112:0/1246621369 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6780aa050 con 0x55a1b22dfa90 2024-05-08T05:56:14.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.557+0000 7fe672a00700 1 -- 172.21.15.112:0/1246621369 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fe680003a60 con 0x55a1b22dfa90 2024-05-08T05:56:14.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.557+0000 7fe69e558b80 10 client.25214 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:14.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.557+0000 7fe69e558b80 10 client.25214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:56:14.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.557+0000 7fe69e558b80 20 client.25214 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:14.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.557+0000 7fe69e558b80 1 --2- 172.21.15.112:0/1246621369 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a1b22f8f00 0x55a1b22fb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.557+0000 7fe69e558b80 4 client.25214 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:14.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.557+0000 7fe69e558b80 1 -- 172.21.15.112:0/1246621369 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1b22a8120 con 0x55a1b22f8f00 2024-05-08T05:56:14.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.558+0000 7fe68be00700 1 --2- 172.21.15.112:0/1246621369 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a1b22f8f00 0x55a1b22fb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.558+0000 7fe68be00700 1 --2- 172.21.15.112:0/1246621369 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a1b22f8f00 0x55a1b22fb3a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace3def594f8d1ff server_cookie=a388496682f76010 in_seq=0 out_seq=0 2024-05-08T05:56:14.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.558+0000 7fe672a00700 10 client.25214 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:14.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.559+0000 7fe672a00700 1 -- 172.21.15.112:0/1246621369 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a1b22a8120 con 0x55a1b22f8f00 2024-05-08T05:56:14.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.559+0000 7fe672a00700 10 client.25214 handle_command_reply: tid=0 2024-05-08T05:56:14.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.559+0000 7fe69e558b80 1 client.25214 shutdown 2024-05-08T05:56:14.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.560+0000 7fe69e558b80 1 -- 172.21.15.112:0/1246621369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 msgr2=0x55a1b22eb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:14.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.560+0000 7fe69e558b80 1 --2- 172.21.15.112:0/1246621369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 0x55a1b22eb340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe6800129d0 tx=0x7fe680012ce0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.560+0000 7fe69e558b80 1 -- 172.21.15.112:0/1246621369 shutdown_connections 2024-05-08T05:56:14.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.560+0000 7fe69e558b80 1 --2- 172.21.15.112:0/1246621369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e5790 0x55a1b22e8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.560+0000 7fe69e558b80 1 --2- 172.21.15.112:0/1246621369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 0x55a1b22eb340 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.560+0000 7fe69e558b80 1 --2- 172.21.15.112:0/1246621369 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a1b22f8f00 0x55a1b22fb3a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.560+0000 7fe69e558b80 1 --2- 172.21.15.112:0/1246621369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e4e70 0x55a1b22e86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.560+0000 7fe69e558b80 1 -- 172.21.15.112:0/1246621369 >> 172.21.15.112:0/1246621369 conn(0x55a1b2209be0 msgr2=0x55a1b22d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:14.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.560+0000 7fe672a00700 0 client.25214 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:14.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.560+0000 7fe69e558b80 1 -- 172.21.15.112:0/1246621369 shutdown_connections 2024-05-08T05:56:14.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.560+0000 7fe69e558b80 1 -- 172.21.15.112:0/1246621369 wait complete. 2024-05-08T05:56:14.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.560+0000 7fe69e558b80 20 client.25214 trim_cache size 0 max 16384 2024-05-08T05:56:14.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.601+0000 7fe69e558b80 1 Processor -- start 2024-05-08T05:56:14.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.601+0000 7fe69e558b80 1 -- start start 2024-05-08T05:56:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.601+0000 7fe69e558b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22dfa90 0x55a1b22df470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.601+0000 7fe69e558b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e4e70 0x55a1b22f5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.601+0000 7fe69e558b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22e5790 0x55a1b22d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.602+0000 7fe69e558b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c00fda0 con 0x55a1b22dfa90 2024-05-08T05:56:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.602+0000 7fe69e558b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c00d630 con 0x55a1b22e5790 2024-05-08T05:56:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.602+0000 7fe69e558b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c00b480 con 0x55a1b22e4e70 2024-05-08T05:56:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.602+0000 7fe68be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e4e70 0x55a1b22f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.602+0000 7fe68be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e4e70 0x55a1b22f5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:32858/0 (socket says 172.21.15.112:32858) 2024-05-08T05:56:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.602+0000 7fe68be00700 1 -- 172.21.15.112:0/3960772842 learned_addr learned my addr 172.21.15.112:0/3960772842 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.602+0000 7fe691200700 1 --2- 172.21.15.112:0/3960772842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22dfa90 0x55a1b22df470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.602+0000 7fe691c00700 1 --2- 172.21.15.112:0/3960772842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22e5790 0x55a1b22d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.602+0000 7fe68be00700 1 -- 172.21.15.112:0/3960772842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22e5790 msgr2=0x55a1b22d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.602+0000 7fe68be00700 1 --2- 172.21.15.112:0/3960772842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22e5790 0x55a1b22d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.602+0000 7fe68be00700 1 -- 172.21.15.112:0/3960772842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22dfa90 msgr2=0x55a1b22df470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.602+0000 7fe68be00700 1 --2- 172.21.15.112:0/3960772842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22dfa90 0x55a1b22df470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.602+0000 7fe68be00700 1 -- 172.21.15.112:0/3960772842 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b22cf6d0 con 0x55a1b22e4e70 2024-05-08T05:56:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.602+0000 7fe691200700 1 --2- 172.21.15.112:0/3960772842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22dfa90 0x55a1b22df470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.603+0000 7fe68be00700 1 --2- 172.21.15.112:0/3960772842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e4e70 0x55a1b22f5270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe6780a9650 tx=0x7fe678009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87bf808e3c297f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.603+0000 7fe67d600700 1 -- 172.21.15.112:0/3960772842 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6780dfeb0 con 0x55a1b22e4e70 2024-05-08T05:56:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.603+0000 7fe67d600700 1 -- 172.21.15.112:0/3960772842 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6780cc0c0 con 0x55a1b22e4e70 2024-05-08T05:56:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.603+0000 7fe67d600700 1 -- 172.21.15.112:0/3960772842 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67806f0e0 con 0x55a1b22e4e70 2024-05-08T05:56:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.604+0000 7fe69e558b80 1 -- 172.21.15.112:0/3960772842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e4e70 msgr2=0x55a1b22f5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.604+0000 7fe69e558b80 1 --2- 172.21.15.112:0/3960772842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e4e70 0x55a1b22f5270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe6780a9650 tx=0x7fe678009790 comp rx=0 tx=0).stop 2024-05-08T05:56:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.604+0000 7fe69e558b80 1 -- 172.21.15.112:0/3960772842 shutdown_connections 2024-05-08T05:56:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.604+0000 7fe69e558b80 1 --2- 172.21.15.112:0/3960772842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22e5790 0x55a1b22d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.604+0000 7fe69e558b80 1 --2- 172.21.15.112:0/3960772842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22dfa90 0x55a1b22df470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.604+0000 7fe69e558b80 1 --2- 172.21.15.112:0/3960772842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e4e70 0x55a1b22f5270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.604+0000 7fe69e558b80 1 -- 172.21.15.112:0/3960772842 >> 172.21.15.112:0/3960772842 conn(0x55a1b2209be0 msgr2=0x55a1b22cfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.605+0000 7fe69e558b80 1 -- 172.21.15.112:0/3960772842 shutdown_connections 2024-05-08T05:56:14.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.605+0000 7fe69e558b80 1 -- 172.21.15.112:0/3960772842 wait complete. 2024-05-08T05:56:14.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.605+0000 7fe69e558b80 1 -- start start 2024-05-08T05:56:14.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.605+0000 7fe69e558b80 1 Processor -- start 2024-05-08T05:56:14.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.606+0000 7fe69e558b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 0x55a1b22f15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.606+0000 7fe69e558b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e4e70 0x55a1b22f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.606+0000 7fe69e558b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e5790 0x55a1b22da050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.606+0000 7fe69e558b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c00baf0 con 0x55a1b22e5790 2024-05-08T05:56:14.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.606+0000 7fe69e558b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c00cf60 con 0x55a1b22dfa90 2024-05-08T05:56:14.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.606+0000 7fe69e558b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c00c1c0 con 0x55a1b22e4e70 2024-05-08T05:56:14.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.606+0000 7fe69e558b80 10 client.0 _ll_register_callbacks cb 0x55a1b21d4cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:14.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.607+0000 7fe691c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e5790 0x55a1b22da050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.607+0000 7fe691c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e5790 0x55a1b22da050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45748/0 (socket says 172.21.15.112:45748) 2024-05-08T05:56:14.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.607+0000 7fe691c00700 1 -- 172.21.15.112:0/710950840 learned_addr learned my addr 172.21.15.112:0/710950840 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:14.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.607+0000 7fe691c00700 1 -- 172.21.15.112:0/710950840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e4e70 msgr2=0x55a1b22f1b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:56:14.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.607+0000 7fe691c00700 1 --2- 172.21.15.112:0/710950840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e4e70 0x55a1b22f1b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.607+0000 7fe691c00700 1 -- 172.21.15.112:0/710950840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 msgr2=0x55a1b22f15f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:56:14.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.607+0000 7fe691c00700 1 --2- 172.21.15.112:0/710950840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 0x55a1b22f15f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.607+0000 7fe691c00700 1 -- 172.21.15.112:0/710950840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b22da590 con 0x55a1b22e5790 2024-05-08T05:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.608+0000 7fe691c00700 1 --2- 172.21.15.112:0/710950840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e5790 0x55a1b22da050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe68400bf40 tx=0x7fe68400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ef2211c98f6968 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.608+0000 7fe672a00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.609+0000 7fe672a00700 1 -- 172.21.15.112:0/710950840 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684007cf0 con 0x55a1b22e5790 2024-05-08T05:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.609+0000 7fe672a00700 1 -- 172.21.15.112:0/710950840 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fe684034440 con 0x55a1b22e5790 2024-05-08T05:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.609+0000 7fe69e558b80 1 -- 172.21.15.112:0/710950840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6780cc0c0 con 0x55a1b22e5790 2024-05-08T05:56:14.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.609+0000 7fe672a00700 1 -- 172.21.15.112:0/710950840 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe684034640 con 0x55a1b22e5790 2024-05-08T05:56:14.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.609+0000 7fe69e558b80 10 client.15102 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:14.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.609+0000 7fe69e558b80 1 -- 172.21.15.112:0/710950840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a1b2368870 con 0x55a1b22e5790 2024-05-08T05:56:14.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.610+0000 7fe672a00700 1 -- 172.21.15.112:0/710950840 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6780cc0c0 con 0x55a1b22e5790 2024-05-08T05:56:14.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.610+0000 7fe672a00700 1 -- 172.21.15.112:0/710950840 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fe684032020 con 0x55a1b22e5790 2024-05-08T05:56:14.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.610+0000 7fe69e558b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:14.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.610+0000 7fe69e558b80 10 client.15102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:56:14.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.611+0000 7fe69e558b80 20 client.15102 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:14.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.611+0000 7fe69e558b80 1 --2- 172.21.15.112:0/710950840 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a1b2368f60 0x55a1b236b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.611+0000 7fe69e558b80 4 client.15102 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T05:56:14.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.611+0000 7fe69e558b80 1 -- 172.21.15.112:0/710950840 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a1b223b480 con 0x55a1b2368f60 2024-05-08T05:56:14.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.614+0000 7fe691200700 1 --2- 172.21.15.112:0/710950840 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a1b2368f60 0x55a1b236b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.615+0000 7fe691200700 1 --2- 172.21.15.112:0/710950840 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a1b2368f60 0x55a1b236b3a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=824f96dc6536c28 server_cookie=d68c29fdbe0489fc in_seq=0 out_seq=0 2024-05-08T05:56:14.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.615+0000 7fe672a00700 10 client.15102 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:14.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.616+0000 7fe672a00700 1 -- 172.21.15.112:0/710950840 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a1b223b480 con 0x55a1b2368f60 2024-05-08T05:56:14.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.616+0000 7fe672a00700 10 client.15102 handle_command_reply: tid=0 2024-05-08T05:56:14.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.617+0000 7fe69e558b80 1 client.15102 shutdown 2024-05-08T05:56:14.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.617+0000 7fe69e558b80 1 -- 172.21.15.112:0/710950840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e5790 msgr2=0x55a1b22da050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:14.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.617+0000 7fe69e558b80 1 --2- 172.21.15.112:0/710950840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e5790 0x55a1b22da050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe68400bf40 tx=0x7fe68400b330 comp rx=0 tx=0).stop 2024-05-08T05:56:14.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.617+0000 7fe69e558b80 1 -- 172.21.15.112:0/710950840 shutdown_connections 2024-05-08T05:56:14.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.617+0000 7fe69e558b80 1 --2- 172.21.15.112:0/710950840 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55a1b2368f60 0x55a1b236b3a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.617+0000 7fe69e558b80 1 --2- 172.21.15.112:0/710950840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a1b22e5790 0x55a1b22da050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.617+0000 7fe69e558b80 1 --2- 172.21.15.112:0/710950840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1b22dfa90 0x55a1b22f15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.617+0000 7fe69e558b80 1 --2- 172.21.15.112:0/710950840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a1b22e4e70 0x55a1b22f1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.617+0000 7fe69e558b80 1 -- 172.21.15.112:0/710950840 >> 172.21.15.112:0/710950840 conn(0x55a1b2209be0 msgr2=0x55a1b22e6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:14.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.618+0000 7fe672a00700 0 client.15102 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:14.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.619+0000 7fe69e558b80 1 -- 172.21.15.112:0/710950840 shutdown_connections 2024-05-08T05:56:14.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.619+0000 7fe69e558b80 1 -- 172.21.15.112:0/710950840 wait complete. 2024-05-08T05:56:14.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.619+0000 7fe69e558b80 20 client.15102 trim_cache size 0 max 16384 2024-05-08T05:56:14.620 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:56:14.621 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-08T05:56:14.621 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "b7682acd-3b28-47df-8c1f-75a551d78622", 2024-05-08T05:56:14.621 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-08T05:56:14.621 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:56:14.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.621+0000 7fe692600700 1 -- 172.21.15.112:0/1885322670 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6740b2900 msgr2=0x7fe6740b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:14.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.621+0000 7fe692600700 1 --2- 172.21.15.112:0/1885322670 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6740b2900 0x7fe6740b4dc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe684009790 tx=0x7fe684009d20 comp rx=0 tx=0).stop 2024-05-08T05:56:14.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.621+0000 7fe692600700 1 -- 172.21.15.112:0/1885322670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe68c103d90 msgr2=0x7fe68c19fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:14.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.621+0000 7fe692600700 1 --2- 172.21.15.112:0/1885322670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe68c103d90 0x7fe68c19fc00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe67800e9a0 tx=0x7fe67800ecb0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.621+0000 7fe692600700 1 -- 172.21.15.112:0/1885322670 shutdown_connections 2024-05-08T05:56:14.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.621+0000 7fe692600700 1 --2- 172.21.15.112:0/1885322670 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6740b2900 0x7fe6740b4dc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.621+0000 7fe692600700 1 --2- 172.21.15.112:0/1885322670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe68c109460 0x7fe68c199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.621+0000 7fe692600700 1 --2- 172.21.15.112:0/1885322670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c1033d0 0x7fe68c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.621+0000 7fe692600700 1 --2- 172.21.15.112:0/1885322670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe68c103d90 0x7fe68c19fc00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.621+0000 7fe692600700 1 -- 172.21.15.112:0/1885322670 >> 172.21.15.112:0/1885322670 conn(0x7fe68c0fef20 msgr2=0x7fe68c0ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:14.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.621+0000 7fe692600700 1 -- 172.21.15.112:0/1885322670 shutdown_connections 2024-05-08T05:56:14.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:14.621+0000 7fe692600700 1 -- 172.21.15.112:0/1885322670 wait complete. 2024-05-08T05:56:14.888 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:14 smithi112 ceph-mon[30003]: pgmap v469: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:56:15.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:14 smithi150 ceph-mon[34315]: pgmap v469: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:56:15.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:14 smithi162 ceph-mon[34800]: pgmap v469: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:56:15.473 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T05:56:15.958 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:16.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:15 smithi150 ceph-mon[34315]: scrub queued for path: / 2024-05-08T05:56:16.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:15 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:56:16.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:15 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T05:56:16.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:15 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:56:16.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:15 smithi112 ceph-mon[30003]: scrub queued for path: / 2024-05-08T05:56:16.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:15 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:56:16.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:15 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T05:56:16.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:15 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:56:16.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:15 smithi162 ceph-mon[34800]: scrub queued for path: / 2024-05-08T05:56:16.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:15 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:56:16.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:15 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T05:56:16.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:15 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:56:17.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:16 smithi150 ceph-mon[34315]: pgmap v470: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:56:17.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:16 smithi112 ceph-mon[30003]: pgmap v470: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:56:17.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:16 smithi162 ceph-mon[34800]: pgmap v470: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:56:19.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:18 smithi150 ceph-mon[34315]: pgmap v471: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:19.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:18 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T05:56:19.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:18 smithi112 ceph-mon[30003]: pgmap v471: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:19.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:18 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T05:56:19.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:18 smithi162 ceph-mon[34800]: pgmap v471: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:19.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:18 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T05:56:21.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.152+0000 7f5739200700 1 -- 172.21.15.112:0/800481843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f573410b680 msgr2=0x7f573410ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:21.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.152+0000 7f5739200700 1 --2- 172.21.15.112:0/800481843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f573410b680 0x7f573410ba60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f57240099c0 tx=0x7f5724009cd0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.152+0000 7f5739200700 1 -- 172.21.15.112:0/800481843 shutdown_connections 2024-05-08T05:56:21.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.152+0000 7f5739200700 1 --2- 172.21.15.112:0/800481843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5734105f90 0x7f5734112450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.152+0000 7f5739200700 1 --2- 172.21.15.112:0/800481843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f573410b680 0x7f573410ba60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.152+0000 7f5739200700 1 --2- 172.21.15.112:0/800481843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57341055d0 0x7f5734105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.152+0000 7f5739200700 1 -- 172.21.15.112:0/800481843 >> 172.21.15.112:0/800481843 conn(0x7f5734101080 msgr2=0x7f57341034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:21.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.152+0000 7f5739200700 1 -- 172.21.15.112:0/800481843 shutdown_connections 2024-05-08T05:56:21.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.152+0000 7f5739200700 1 -- 172.21.15.112:0/800481843 wait complete. 2024-05-08T05:56:21.154 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:20 smithi112 ceph-mon[30003]: pgmap v472: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:56:21.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.154+0000 7f5739200700 1 Processor -- start 2024-05-08T05:56:21.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.154+0000 7f5739200700 1 -- start start 2024-05-08T05:56:21.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.154+0000 7f5739200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57341055d0 0x7f5734112340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.154+0000 7f5739200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5734105f90 0x7f573410f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.154+0000 7f5739200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f573410b680 0x7f573410f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.154+0000 7f5739200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57341157d0 con 0x7f57341055d0 2024-05-08T05:56:21.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.154+0000 7f5739200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5734115650 con 0x7f573410b680 2024-05-08T05:56:21.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.154+0000 7f5739200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5734115950 con 0x7f5734105f90 2024-05-08T05:56:21.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.155+0000 7f5733e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f573410b680 0x7f573410f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.155+0000 7f5733e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f573410b680 0x7f573410f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:36470/0 (socket says 172.21.15.112:36470) 2024-05-08T05:56:21.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.155+0000 7f5733e00700 1 -- 172.21.15.112:0/816212073 learned_addr learned my addr 172.21.15.112:0/816212073 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:21.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.155+0000 7f5732a00700 1 --2- 172.21.15.112:0/816212073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5734105f90 0x7f573410f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.155+0000 7f5733e00700 1 -- 172.21.15.112:0/816212073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5734105f90 msgr2=0x7f573410f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:21.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.156+0000 7f5733e00700 1 --2- 172.21.15.112:0/816212073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5734105f90 0x7f573410f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.156+0000 7f5733400700 1 --2- 172.21.15.112:0/816212073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57341055d0 0x7f5734112340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.156+0000 7f5733e00700 1 -- 172.21.15.112:0/816212073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57341055d0 msgr2=0x7f5734112340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:21.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.156+0000 7f5733e00700 1 --2- 172.21.15.112:0/816212073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57341055d0 0x7f5734112340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.156+0000 7f5733e00700 1 -- 172.21.15.112:0/816212073 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57341a1cd0 con 0x7f573410b680 2024-05-08T05:56:21.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.157+0000 7f5732a00700 1 --2- 172.21.15.112:0/816212073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5734105f90 0x7f573410f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:21.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.157+0000 7f5733e00700 1 --2- 172.21.15.112:0/816212073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f573410b680 0x7f573410f990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f572c00ef30 tx=0x7f572c00c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:21.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.157+0000 7f572be00700 1 -- 172.21.15.112:0/816212073 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c0092e0 con 0x7f573410b680 2024-05-08T05:56:21.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.157+0000 7f572be00700 1 -- 172.21.15.112:0/816212073 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f572c004bb0 con 0x7f573410b680 2024-05-08T05:56:21.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.157+0000 7f572be00700 1 -- 172.21.15.112:0/816212073 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c004280 con 0x7f573410b680 2024-05-08T05:56:21.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.157+0000 7f5739200700 1 -- 172.21.15.112:0/816212073 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57341a1f60 con 0x7f573410b680 2024-05-08T05:56:21.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.158+0000 7f5739200700 1 -- 172.21.15.112:0/816212073 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57341a2450 con 0x7f573410b680 2024-05-08T05:56:21.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.164+0000 7f572be00700 1 -- 172.21.15.112:0/816212073 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f572c01b030 con 0x7f573410b680 2024-05-08T05:56:21.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.164+0000 7f574527cb80 1 Processor -- start 2024-05-08T05:56:21.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.164+0000 7f574527cb80 1 -- start start 2024-05-08T05:56:21.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.165+0000 7f574527cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c69013c9e0 0x55c69013cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.165+0000 7f574527cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013d300 0x55c690136fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.165+0000 7f574527cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 0x55c690137a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.166+0000 7f574527cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c690089a20 con 0x55c69013d300 2024-05-08T05:56:21.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.166+0000 7f574527cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c690027760 con 0x55c690137600 2024-05-08T05:56:21.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.166+0000 7f574527cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c6900859e0 con 0x55c69013c9e0 2024-05-08T05:56:21.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.166+0000 7f572be00700 1 --2- 172.21.15.112:0/816212073 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f57180b2a50 0x7f57180b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.166+0000 7f5733400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013d300 0x55c690136fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.166+0000 7f5733400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013d300 0x55c690136fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45772/0 (socket says 172.21.15.112:45772) 2024-05-08T05:56:21.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.166+0000 7f5732a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 0x55c690137a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.166+0000 7f5733400700 1 -- 172.21.15.112:0/4233517500 learned_addr learned my addr 172.21.15.112:0/4233517500 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:21.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.166+0000 7f572be00700 1 -- 172.21.15.112:0/816212073 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f572c0e1df0 con 0x7f573410b680 2024-05-08T05:56:21.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.166+0000 7f5732a00700 1 --2- 172.21.15.112:0/4233517500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c69013c9e0 0x55c69013cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.167+0000 7f5733400700 1 -- 172.21.15.112:0/4233517500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c69013c9e0 msgr2=0x55c69013cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:21.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.167+0000 7f5733400700 1 --2- 172.21.15.112:0/4233517500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c69013c9e0 0x55c69013cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.167+0000 7f5733400700 1 -- 172.21.15.112:0/4233517500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 msgr2=0x55c690137a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:21.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.167+0000 7f5733400700 1 --2- 172.21.15.112:0/4233517500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 0x55c690137a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.167+0000 7f5733400700 1 -- 172.21.15.112:0/4233517500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c68ffd68d0 con 0x55c69013d300 2024-05-08T05:56:21.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.167+0000 7f5732a00700 1 --2- 172.21.15.112:0/4233517500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 0x55c690137a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:21.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.168+0000 7f5733400700 1 --2- 172.21.15.112:0/4233517500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013d300 0x55c690136fe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5724005f40 tx=0x7f5724005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6278e592546dbf74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:21.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.168+0000 7f5733e00700 1 --2- 172.21.15.112:0/816212073 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f57180b2a50 0x7f57180b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.168+0000 7f5728c00700 1 -- 172.21.15.112:0/4233517500 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724026070 con 0x55c69013d300 2024-05-08T05:56:21.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.169+0000 7f5728c00700 1 -- 172.21.15.112:0/4233517500 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5724009590 con 0x55c69013d300 2024-05-08T05:56:21.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.169+0000 7f5728c00700 1 -- 172.21.15.112:0/4233517500 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724026070 con 0x55c69013d300 2024-05-08T05:56:21.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.170+0000 7f574527cb80 1 -- 172.21.15.112:0/4233517500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013d300 msgr2=0x55c690136fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:21.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.170+0000 7f5733e00700 1 --2- 172.21.15.112:0/816212073 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f57180b2a50 0x7f57180b4f10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f572c0a93a0 tx=0x7f572c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:21.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.170+0000 7f574527cb80 1 --2- 172.21.15.112:0/4233517500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013d300 0x55c690136fe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5724005f40 tx=0x7f5724005c00 comp rx=0 tx=0).stop 2024-05-08T05:56:21.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.170+0000 7f574527cb80 1 -- 172.21.15.112:0/4233517500 shutdown_connections 2024-05-08T05:56:21.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.170+0000 7f574527cb80 1 --2- 172.21.15.112:0/4233517500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 0x55c690137a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.170+0000 7f574527cb80 1 --2- 172.21.15.112:0/4233517500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c69013c9e0 0x55c69013cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.170+0000 7f574527cb80 1 --2- 172.21.15.112:0/4233517500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013d300 0x55c690136fe0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.170+0000 7f574527cb80 1 -- 172.21.15.112:0/4233517500 >> 172.21.15.112:0/4233517500 conn(0x55c690065b60 msgr2=0x55c6901319b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:21.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.170+0000 7f574527cb80 1 -- 172.21.15.112:0/4233517500 shutdown_connections 2024-05-08T05:56:21.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.170+0000 7f574527cb80 1 -- 172.21.15.112:0/4233517500 wait complete. 2024-05-08T05:56:21.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.171+0000 7f574527cb80 1 -- start start 2024-05-08T05:56:21.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.171+0000 7f574527cb80 1 Processor -- start 2024-05-08T05:56:21.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.172+0000 7f574527cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 0x55c690142e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.172+0000 7f574527cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013c9e0 0x55c6901433d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.172+0000 7f574527cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c69013d300 0x55c690140430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.172+0000 7f574527cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c690143d50 con 0x55c69013c9e0 2024-05-08T05:56:21.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.172+0000 7f574527cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c68ffefb20 con 0x55c690137600 2024-05-08T05:56:21.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.172+0000 7f574527cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c690143ed0 con 0x55c69013d300 2024-05-08T05:56:21.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.172+0000 7f574527cb80 10 client.0 _ll_register_callbacks cb 0x55c690030c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:21.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.173+0000 7f5732a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 0x55c690142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.173+0000 7f5733400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013c9e0 0x55c6901433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.173+0000 7f5733400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013c9e0 0x55c6901433d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45788/0 (socket says 172.21.15.112:45788) 2024-05-08T05:56:21.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.173+0000 7f5733400700 1 -- 172.21.15.112:0/583539498 learned_addr learned my addr 172.21.15.112:0/583539498 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:21.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.173+0000 7f5732a00700 1 --2- 172.21.15.112:0/583539498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c69013d300 0x55c690140430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.174+0000 7f5733400700 1 -- 172.21.15.112:0/583539498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c69013d300 msgr2=0x55c690140430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:21.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.174+0000 7f5733400700 1 --2- 172.21.15.112:0/583539498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c69013d300 0x55c690140430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.174+0000 7f5733400700 1 -- 172.21.15.112:0/583539498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 msgr2=0x55c690142e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:21.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.174+0000 7f5733400700 1 --2- 172.21.15.112:0/583539498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 0x55c690142e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.174+0000 7f5733400700 1 -- 172.21.15.112:0/583539498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c69004f420 con 0x55c69013c9e0 2024-05-08T05:56:21.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.174+0000 7f5732a00700 1 --2- 172.21.15.112:0/583539498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c69013d300 0x55c690140430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:21.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.174+0000 7f5733400700 1 --2- 172.21.15.112:0/583539498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013c9e0 0x55c6901433d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5724005910 tx=0x7f5724009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fe893665cae67e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:21.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.174+0000 7f571d600700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:56:21.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.174+0000 7f571d600700 1 -- 172.21.15.112:0/583539498 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572402f060 con 0x55c69013c9e0 2024-05-08T05:56:21.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.175+0000 7f571d600700 1 -- 172.21.15.112:0/583539498 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f57240054f0 con 0x55c69013c9e0 2024-05-08T05:56:21.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.175+0000 7f574527cb80 1 -- 172.21.15.112:0/583539498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5724009590 con 0x55c69013c9e0 2024-05-08T05:56:21.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.175+0000 7f571d600700 1 -- 172.21.15.112:0/583539498 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5724026040 con 0x55c69013c9e0 2024-05-08T05:56:21.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.175+0000 7f574527cb80 10 client.15111 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:21.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.175+0000 7f574527cb80 1 -- 172.21.15.112:0/583539498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c69014e230 con 0x55c69013c9e0 2024-05-08T05:56:21.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.175+0000 7f571d600700 1 -- 172.21.15.112:0/583539498 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5724009590 con 0x55c69013c9e0 2024-05-08T05:56:21.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.176+0000 7f571d600700 1 -- 172.21.15.112:0/583539498 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5724029030 con 0x55c69013c9e0 2024-05-08T05:56:21.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.177+0000 7f574527cb80 10 client.15111 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:21.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.177+0000 7f574527cb80 10 client.15111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:56:21.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.177+0000 7f574527cb80 20 client.15111 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:21.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.177+0000 7f574527cb80 1 --2- 172.21.15.112:0/583539498 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55c690150700 0x55c690152af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.177+0000 7f574527cb80 4 client.15111 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:21.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.177+0000 7f574527cb80 1 -- 172.21.15.112:0/583539498 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6901040a0 con 0x55c690150700 2024-05-08T05:56:21.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.177+0000 7f5732a00700 1 --2- 172.21.15.112:0/583539498 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55c690150700 0x55c690152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.178+0000 7f5732a00700 1 --2- 172.21.15.112:0/583539498 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55c690150700 0x55c690152af0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10cb349fd6ab6921 server_cookie=934b9e25c5cb8b9e in_seq=0 out_seq=0 2024-05-08T05:56:21.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.178+0000 7f571d600700 10 client.15111 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:21.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.179+0000 7f571d600700 1 -- 172.21.15.112:0/583539498 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c6901040a0 con 0x55c690150700 2024-05-08T05:56:21.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.179+0000 7f571d600700 10 client.15111 handle_command_reply: tid=0 2024-05-08T05:56:21.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.180+0000 7f574527cb80 1 client.15111 shutdown 2024-05-08T05:56:21.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.180+0000 7f574527cb80 1 -- 172.21.15.112:0/583539498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013c9e0 msgr2=0x55c6901433d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:21.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.180+0000 7f574527cb80 1 --2- 172.21.15.112:0/583539498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013c9e0 0x55c6901433d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5724005910 tx=0x7f5724009310 comp rx=0 tx=0).stop 2024-05-08T05:56:21.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.180+0000 7f574527cb80 1 -- 172.21.15.112:0/583539498 shutdown_connections 2024-05-08T05:56:21.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.180+0000 7f574527cb80 1 --2- 172.21.15.112:0/583539498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c69013d300 0x55c690140430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.180+0000 7f574527cb80 1 --2- 172.21.15.112:0/583539498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 0x55c690142e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.180+0000 7f574527cb80 1 --2- 172.21.15.112:0/583539498 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55c690150700 0x55c690152af0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.181+0000 7f571d600700 0 client.15111 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:21.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.181+0000 7f574527cb80 1 --2- 172.21.15.112:0/583539498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013c9e0 0x55c6901433d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.181+0000 7f574527cb80 1 -- 172.21.15.112:0/583539498 >> 172.21.15.112:0/583539498 conn(0x55c690065b60 msgr2=0x55c69012c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:21.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.181+0000 7f574527cb80 1 -- 172.21.15.112:0/583539498 shutdown_connections 2024-05-08T05:56:21.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.181+0000 7f574527cb80 1 -- 172.21.15.112:0/583539498 wait complete. 2024-05-08T05:56:21.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.182+0000 7f574527cb80 20 client.15111 trim_cache size 0 max 16384 2024-05-08T05:56:21.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:20 smithi150 ceph-mon[34315]: pgmap v472: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:56:21.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.223+0000 7f574527cb80 1 Processor -- start 2024-05-08T05:56:21.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.223+0000 7f574527cb80 1 -- start start 2024-05-08T05:56:21.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.223+0000 7f574527cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c690137600 0x55c690136fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.223+0000 7f574527cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c69013c9e0 0x55c69013eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.223+0000 7f574527cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013d300 0x55c69013f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.223+0000 7f574527cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57341bf100 con 0x55c69013d300 2024-05-08T05:56:21.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.223+0000 7f574527cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f57341bc990 con 0x55c69013c9e0 2024-05-08T05:56:21.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.223+0000 7f574527cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f57341ba7e0 con 0x55c690137600 2024-05-08T05:56:21.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.224+0000 7f5732a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c690137600 0x55c690136fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.224+0000 7f5732a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c690137600 0x55c690136fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:32912/0 (socket says 172.21.15.112:32912) 2024-05-08T05:56:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.224+0000 7f5732a00700 1 -- 172.21.15.112:0/3911305249 learned_addr learned my addr 172.21.15.112:0/3911305249 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.224+0000 7f5732a00700 1 --2- 172.21.15.112:0/3911305249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013d300 0x55c69013f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.224+0000 7f5733400700 1 --2- 172.21.15.112:0/3911305249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c69013c9e0 0x55c69013eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.225+0000 7f5732a00700 1 -- 172.21.15.112:0/3911305249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c690137600 msgr2=0x55c690136fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.225+0000 7f5732a00700 1 --2- 172.21.15.112:0/3911305249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c690137600 0x55c690136fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.225+0000 7f5732a00700 1 -- 172.21.15.112:0/3911305249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c69013c9e0 msgr2=0x55c69013eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.225+0000 7f5732a00700 1 --2- 172.21.15.112:0/3911305249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c69013c9e0 0x55c69013eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.225+0000 7f5732a00700 1 -- 172.21.15.112:0/3911305249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c690143720 con 0x55c69013d300 2024-05-08T05:56:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.225+0000 7f5732a00700 1 --2- 172.21.15.112:0/3911305249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013d300 0x55c69013f080 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5720005810 tx=0x7f5720016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370224207f1bce9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.226+0000 7f571fe00700 1 -- 172.21.15.112:0/3911305249 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5720004460 con 0x55c69013d300 2024-05-08T05:56:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.226+0000 7f571fe00700 1 -- 172.21.15.112:0/3911305249 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5720003b30 con 0x55c69013d300 2024-05-08T05:56:21.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.226+0000 7f571fe00700 1 -- 172.21.15.112:0/3911305249 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5720024a10 con 0x55c69013d300 2024-05-08T05:56:21.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.227+0000 7f574527cb80 1 -- 172.21.15.112:0/3911305249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013d300 msgr2=0x55c69013f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:21.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.227+0000 7f574527cb80 1 --2- 172.21.15.112:0/3911305249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013d300 0x55c69013f080 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5720005810 tx=0x7f5720016040 comp rx=0 tx=0).stop 2024-05-08T05:56:21.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.227+0000 7f574527cb80 1 -- 172.21.15.112:0/3911305249 shutdown_connections 2024-05-08T05:56:21.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.227+0000 7f574527cb80 1 --2- 172.21.15.112:0/3911305249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013d300 0x55c69013f080 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.227+0000 7f574527cb80 1 --2- 172.21.15.112:0/3911305249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c690137600 0x55c690136fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.227+0000 7f574527cb80 1 --2- 172.21.15.112:0/3911305249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c69013c9e0 0x55c69013eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.227+0000 7f574527cb80 1 -- 172.21.15.112:0/3911305249 >> 172.21.15.112:0/3911305249 conn(0x55c690065b60 msgr2=0x55c69012b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:21.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.227+0000 7f574527cb80 1 -- 172.21.15.112:0/3911305249 shutdown_connections 2024-05-08T05:56:21.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.227+0000 7f574527cb80 1 -- 172.21.15.112:0/3911305249 wait complete. 2024-05-08T05:56:21.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.228+0000 7f574527cb80 1 -- start start 2024-05-08T05:56:21.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.228+0000 7f574527cb80 1 Processor -- start 2024-05-08T05:56:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.229+0000 7f574527cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 0x55c690153a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.229+0000 7f574527cb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013c9e0 0x55c690153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.229+0000 7f574527cb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c690154660 0x55c6901b9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.229+0000 7f574527cb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57341bae50 con 0x55c69013c9e0 2024-05-08T05:56:21.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.229+0000 7f574527cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f57341bc2c0 con 0x55c690137600 2024-05-08T05:56:21.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.229+0000 7f574527cb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f57341bb520 con 0x55c690154660 2024-05-08T05:56:21.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.229+0000 7f574527cb80 10 client.0 _ll_register_callbacks cb 0x55c690030c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:21.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.230+0000 7f5732a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 0x55c690153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.230+0000 7f5732a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 0x55c690153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:36496/0 (socket says 172.21.15.112:36496) 2024-05-08T05:56:21.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.230+0000 7f5732a00700 1 -- 172.21.15.112:0/3450906796 learned_addr learned my addr 172.21.15.112:0/3450906796 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:21.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.230+0000 7f5733400700 1 --2- 172.21.15.112:0/3450906796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013c9e0 0x55c690153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.230+0000 7f5732a00700 1 --2- 172.21.15.112:0/3450906796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c690154660 0x55c6901b9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.230+0000 7f5732a00700 1 -- 172.21.15.112:0/3450906796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c690154660 msgr2=0x55c6901b9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:21.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.230+0000 7f5732a00700 1 --2- 172.21.15.112:0/3450906796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c690154660 0x55c6901b9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.230+0000 7f5732a00700 1 -- 172.21.15.112:0/3450906796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013c9e0 msgr2=0x55c690153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:21.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.231+0000 7f5732a00700 1 --2- 172.21.15.112:0/3450906796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013c9e0 0x55c690153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.231+0000 7f5732a00700 1 -- 172.21.15.112:0/3450906796 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6901ba460 con 0x55c690137600 2024-05-08T05:56:21.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.231+0000 7f5733400700 1 --2- 172.21.15.112:0/3450906796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013c9e0 0x55c690153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:21.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.231+0000 7f5732a00700 1 --2- 172.21.15.112:0/3450906796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 0x55c690153a80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5720005810 tx=0x7f57200095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b452b2dc719701c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:21.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.231+0000 7f571d600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:56:21.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.232+0000 7f571d600700 1 -- 172.21.15.112:0/3450906796 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5720003880 con 0x55c690137600 2024-05-08T05:56:21.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.232+0000 7f571d600700 1 -- 172.21.15.112:0/3450906796 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f5720005030 con 0x55c690137600 2024-05-08T05:56:21.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.232+0000 7f574527cb80 1 -- 172.21.15.112:0/3450906796 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5720003b30 con 0x55c690137600 2024-05-08T05:56:21.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.232+0000 7f571d600700 1 -- 172.21.15.112:0/3450906796 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5720003e60 con 0x55c690137600 2024-05-08T05:56:21.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.232+0000 7f574527cb80 10 client.25235 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:21.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.232+0000 7f574527cb80 1 -- 172.21.15.112:0/3450906796 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c6901bb770 con 0x55c690137600 2024-05-08T05:56:21.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.233+0000 7f571d600700 1 -- 172.21.15.112:0/3450906796 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5720003b30 con 0x55c690137600 2024-05-08T05:56:21.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.233+0000 7f571d600700 1 -- 172.21.15.112:0/3450906796 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5720024460 con 0x55c690137600 2024-05-08T05:56:21.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.233+0000 7f574527cb80 10 client.25235 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:21.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.233+0000 7f574527cb80 10 client.25235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:56:21.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.233+0000 7f574527cb80 20 client.25235 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:21.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.233+0000 7f574527cb80 1 --2- 172.21.15.112:0/3450906796 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55c6901bbca0 0x55c6901be0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:21.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.233+0000 7f574527cb80 4 client.25235 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:56:21.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.233+0000 7f574527cb80 1 -- 172.21.15.112:0/3450906796 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c690097400 con 0x55c6901bbca0 2024-05-08T05:56:21.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.234+0000 7f5733400700 1 --2- 172.21.15.112:0/3450906796 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55c6901bbca0 0x55c6901be0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:21.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.234+0000 7f5733400700 1 --2- 172.21.15.112:0/3450906796 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55c6901bbca0 0x55c6901be0c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb42036968d6c644 server_cookie=f85386c98621e39a in_seq=0 out_seq=0 2024-05-08T05:56:21.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.234+0000 7f571d600700 10 client.25235 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:21.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.235+0000 7f571d600700 1 -- 172.21.15.112:0/3450906796 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c690097400 con 0x55c6901bbca0 2024-05-08T05:56:21.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.235+0000 7f571d600700 10 client.25235 handle_command_reply: tid=0 2024-05-08T05:56:21.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.235+0000 7f574527cb80 1 client.25235 shutdown 2024-05-08T05:56:21.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.235+0000 7f574527cb80 1 -- 172.21.15.112:0/3450906796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 msgr2=0x55c690153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:21.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.235+0000 7f574527cb80 1 --2- 172.21.15.112:0/3450906796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 0x55c690153a80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5720005810 tx=0x7f57200095c0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.235+0000 7f574527cb80 1 -- 172.21.15.112:0/3450906796 shutdown_connections 2024-05-08T05:56:21.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.235+0000 7f574527cb80 1 --2- 172.21.15.112:0/3450906796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c690154660 0x55c6901b9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.235+0000 7f574527cb80 1 --2- 172.21.15.112:0/3450906796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c690137600 0x55c690153a80 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5720005810 tx=0x7f57200095c0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.235+0000 7f574527cb80 1 --2- 172.21.15.112:0/3450906796 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55c6901bbca0 0x55c6901be0c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.235+0000 7f574527cb80 1 --2- 172.21.15.112:0/3450906796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c69013c9e0 0x55c690153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.235+0000 7f574527cb80 1 -- 172.21.15.112:0/3450906796 >> 172.21.15.112:0/3450906796 conn(0x55c690065b60 msgr2=0x55c69013eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:21.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.235+0000 7f571d600700 0 client.25235 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:21.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.236+0000 7f574527cb80 1 -- 172.21.15.112:0/3450906796 shutdown_connections 2024-05-08T05:56:21.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.236+0000 7f574527cb80 1 -- 172.21.15.112:0/3450906796 wait complete. 2024-05-08T05:56:21.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.237+0000 7f574527cb80 20 client.25235 trim_cache size 0 max 16384 2024-05-08T05:56:21.238 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:56:21.238 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-08T05:56:21.238 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-08T05:56:21.238 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:56:21.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.237+0000 7f5739200700 1 -- 172.21.15.112:0/816212073 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f57180b2a50 msgr2=0x7f57180b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:21.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.237+0000 7f5739200700 1 --2- 172.21.15.112:0/816212073 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f57180b2a50 0x7f57180b4f10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f572c0a93a0 tx=0x7f572c009040 comp rx=0 tx=0).stop 2024-05-08T05:56:21.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.237+0000 7f5739200700 1 -- 172.21.15.112:0/816212073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f573410b680 msgr2=0x7f573410f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:21.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.237+0000 7f5739200700 1 --2- 172.21.15.112:0/816212073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f573410b680 0x7f573410f990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f572c00ef30 tx=0x7f572c00c450 comp rx=0 tx=0).stop 2024-05-08T05:56:21.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.238+0000 7f5739200700 1 -- 172.21.15.112:0/816212073 shutdown_connections 2024-05-08T05:56:21.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.238+0000 7f5739200700 1 --2- 172.21.15.112:0/816212073 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f57180b2a50 0x7f57180b4f10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.238+0000 7f5739200700 1 --2- 172.21.15.112:0/816212073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f573410b680 0x7f573410f990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.238+0000 7f5739200700 1 --2- 172.21.15.112:0/816212073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57341055d0 0x7f5734112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.238+0000 7f5739200700 1 --2- 172.21.15.112:0/816212073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5734105f90 0x7f573410f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:21.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.238+0000 7f5739200700 1 -- 172.21.15.112:0/816212073 >> 172.21.15.112:0/816212073 conn(0x7f5734101080 msgr2=0x7f57341028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:21.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.238+0000 7f5739200700 1 -- 172.21.15.112:0/816212073 shutdown_connections 2024-05-08T05:56:21.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:21.239+0000 7f5739200700 1 -- 172.21.15.112:0/816212073 wait complete. 2024-05-08T05:56:21.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:20 smithi162 ceph-mon[34800]: pgmap v472: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:56:22.095 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:56:22.095 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:56:22.500 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:23.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:22 smithi150 ceph-mon[34315]: pgmap v473: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-08T05:56:23.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:22 smithi112 ceph-mon[30003]: pgmap v473: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-08T05:56:23.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:22 smithi162 ceph-mon[34800]: pgmap v473: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-08T05:56:24.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.959+0000 7f47d3200700 1 -- 172.21.15.112:0/3921571888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d41033d0 msgr2=0x7f47d4103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:24.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.959+0000 7f47d3200700 1 --2- 172.21.15.112:0/3921571888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d41033d0 0x7f47d4103850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009990 tx=0x7f47c0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:56:24.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.960+0000 7f47d3200700 1 -- 172.21.15.112:0/3921571888 shutdown_connections 2024-05-08T05:56:24.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.960+0000 7f47d3200700 1 --2- 172.21.15.112:0/3921571888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47d4103d90 0x7f47d41102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:24.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.960+0000 7f47d3200700 1 --2- 172.21.15.112:0/3921571888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47d4109460 0x7f47d4109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:24.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.960+0000 7f47d3200700 1 --2- 172.21.15.112:0/3921571888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d41033d0 0x7f47d4103850 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:24.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.960+0000 7f47d3200700 1 -- 172.21.15.112:0/3921571888 >> 172.21.15.112:0/3921571888 conn(0x7f47d40fef20 msgr2=0x7f47d4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:24.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.960+0000 7f47d3200700 1 -- 172.21.15.112:0/3921571888 shutdown_connections 2024-05-08T05:56:24.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.960+0000 7f47d3200700 1 -- 172.21.15.112:0/3921571888 wait complete. 2024-05-08T05:56:24.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.961+0000 7f47d3200700 1 Processor -- start 2024-05-08T05:56:24.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.962+0000 7f47d3200700 1 -- start start 2024-05-08T05:56:24.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.963+0000 7f47d3200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47d41033d0 0x7f47d41a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:24.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.963+0000 7f47d3200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d4103d90 0x7f47d41a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:24.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.963+0000 7f47d3200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47d4109460 0x7f47d41a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:24.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.963+0000 7f47d3200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47d4111460 con 0x7f47d4103d90 2024-05-08T05:56:24.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.963+0000 7f47d3200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f47d41112e0 con 0x7f47d41033d0 2024-05-08T05:56:24.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.963+0000 7f47d3200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f47d41115e0 con 0x7f47d4109460 2024-05-08T05:56:24.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.963+0000 7f47d2800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47d4109460 0x7f47d41a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.963+0000 7f47d2800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47d4109460 0x7f47d41a4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:47148/0 (socket says 172.21.15.112:47148) 2024-05-08T05:56:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.963+0000 7f47d2800700 1 -- 172.21.15.112:0/1308128770 learned_addr learned my addr 172.21.15.112:0/1308128770 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.963+0000 7f47d1e00700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47d41033d0 0x7f47d41a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.964+0000 7f47d2800700 1 -- 172.21.15.112:0/1308128770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47d41033d0 msgr2=0x7f47d41a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.964+0000 7f47d2800700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47d41033d0 0x7f47d41a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.964+0000 7f47d2800700 1 -- 172.21.15.112:0/1308128770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d4103d90 msgr2=0x7f47d41a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.964+0000 7f47d1400700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d4103d90 0x7f47d41a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.964+0000 7f47d2800700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d4103d90 0x7f47d41a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.964+0000 7f47d2800700 1 -- 172.21.15.112:0/1308128770 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d41a50b0 con 0x7f47d4109460 2024-05-08T05:56:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.964+0000 7f47d1e00700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47d41033d0 0x7f47d41a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.964+0000 7f47d1400700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d4103d90 0x7f47d41a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.965+0000 7f47d2800700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47d4109460 0x7f47d41a4970 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00ef30 tx=0x7f47cc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.965+0000 7f47caa00700 1 -- 172.21.15.112:0/1308128770 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0092e0 con 0x7f47d4109460 2024-05-08T05:56:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.965+0000 7f47d3200700 1 -- 172.21.15.112:0/1308128770 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d41a5340 con 0x7f47d4109460 2024-05-08T05:56:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.966+0000 7f47d3200700 1 -- 172.21.15.112:0/1308128770 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d41acc20 con 0x7f47d4109460 2024-05-08T05:56:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.966+0000 7f47caa00700 1 -- 172.21.15.112:0/1308128770 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47cc004bb0 con 0x7f47d4109460 2024-05-08T05:56:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.966+0000 7f47caa00700 1 -- 172.21.15.112:0/1308128770 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc004280 con 0x7f47d4109460 2024-05-08T05:56:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.967+0000 7f47d3200700 1 -- 172.21.15.112:0/1308128770 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b4005330 con 0x7f47d4109460 2024-05-08T05:56:24.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.971+0000 7f47caa00700 1 -- 172.21.15.112:0/1308128770 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f47cc01b030 con 0x7f47d4109460 2024-05-08T05:56:24.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.973+0000 7f47caa00700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f47b80b2900 0x7f47b80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:24.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.974+0000 7f47caa00700 1 -- 172.21.15.112:0/1308128770 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f47cc0e1df0 con 0x7f47d4109460 2024-05-08T05:56:24.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.974+0000 7f47caa00700 1 -- 172.21.15.112:0/1308128770 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f47cc022020 con 0x7f47d4109460 2024-05-08T05:56:24.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.975+0000 7f47d1e00700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f47b80b2900 0x7f47b80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:24.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:24.976+0000 7f47d1e00700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f47b80b2900 0x7f47b80b4dc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f47c400f8f0 tx=0x7f47c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:25.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:24 smithi112 ceph-mon[30003]: pgmap v474: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T05:56:25.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:24 smithi150 ceph-mon[34315]: pgmap v474: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T05:56:25.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:24 smithi162 ceph-mon[34800]: pgmap v474: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T05:56:25.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:25.350+0000 7f47d3200700 1 -- 172.21.15.112:0/1308128770 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47b4005620 con 0x7f47d4109460 2024-05-08T05:56:25.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:25.352+0000 7f47caa00700 1 -- 172.21.15.112:0/1308128770 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11891 (secure 0 0 0) 0x7f47cc010040 con 0x7f47d4109460 2024-05-08T05:56:25.352 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:56:25.353 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T05:54:58.248303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24518,24511]},"id":1}]} 2024-05-08T05:56:25.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:25.355+0000 7f47d3200700 1 -- 172.21.15.112:0/1308128770 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f47b80b2900 msgr2=0x7f47b80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:25.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:25.355+0000 7f47d3200700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f47b80b2900 0x7f47b80b4dc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f47c400f8f0 tx=0x7f47c4009260 comp rx=0 tx=0).stop 2024-05-08T05:56:25.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:25.355+0000 7f47d3200700 1 -- 172.21.15.112:0/1308128770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47d4109460 msgr2=0x7f47d41a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:25.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:25.355+0000 7f47d3200700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47d4109460 0x7f47d41a4970 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00ef30 tx=0x7f47cc00c3a0 comp rx=0 tx=0).stop 2024-05-08T05:56:25.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:25.355+0000 7f47d3200700 1 -- 172.21.15.112:0/1308128770 shutdown_connections 2024-05-08T05:56:25.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:25.355+0000 7f47d3200700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f47b80b2900 0x7f47b80b4dc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:25.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:25.355+0000 7f47d3200700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47d4109460 0x7f47d41a4970 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:25.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:25.355+0000 7f47d3200700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47d41033d0 0x7f47d41a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:25.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:25.355+0000 7f47d3200700 1 --2- 172.21.15.112:0/1308128770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d4103d90 0x7f47d41a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:25.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:25.355+0000 7f47d3200700 1 -- 172.21.15.112:0/1308128770 >> 172.21.15.112:0/1308128770 conn(0x7f47d40fef20 msgr2=0x7f47d40ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:25.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:25.355+0000 7f47d3200700 1 -- 172.21.15.112:0/1308128770 shutdown_connections 2024-05-08T05:56:25.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:25.355+0000 7f47d3200700 1 -- 172.21.15.112:0/1308128770 wait complete. 2024-05-08T05:56:25.358 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-05-08T05:56:26.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:25 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1308128770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:56:26.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:25 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1308128770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:56:26.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-08T05:56:26.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:25 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1308128770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:56:26.722 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:27.141 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:26 smithi112 ceph-mon[30003]: pgmap v475: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:56:27.141 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:26 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:56:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:26 smithi150 ceph-mon[34315]: pgmap v475: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:56:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:26 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:56:27.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:26 smithi162 ceph-mon[34800]: pgmap v475: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:56:27.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:26 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:56:29.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.173+0000 7f3806000700 1 -- 172.21.15.112:0/687639002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380010fcd0 msgr2=0x7f38001120c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:29.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.173+0000 7f3806000700 1 --2- 172.21.15.112:0/687639002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380010fcd0 0x7f38001120c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f37f800b0a0 tx=0x7f37f800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.173+0000 7f3806000700 1 -- 172.21.15.112:0/687639002 shutdown_connections 2024-05-08T05:56:29.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.173+0000 7f3806000700 1 --2- 172.21.15.112:0/687639002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380010fcd0 0x7f38001120c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.173+0000 7f3806000700 1 --2- 172.21.15.112:0/687639002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3800102120 0x7f3800102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.173+0000 7f3806000700 1 --2- 172.21.15.112:0/687639002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3800102ad0 0x7f380010f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.173+0000 7f3806000700 1 -- 172.21.15.112:0/687639002 >> 172.21.15.112:0/687639002 conn(0x7f38000fddc0 msgr2=0x7f38001001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:29.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.173+0000 7f3806000700 1 -- 172.21.15.112:0/687639002 shutdown_connections 2024-05-08T05:56:29.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.173+0000 7f3806000700 1 -- 172.21.15.112:0/687639002 wait complete. 2024-05-08T05:56:29.175 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:28 smithi112 ceph-mon[30003]: pgmap v476: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-08T05:56:29.175 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:28 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:56:29.176 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:28 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:56:29.176 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:28 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:56:29.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.175+0000 7f3806000700 1 Processor -- start 2024-05-08T05:56:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.175+0000 7f3806000700 1 -- start start 2024-05-08T05:56:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.175+0000 7f3806000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3800102120 0x7f38001a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.175+0000 7f3806000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3800102ad0 0x7f38001a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.175+0000 7f3806000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380010fcd0 0x7f380019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.175+0000 7f3806000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38001157f0 con 0x7f380010fcd0 2024-05-08T05:56:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.175+0000 7f3806000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3800115670 con 0x7f3800102120 2024-05-08T05:56:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.175+0000 7f3806000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3800115970 con 0x7f3800102ad0 2024-05-08T05:56:29.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.176+0000 7f3805600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380010fcd0 0x7f380019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.176+0000 7f3805600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380010fcd0 0x7f380019bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44788/0 (socket says 172.21.15.112:44788) 2024-05-08T05:56:29.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.176+0000 7f3805600700 1 -- 172.21.15.112:0/3094293191 learned_addr learned my addr 172.21.15.112:0/3094293191 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:29.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.177+0000 7f37ffe00700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3800102ad0 0x7f38001a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.177+0000 7f3805600700 1 -- 172.21.15.112:0/3094293191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3800102ad0 msgr2=0x7f38001a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:29.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.177+0000 7f3804c00700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3800102120 0x7f38001a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.177+0000 7f3805600700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3800102ad0 0x7f38001a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.177+0000 7f3805600700 1 -- 172.21.15.112:0/3094293191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3800102120 msgr2=0x7f38001a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:29.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.177+0000 7f3805600700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3800102120 0x7f38001a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.177+0000 7f3805600700 1 -- 172.21.15.112:0/3094293191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380019c270 con 0x7f380010fcd0 2024-05-08T05:56:29.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.177+0000 7f37ffe00700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3800102ad0 0x7f38001a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:29.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.177+0000 7f3804c00700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3800102120 0x7f38001a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:29.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.178+0000 7f3805600700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380010fcd0 0x7f380019bb10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f37f800b070 tx=0x7f37f8004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:29.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.179+0000 7f37fd600700 1 -- 172.21.15.112:0/3094293191 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8021070 con 0x7f380010fcd0 2024-05-08T05:56:29.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.179+0000 7f37fd600700 1 -- 172.21.15.112:0/3094293191 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37f8007e30 con 0x7f380010fcd0 2024-05-08T05:56:29.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.179+0000 7f3806000700 1 -- 172.21.15.112:0/3094293191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380019c500 con 0x7f380010fcd0 2024-05-08T05:56:29.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.179+0000 7f37fd600700 1 -- 172.21.15.112:0/3094293191 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8004780 con 0x7f380010fcd0 2024-05-08T05:56:29.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.179+0000 7f3806000700 1 -- 172.21.15.112:0/3094293191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38001a87d0 con 0x7f380010fcd0 2024-05-08T05:56:29.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:28 smithi150 ceph-mon[34315]: pgmap v476: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-08T05:56:29.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:28 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:56:29.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:28 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:56:29.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:28 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:56:29.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.185+0000 7f37fd600700 1 -- 172.21.15.112:0/3094293191 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f37f8007980 con 0x7f380010fcd0 2024-05-08T05:56:29.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.185+0000 7f3811f28b80 1 Processor -- start 2024-05-08T05:56:29.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.185+0000 7f3811f28b80 1 -- start start 2024-05-08T05:56:29.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.186+0000 7f3811f28b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebeb1d0 0x564e7ebeb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.186+0000 7f3811f28b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 0x564e7ebe57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.186+0000 7f3811f28b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebe5e10 0x564e7ebe6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.186+0000 7f3811f28b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564e7eb2fa20 con 0x564e7ebebaf0 2024-05-08T05:56:29.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.186+0000 7f3811f28b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564e7eacd760 con 0x564e7ebe5e10 2024-05-08T05:56:29.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.186+0000 7f3811f28b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x564e7eb2b9e0 con 0x564e7ebeb1d0 2024-05-08T05:56:29.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.186+0000 7f3804c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 0x564e7ebe57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.186+0000 7f3804c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 0x564e7ebe57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44804/0 (socket says 172.21.15.112:44804) 2024-05-08T05:56:29.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.186+0000 7f3804c00700 1 -- 172.21.15.112:0/2014825516 learned_addr learned my addr 172.21.15.112:0/2014825516 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:29.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.187+0000 7f37ffe00700 1 --2- 172.21.15.112:0/2014825516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebe5e10 0x564e7ebe6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.187+0000 7f37ffe00700 1 --2- 172.21.15.112:0/2014825516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebeb1d0 0x564e7ebeb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.187+0000 7f3804c00700 1 -- 172.21.15.112:0/2014825516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebeb1d0 msgr2=0x564e7ebeb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.187+0000 7f3804c00700 1 --2- 172.21.15.112:0/2014825516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebeb1d0 0x564e7ebeb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.187+0000 7f3804c00700 1 -- 172.21.15.112:0/2014825516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebe5e10 msgr2=0x564e7ebe6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.187+0000 7f3804c00700 1 --2- 172.21.15.112:0/2014825516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebe5e10 0x564e7ebe6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.187+0000 7f3804c00700 1 -- 172.21.15.112:0/2014825516 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e7ea7c8d0 con 0x564e7ebebaf0 2024-05-08T05:56:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.187+0000 7f37fd600700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f37e80b2900 0x7f37e80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.187+0000 7f37ffe00700 1 --2- 172.21.15.112:0/2014825516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebeb1d0 0x564e7ebeb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.187+0000 7f37fd600700 1 -- 172.21.15.112:0/3094293191 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f37f800f020 con 0x7f380010fcd0 2024-05-08T05:56:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.188+0000 7f37ffe00700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f37e80b2900 0x7f37e80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.188+0000 7f3804c00700 1 --2- 172.21.15.112:0/2014825516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 0x564e7ebe57f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009cb0 tx=0x7f37f4010370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8126c7682dad1c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:29.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.189+0000 7f37f2000700 1 -- 172.21.15.112:0/2014825516 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4010ca0 con 0x564e7ebebaf0 2024-05-08T05:56:29.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.189+0000 7f37f2000700 1 -- 172.21.15.112:0/2014825516 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37f4010e40 con 0x564e7ebebaf0 2024-05-08T05:56:29.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.189+0000 7f37f2000700 1 -- 172.21.15.112:0/2014825516 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4012450 con 0x564e7ebebaf0 2024-05-08T05:56:29.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.190+0000 7f3811f28b80 1 -- 172.21.15.112:0/2014825516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 msgr2=0x564e7ebe57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:29.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.190+0000 7f37ffe00700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f37e80b2900 0x7f37e80b4dc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3800103b00 tx=0x7f37ec0096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:29.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.190+0000 7f3811f28b80 1 --2- 172.21.15.112:0/2014825516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 0x564e7ebe57f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009cb0 tx=0x7f37f4010370 comp rx=0 tx=0).stop 2024-05-08T05:56:29.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.190+0000 7f3811f28b80 1 -- 172.21.15.112:0/2014825516 shutdown_connections 2024-05-08T05:56:29.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.191+0000 7f3811f28b80 1 --2- 172.21.15.112:0/2014825516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebe5e10 0x564e7ebe6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.191+0000 7f3811f28b80 1 --2- 172.21.15.112:0/2014825516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebeb1d0 0x564e7ebeb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.191+0000 7f3811f28b80 1 --2- 172.21.15.112:0/2014825516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 0x564e7ebe57f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.191+0000 7f3811f28b80 1 -- 172.21.15.112:0/2014825516 >> 172.21.15.112:0/2014825516 conn(0x564e7eb0bb60 msgr2=0x564e7ebd79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:29.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.191+0000 7f3811f28b80 1 -- 172.21.15.112:0/2014825516 shutdown_connections 2024-05-08T05:56:29.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.191+0000 7f3811f28b80 1 -- 172.21.15.112:0/2014825516 wait complete. 2024-05-08T05:56:29.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.192+0000 7f3811f28b80 1 -- start start 2024-05-08T05:56:29.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.192+0000 7f3811f28b80 1 Processor -- start 2024-05-08T05:56:29.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.193+0000 7f3811f28b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebe5e10 0x564e7ebf1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.193+0000 7f3811f28b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebeb1d0 0x564e7ebf1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.193+0000 7f3811f28b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 0x564e7ebeec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.193+0000 7f3811f28b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564e7ebf25f0 con 0x564e7ebebaf0 2024-05-08T05:56:29.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.193+0000 7f3811f28b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564e7ea95b20 con 0x564e7ebeb1d0 2024-05-08T05:56:29.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.193+0000 7f3811f28b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x564e7ebf2770 con 0x564e7ebe5e10 2024-05-08T05:56:29.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.193+0000 7f3811f28b80 10 client.0 _ll_register_callbacks cb 0x564e7ead6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:29.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.193+0000 7f3804c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebe5e10 0x564e7ebf1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.193+0000 7f3804c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebe5e10 0x564e7ebf1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:47200/0 (socket says 172.21.15.112:47200) 2024-05-08T05:56:29.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.193+0000 7f3804c00700 1 -- 172.21.15.112:0/1825091047 learned_addr learned my addr 172.21.15.112:0/1825091047 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:29.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.193+0000 7f3805600700 1 --2- 172.21.15.112:0/1825091047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 0x564e7ebeec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.194+0000 7f37ffe00700 1 --2- 172.21.15.112:0/1825091047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebeb1d0 0x564e7ebf1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.194+0000 7f3804c00700 1 -- 172.21.15.112:0/1825091047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebeb1d0 msgr2=0x564e7ebf1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:29.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.194+0000 7f3804c00700 1 --2- 172.21.15.112:0/1825091047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebeb1d0 0x564e7ebf1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.194+0000 7f3804c00700 1 -- 172.21.15.112:0/1825091047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 msgr2=0x564e7ebeec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:29.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.194+0000 7f3804c00700 1 --2- 172.21.15.112:0/1825091047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 0x564e7ebeec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.194+0000 7f3804c00700 1 -- 172.21.15.112:0/1825091047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e7eaf5420 con 0x564e7ebe5e10 2024-05-08T05:56:29.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.194+0000 7f3804c00700 1 --2- 172.21.15.112:0/1825091047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebe5e10 0x564e7ebf1670 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f37f4011010 tx=0x7f37f4010cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df4a7ee62a7c17a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:29.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.195+0000 7f37e6a00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:56:29.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.195+0000 7f37e6a00700 1 -- 172.21.15.112:0/1825091047 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4004200 con 0x564e7ebe5e10 2024-05-08T05:56:29.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.195+0000 7f37e6a00700 1 -- 172.21.15.112:0/1825091047 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f37f40043a0 con 0x564e7ebe5e10 2024-05-08T05:56:29.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.195+0000 7f3811f28b80 1 -- 172.21.15.112:0/1825091047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37f4010e40 con 0x564e7ebe5e10 2024-05-08T05:56:29.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.196+0000 7f37e6a00700 1 -- 172.21.15.112:0/1825091047 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37f4005030 con 0x564e7ebe5e10 2024-05-08T05:56:29.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.196+0000 7f3811f28b80 10 client.25261 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:29.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.196+0000 7f3811f28b80 1 -- 172.21.15.112:0/1825091047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564e7ebfcc50 con 0x564e7ebe5e10 2024-05-08T05:56:29.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.196+0000 7f37e6a00700 1 -- 172.21.15.112:0/1825091047 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37f4010e40 con 0x564e7ebe5e10 2024-05-08T05:56:29.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.196+0000 7f37e6a00700 1 -- 172.21.15.112:0/1825091047 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f37f400f040 con 0x564e7ebe5e10 2024-05-08T05:56:29.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.196+0000 7f3811f28b80 10 client.25261 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:29.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.196+0000 7f3811f28b80 10 client.25261 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T05:56:29.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.196+0000 7f3811f28b80 20 client.25261 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:29.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.196+0000 7f3811f28b80 1 --2- 172.21.15.112:0/1825091047 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x564e7ebff120 0x564e7ec01510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.196+0000 7f3811f28b80 4 client.25261 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:29.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.196+0000 7f3811f28b80 1 -- 172.21.15.112:0/1825091047 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e7ebaa0a0 con 0x564e7ebff120 2024-05-08T05:56:29.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.197+0000 7f37ffe00700 1 --2- 172.21.15.112:0/1825091047 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x564e7ebff120 0x564e7ec01510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.198+0000 7f37ffe00700 1 --2- 172.21.15.112:0/1825091047 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x564e7ebff120 0x564e7ec01510 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25466aaf4ef30d1e server_cookie=1e858b06a8bf25fc in_seq=0 out_seq=0 2024-05-08T05:56:29.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.198+0000 7f37e6a00700 10 client.25261 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:56:29.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.199+0000 7f37e6a00700 1 -- 172.21.15.112:0/1825091047 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564e7ebaa0a0 con 0x564e7ebff120 2024-05-08T05:56:29.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.199+0000 7f37e6a00700 10 client.25261 handle_command_reply: tid=0 2024-05-08T05:56:29.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.199+0000 7f3811f28b80 1 client.25261 shutdown 2024-05-08T05:56:29.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.199+0000 7f3811f28b80 1 -- 172.21.15.112:0/1825091047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebe5e10 msgr2=0x564e7ebf1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:29.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.199+0000 7f3811f28b80 1 --2- 172.21.15.112:0/1825091047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebe5e10 0x564e7ebf1670 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f37f4011010 tx=0x7f37f4010cd0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.200+0000 7f3811f28b80 1 -- 172.21.15.112:0/1825091047 shutdown_connections 2024-05-08T05:56:29.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.200+0000 7f3811f28b80 1 --2- 172.21.15.112:0/1825091047 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x564e7ebff120 0x564e7ec01510 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.200+0000 7f3811f28b80 1 --2- 172.21.15.112:0/1825091047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 0x564e7ebeec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.200+0000 7f3811f28b80 1 --2- 172.21.15.112:0/1825091047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebe5e10 0x564e7ebf1670 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.200+0000 7f3811f28b80 1 --2- 172.21.15.112:0/1825091047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebeb1d0 0x564e7ebf1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.200+0000 7f3811f28b80 1 -- 172.21.15.112:0/1825091047 >> 172.21.15.112:0/1825091047 conn(0x564e7eb0bb60 msgr2=0x564e7ebd2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:29.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.200+0000 7f37e6a00700 0 client.25261 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:56:29.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.200+0000 7f3811f28b80 1 -- 172.21.15.112:0/1825091047 shutdown_connections 2024-05-08T05:56:29.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.200+0000 7f3811f28b80 1 -- 172.21.15.112:0/1825091047 wait complete. 2024-05-08T05:56:29.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.200+0000 7f3811f28b80 20 client.25261 trim_cache size 0 max 16384 2024-05-08T05:56:29.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.218+0000 7f3811f28b80 1 Processor -- start 2024-05-08T05:56:29.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.218+0000 7f3811f28b80 1 -- start start 2024-05-08T05:56:29.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.218+0000 7f3811f28b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebe5e10 0x564e7ebf1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.218+0000 7f3811f28b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebeb1d0 0x564e7ebf1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.218+0000 7f3811f28b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebebaf0 0x564e7ebed320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.219+0000 7f3811f28b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38001ae120 con 0x564e7ebe5e10 2024-05-08T05:56:29.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.219+0000 7f3811f28b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38001ab9b0 con 0x564e7ebeb1d0 2024-05-08T05:56:29.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.219+0000 7f3811f28b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3800108390 con 0x564e7ebebaf0 2024-05-08T05:56:29.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.219+0000 7f3804c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebe5e10 0x564e7ebf1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.219+0000 7f37ffe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebeb1d0 0x564e7ebf1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.219+0000 7f3805600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebebaf0 0x564e7ebed320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.219+0000 7f3805600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebebaf0 0x564e7ebed320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:47204/0 (socket says 172.21.15.112:47204) 2024-05-08T05:56:29.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.219+0000 7f3805600700 1 -- 172.21.15.112:0/627166137 learned_addr learned my addr 172.21.15.112:0/627166137 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:29.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.219+0000 7f3805600700 1 -- 172.21.15.112:0/627166137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebeb1d0 msgr2=0x564e7ebf1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:29.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.219+0000 7f3805600700 1 --2- 172.21.15.112:0/627166137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebeb1d0 0x564e7ebf1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.219+0000 7f3805600700 1 -- 172.21.15.112:0/627166137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebe5e10 msgr2=0x564e7ebf1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.220+0000 7f3805600700 1 --2- 172.21.15.112:0/627166137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebe5e10 0x564e7ebf1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.220+0000 7f3805600700 1 -- 172.21.15.112:0/627166137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e7ebd7760 con 0x564e7ebebaf0 2024-05-08T05:56:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.220+0000 7f3804c00700 1 --2- 172.21.15.112:0/627166137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebe5e10 0x564e7ebf1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.221+0000 7f3805600700 1 --2- 172.21.15.112:0/627166137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebebaf0 0x564e7ebed320 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f37f80e10e0 tx=0x7f37f8004f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44eece6da0ab0175 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.221+0000 7f37f1600700 1 -- 172.21.15.112:0/627166137 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f80aa7a0 con 0x564e7ebebaf0 2024-05-08T05:56:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.221+0000 7f37f1600700 1 -- 172.21.15.112:0/627166137 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37f80ab390 con 0x564e7ebebaf0 2024-05-08T05:56:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.221+0000 7f37f1600700 1 -- 172.21.15.112:0/627166137 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f80b3a60 con 0x564e7ebebaf0 2024-05-08T05:56:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.222+0000 7f3811f28b80 1 -- 172.21.15.112:0/627166137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebebaf0 msgr2=0x564e7ebed320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.222+0000 7f3811f28b80 1 --2- 172.21.15.112:0/627166137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebebaf0 0x564e7ebed320 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f37f80e10e0 tx=0x7f37f8004f60 comp rx=0 tx=0).stop 2024-05-08T05:56:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.222+0000 7f3811f28b80 1 -- 172.21.15.112:0/627166137 shutdown_connections 2024-05-08T05:56:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.222+0000 7f3811f28b80 1 --2- 172.21.15.112:0/627166137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebebaf0 0x564e7ebed320 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.222+0000 7f3811f28b80 1 --2- 172.21.15.112:0/627166137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebe5e10 0x564e7ebf1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.222+0000 7f3811f28b80 1 --2- 172.21.15.112:0/627166137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebeb1d0 0x564e7ebf1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.222+0000 7f3811f28b80 1 -- 172.21.15.112:0/627166137 >> 172.21.15.112:0/627166137 conn(0x564e7eb0bb60 msgr2=0x564e7ec01130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:29.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.222+0000 7f3811f28b80 1 -- 172.21.15.112:0/627166137 shutdown_connections 2024-05-08T05:56:29.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.222+0000 7f3811f28b80 1 -- 172.21.15.112:0/627166137 wait complete. 2024-05-08T05:56:29.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.223+0000 7f3811f28b80 1 -- start start 2024-05-08T05:56:29.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.224+0000 7f3811f28b80 1 Processor -- start 2024-05-08T05:56:29.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.224+0000 7f3811f28b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebe5e10 0x564e7ebf7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.224+0000 7f3811f28b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebeb1d0 0x564e7ec64a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.224+0000 7f3811f28b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 0x564e7ec64f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.224+0000 7f3811f28b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38001a9e70 con 0x564e7ebebaf0 2024-05-08T05:56:29.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.224+0000 7f3811f28b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38001ab2e0 con 0x564e7ebeb1d0 2024-05-08T05:56:29.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.224+0000 7f3811f28b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f38001aa540 con 0x564e7ebe5e10 2024-05-08T05:56:29.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.224+0000 7f3811f28b80 10 client.0 _ll_register_callbacks cb 0x564e7ead6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:29.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.225+0000 7f3804c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebe5e10 0x564e7ebf7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.225+0000 7f3804c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebe5e10 0x564e7ebf7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:47206/0 (socket says 172.21.15.112:47206) 2024-05-08T05:56:29.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.225+0000 7f3804c00700 1 -- 172.21.15.112:0/466543943 learned_addr learned my addr 172.21.15.112:0/466543943 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:29.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.225+0000 7f37ffe00700 1 --2- 172.21.15.112:0/466543943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebeb1d0 0x564e7ec64a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.225+0000 7f3805600700 1 --2- 172.21.15.112:0/466543943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 0x564e7ec64f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.225+0000 7f3804c00700 1 -- 172.21.15.112:0/466543943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebeb1d0 msgr2=0x564e7ec64a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:29.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.226+0000 7f3804c00700 1 --2- 172.21.15.112:0/466543943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebeb1d0 0x564e7ec64a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.226+0000 7f3804c00700 1 -- 172.21.15.112:0/466543943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 msgr2=0x564e7ec64f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:29.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.226+0000 7f3804c00700 1 --2- 172.21.15.112:0/466543943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 0x564e7ec64f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.226+0000 7f3804c00700 1 -- 172.21.15.112:0/466543943 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e7ec637b0 con 0x564e7ebe5e10 2024-05-08T05:56:29.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.226+0000 7f3805600700 1 --2- 172.21.15.112:0/466543943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 0x564e7ec64f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:29.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.226+0000 7f3804c00700 1 --2- 172.21.15.112:0/466543943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebe5e10 0x564e7ebf7e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f37f4001eb0 tx=0x7f37f40149d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0f61f6379387481 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:29.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.226+0000 7f37e6a00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:56:29.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.226+0000 7f37e6a00700 1 -- 172.21.15.112:0/466543943 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4035040 con 0x564e7ebe5e10 2024-05-08T05:56:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.226+0000 7f3811f28b80 1 -- 172.21.15.112:0/466543943 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37f80ab390 con 0x564e7ebe5e10 2024-05-08T05:56:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.227+0000 7f37e6a00700 1 -- 172.21.15.112:0/466543943 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f37f4014e70 con 0x564e7ebe5e10 2024-05-08T05:56:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.227+0000 7f37e6a00700 1 -- 172.21.15.112:0/466543943 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37f4040030 con 0x564e7ebe5e10 2024-05-08T05:56:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.227+0000 7f3811f28b80 10 client.25267 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.227+0000 7f3811f28b80 1 -- 172.21.15.112:0/466543943 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564e7ebf6b40 con 0x564e7ebe5e10 2024-05-08T05:56:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.228+0000 7f37e6a00700 1 -- 172.21.15.112:0/466543943 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37f80ab390 con 0x564e7ebe5e10 2024-05-08T05:56:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.228+0000 7f37e6a00700 1 -- 172.21.15.112:0/466543943 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f37f4038020 con 0x564e7ebe5e10 2024-05-08T05:56:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.228+0000 7f3811f28b80 10 client.25267 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.228+0000 7f3811f28b80 10 client.25267 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T05:56:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.228+0000 7f3811f28b80 20 client.25267 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.228+0000 7f3811f28b80 1 --2- 172.21.15.112:0/466543943 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x564e7ebf7250 0x564e7ebe4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.228+0000 7f3811f28b80 4 client.25267 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:56:29.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.228+0000 7f3811f28b80 1 -- 172.21.15.112:0/466543943 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e7eb3d400 con 0x564e7ebf7250 2024-05-08T05:56:29.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.229+0000 7f37ffe00700 1 --2- 172.21.15.112:0/466543943 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x564e7ebf7250 0x564e7ebe4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.229+0000 7f37e6a00700 10 client.25267 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:56:29.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.229+0000 7f37ffe00700 1 --2- 172.21.15.112:0/466543943 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x564e7ebf7250 0x564e7ebe4010 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db70119247667d6d server_cookie=e20857be1b5350e9 in_seq=0 out_seq=0 2024-05-08T05:56:29.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.229+0000 7f37e6a00700 1 -- 172.21.15.112:0/466543943 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e7eb3d400 con 0x564e7ebf7250 2024-05-08T05:56:29.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.229+0000 7f37e6a00700 10 client.25267 handle_command_reply: tid=0 2024-05-08T05:56:29.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.229+0000 7f3811f28b80 1 client.25267 shutdown 2024-05-08T05:56:29.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.230+0000 7f3811f28b80 1 -- 172.21.15.112:0/466543943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebe5e10 msgr2=0x564e7ebf7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:29.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.230+0000 7f3811f28b80 1 --2- 172.21.15.112:0/466543943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebe5e10 0x564e7ebf7e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f37f4001eb0 tx=0x7f37f40149d0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.230+0000 7f3811f28b80 1 -- 172.21.15.112:0/466543943 shutdown_connections 2024-05-08T05:56:29.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.230+0000 7f3811f28b80 1 --2- 172.21.15.112:0/466543943 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x564e7ebf7250 0x564e7ebe4010 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.230+0000 7f3811f28b80 1 --2- 172.21.15.112:0/466543943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7ebebaf0 0x564e7ec64f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.230+0000 7f3811f28b80 1 --2- 172.21.15.112:0/466543943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564e7ebe5e10 0x564e7ebf7e00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.230+0000 7f3811f28b80 1 --2- 172.21.15.112:0/466543943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e7ebeb1d0 0x564e7ec64a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.230+0000 7f3811f28b80 1 -- 172.21.15.112:0/466543943 >> 172.21.15.112:0/466543943 conn(0x564e7eb0bb60 msgr2=0x564e7ebf1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:29.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.230+0000 7f37e6a00700 0 client.25267 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:56:29.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.230+0000 7f3811f28b80 1 -- 172.21.15.112:0/466543943 shutdown_connections 2024-05-08T05:56:29.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.231+0000 7f3811f28b80 1 -- 172.21.15.112:0/466543943 wait complete. 2024-05-08T05:56:29.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.231+0000 7f3811f28b80 20 client.25267 trim_cache size 0 max 16384 2024-05-08T05:56:29.233 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:56:29.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.231+0000 7f3806000700 1 -- 172.21.15.112:0/3094293191 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f37e80b2900 msgr2=0x7f37e80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:29.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.231+0000 7f3806000700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f37e80b2900 0x7f37e80b4dc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3800103b00 tx=0x7f37ec0096c0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.232+0000 7f3806000700 1 -- 172.21.15.112:0/3094293191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380010fcd0 msgr2=0x7f380019bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:29.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.232+0000 7f3806000700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380010fcd0 0x7f380019bb10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f37f800b070 tx=0x7f37f8004c80 comp rx=0 tx=0).stop 2024-05-08T05:56:29.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.232+0000 7f3806000700 1 -- 172.21.15.112:0/3094293191 shutdown_connections 2024-05-08T05:56:29.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.232+0000 7f3806000700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f37e80b2900 0x7f37e80b4dc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.232+0000 7f3806000700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380010fcd0 0x7f380019bb10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.232+0000 7f3806000700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3800102120 0x7f38001a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.232+0000 7f3806000700 1 --2- 172.21.15.112:0/3094293191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3800102ad0 0x7f38001a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.232+0000 7f3806000700 1 -- 172.21.15.112:0/3094293191 >> 172.21.15.112:0/3094293191 conn(0x7f38000fddc0 msgr2=0x7f3800110fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:29.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.232+0000 7f3806000700 1 -- 172.21.15.112:0/3094293191 shutdown_connections 2024-05-08T05:56:29.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:29.233+0000 7f3806000700 1 -- 172.21.15.112:0/3094293191 wait complete. 2024-05-08T05:56:29.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:28 smithi162 ceph-mon[34800]: pgmap v476: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-08T05:56:29.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:28 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:56:29.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:28 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:56:29.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:28 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:56:30.170 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-08T05:56:30.615 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:31.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:30 smithi112 ceph-mon[30003]: pgmap v477: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:56:31.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:30 smithi150 ceph-mon[34315]: pgmap v477: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:56:31.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:30 smithi162 ceph-mon[34800]: pgmap v477: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:56:32.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.865+0000 7fad11400700 1 -- 172.21.15.112:0/4225250234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad0c10adc0 msgr2=0x7fad0c1160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:32.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.865+0000 7fad11400700 1 --2- 172.21.15.112:0/4225250234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad0c10adc0 0x7fad0c1160b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7facf80099c0 tx=0x7facf8009cd0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.865+0000 7fad11400700 1 -- 172.21.15.112:0/4225250234 shutdown_connections 2024-05-08T05:56:32.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.865+0000 7fad11400700 1 --2- 172.21.15.112:0/4225250234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad0c1165f0 0x7fad0c1189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.865+0000 7fad11400700 1 --2- 172.21.15.112:0/4225250234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad0c10a4a0 0x7fad0c10a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.865+0000 7fad11400700 1 --2- 172.21.15.112:0/4225250234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad0c10adc0 0x7fad0c1160b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.865+0000 7fad11400700 1 -- 172.21.15.112:0/4225250234 >> 172.21.15.112:0/4225250234 conn(0x7fad0c06e480 msgr2=0x7fad0c06e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:32.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.866+0000 7fad11400700 1 -- 172.21.15.112:0/4225250234 shutdown_connections 2024-05-08T05:56:32.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.866+0000 7fad11400700 1 -- 172.21.15.112:0/4225250234 wait complete. 2024-05-08T05:56:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.867+0000 7fad11400700 1 Processor -- start 2024-05-08T05:56:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.867+0000 7fad11400700 1 -- start start 2024-05-08T05:56:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.868+0000 7fad11400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad0c10a4a0 0x7fad0c1a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.868+0000 7fad11400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad0c10adc0 0x7fad0c1a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.868+0000 7fad11400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad0c1165f0 0x7fad0c1a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.868+0000 7fad11400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c11bd60 con 0x7fad0c10adc0 2024-05-08T05:56:32.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.868+0000 7fad11400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c11bbe0 con 0x7fad0c1165f0 2024-05-08T05:56:32.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.868+0000 7fad11400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c11bee0 con 0x7fad0c10a4a0 2024-05-08T05:56:32.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.868+0000 7fad0b400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad0c10adc0 0x7fad0c1a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.868+0000 7fad0b400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad0c10adc0 0x7fad0c1a4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44848/0 (socket says 172.21.15.112:44848) 2024-05-08T05:56:32.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.868+0000 7fad0b400700 1 -- 172.21.15.112:0/2993137476 learned_addr learned my addr 172.21.15.112:0/2993137476 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:32.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.868+0000 7fad0be00700 1 --2- 172.21.15.112:0/2993137476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad0c10a4a0 0x7fad0c1a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.869+0000 7fad0b400700 1 -- 172.21.15.112:0/2993137476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad0c10a4a0 msgr2=0x7fad0c1a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:32.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.869+0000 7fad0b400700 1 --2- 172.21.15.112:0/2993137476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad0c10a4a0 0x7fad0c1a4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.869+0000 7fad0b400700 1 -- 172.21.15.112:0/2993137476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad0c1165f0 msgr2=0x7fad0c1a8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:32.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.869+0000 7fad0b400700 1 --2- 172.21.15.112:0/2993137476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad0c1165f0 0x7fad0c1a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.869+0000 7fad0b400700 1 -- 172.21.15.112:0/2993137476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c1a9440 con 0x7fad0c10adc0 2024-05-08T05:56:32.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.869+0000 7fad0be00700 1 --2- 172.21.15.112:0/2993137476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad0c10a4a0 0x7fad0c1a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:32.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.869+0000 7fad0b400700 1 --2- 172.21.15.112:0/2993137476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad0c10adc0 0x7fad0c1a4970 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7facf8006010 tx=0x7facf8005330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:32.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.870+0000 7fad08c00700 1 -- 172.21.15.112:0/2993137476 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8026070 con 0x7fad0c10adc0 2024-05-08T05:56:32.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.870+0000 7fad08c00700 1 -- 172.21.15.112:0/2993137476 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facf8003c70 con 0x7fad0c10adc0 2024-05-08T05:56:32.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.870+0000 7fad11400700 1 -- 172.21.15.112:0/2993137476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0c1a96d0 con 0x7fad0c10adc0 2024-05-08T05:56:32.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.870+0000 7fad08c00700 1 -- 172.21.15.112:0/2993137476 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf800fc20 con 0x7fad0c10adc0 2024-05-08T05:56:32.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.871+0000 7fad11400700 1 -- 172.21.15.112:0/2993137476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0c1b0fb0 con 0x7fad0c10adc0 2024-05-08T05:56:32.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.875+0000 7fad08c00700 1 -- 172.21.15.112:0/2993137476 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7facf8029070 con 0x7fad0c10adc0 2024-05-08T05:56:32.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.877+0000 7fad1d2dfb80 1 Processor -- start 2024-05-08T05:56:32.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.877+0000 7fad1d2dfb80 1 -- start start 2024-05-08T05:56:32.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.878+0000 7fad08c00700 1 --2- 172.21.15.112:0/2993137476 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7facf00b2900 0x7facf00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.878+0000 7fad1d2dfb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae194e5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.878+0000 7fad1d2dfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194e56c0 0x55ae194df430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.878+0000 7fad1d2dfb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194dfa50 0x55ae194dfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.878+0000 7fad1d2dfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ae19431a20 con 0x55ae194e56c0 2024-05-08T05:56:32.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.878+0000 7fad1d2dfb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ae193cf760 con 0x55ae194dfa50 2024-05-08T05:56:32.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.878+0000 7fad1d2dfb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55ae1942d9e0 con 0x55ae194e4da0 2024-05-08T05:56:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.878+0000 7fad08c00700 1 -- 172.21.15.112:0/2993137476 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7facf80e2ab0 con 0x7fad0c10adc0 2024-05-08T05:56:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.878+0000 7fad0be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae194e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.878+0000 7fad0b400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194e56c0 0x55ae194df430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.878+0000 7fad0be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae194e5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:47234/0 (socket says 172.21.15.112:47234) 2024-05-08T05:56:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.878+0000 7fad0be00700 1 -- 172.21.15.112:0/2395161212 learned_addr learned my addr 172.21.15.112:0/2395161212 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.879+0000 7fad10a00700 1 --2- 172.21.15.112:0/2993137476 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7facf00b2900 0x7facf00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.879+0000 7fad0be00700 1 -- 172.21.15.112:0/2395161212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194dfa50 msgr2=0x55ae194dfed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:56:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.879+0000 7fad0be00700 1 --2- 172.21.15.112:0/2395161212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194dfa50 0x55ae194dfed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.879+0000 7fad0be00700 1 -- 172.21.15.112:0/2395161212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194e56c0 msgr2=0x55ae194df430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:32.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.879+0000 7fad0be00700 1 --2- 172.21.15.112:0/2395161212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194e56c0 0x55ae194df430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.879+0000 7fad0be00700 1 -- 172.21.15.112:0/2395161212 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae1937e8d0 con 0x55ae194e4da0 2024-05-08T05:56:32.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.879+0000 7fad0be00700 1 --2- 172.21.15.112:0/2395161212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae194e5180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7facfc009df0 tx=0x7facfc0116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23292e36bc8de69e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:32.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.879+0000 7fad01600700 1 -- 172.21.15.112:0/2395161212 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc011e20 con 0x55ae194e4da0 2024-05-08T05:56:32.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.879+0000 7fad01600700 1 -- 172.21.15.112:0/2395161212 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facfc012900 con 0x55ae194e4da0 2024-05-08T05:56:32.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.879+0000 7fad01600700 1 -- 172.21.15.112:0/2395161212 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc00f900 con 0x55ae194e4da0 2024-05-08T05:56:32.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.880+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/2395161212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 msgr2=0x55ae194e5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:32.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.880+0000 7fad10a00700 1 --2- 172.21.15.112:0/2993137476 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7facf00b2900 0x7facf00b4dc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55ae194e0410 tx=0x7fad0400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:32.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.880+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/2395161212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae194e5180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7facfc009df0 tx=0x7facfc0116c0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.881+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/2395161212 shutdown_connections 2024-05-08T05:56:32.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.881+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/2395161212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194dfa50 0x55ae194dfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.881+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/2395161212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae194e5180 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.881+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/2395161212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194e56c0 0x55ae194df430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.881+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/2395161212 >> 172.21.15.112:0/2395161212 conn(0x55ae194576d0 msgr2=0x55ae194d9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:32.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.881+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/2395161212 shutdown_connections 2024-05-08T05:56:32.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.881+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/2395161212 wait complete. 2024-05-08T05:56:32.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.882+0000 7fad1d2dfb80 1 -- start start 2024-05-08T05:56:32.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.882+0000 7fad1d2dfb80 1 Processor -- start 2024-05-08T05:56:32.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.883+0000 7fad1d2dfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194dfa50 0x55ae194eb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.883+0000 7fad1d2dfb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194e4da0 0x55ae194e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.883+0000 7fad1d2dfb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e56c0 0x55ae194e8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.883+0000 7fad1d2dfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ae194ec150 con 0x55ae194dfa50 2024-05-08T05:56:32.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.883+0000 7fad1d2dfb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ae19397b20 con 0x55ae194e4da0 2024-05-08T05:56:32.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.883+0000 7fad1d2dfb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55ae194ec2d0 con 0x55ae194e56c0 2024-05-08T05:56:32.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.883+0000 7fad1d2dfb80 10 client.0 _ll_register_callbacks cb 0x55ae193d8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:32.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.883+0000 7fad0be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194dfa50 0x55ae194eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.883+0000 7fad0be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194dfa50 0x55ae194eb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44856/0 (socket says 172.21.15.112:44856) 2024-05-08T05:56:32.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.883+0000 7fad0be00700 1 -- 172.21.15.112:0/1698864957 learned_addr learned my addr 172.21.15.112:0/1698864957 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:32.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.883+0000 7fad10a00700 1 --2- 172.21.15.112:0/1698864957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e56c0 0x55ae194e8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.884+0000 7fad0b400700 1 --2- 172.21.15.112:0/1698864957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194e4da0 0x55ae194e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.884+0000 7fad0be00700 1 -- 172.21.15.112:0/1698864957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e56c0 msgr2=0x55ae194e8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:32.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.884+0000 7fad0be00700 1 --2- 172.21.15.112:0/1698864957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e56c0 0x55ae194e8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.884+0000 7fad0be00700 1 -- 172.21.15.112:0/1698864957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194e4da0 msgr2=0x55ae194e87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:32.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.884+0000 7fad0be00700 1 --2- 172.21.15.112:0/1698864957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194e4da0 0x55ae194e87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.884+0000 7fad0be00700 1 -- 172.21.15.112:0/1698864957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae193f7420 con 0x55ae194dfa50 2024-05-08T05:56:32.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.884+0000 7fad10a00700 1 --2- 172.21.15.112:0/1698864957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e56c0 0x55ae194e8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:32.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.884+0000 7fad0b400700 1 --2- 172.21.15.112:0/1698864957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194e4da0 0x55ae194e87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:32.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.885+0000 7fad0be00700 1 --2- 172.21.15.112:0/1698864957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194dfa50 0x55ae194eb410 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7facfc01c040 tx=0x7facfc012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81071b96ef2e4ce5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:32.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.885+0000 7facf6000700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:56:32.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.885+0000 7facf6000700 1 -- 172.21.15.112:0/1698864957 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc029020 con 0x55ae194dfa50 2024-05-08T05:56:32.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.885+0000 7facf6000700 1 -- 172.21.15.112:0/1698864957 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7facfc013d50 con 0x55ae194dfa50 2024-05-08T05:56:32.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.885+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/1698864957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facfc012900 con 0x55ae194dfa50 2024-05-08T05:56:32.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.886+0000 7facf6000700 1 -- 172.21.15.112:0/1698864957 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facfc034050 con 0x55ae194dfa50 2024-05-08T05:56:32.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.886+0000 7fad1d2dfb80 10 client.15150 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:32.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.886+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/1698864957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae194ff1e0 con 0x55ae194dfa50 2024-05-08T05:56:32.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.886+0000 7facf6000700 1 -- 172.21.15.112:0/1698864957 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facfc012900 con 0x55ae194dfa50 2024-05-08T05:56:32.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.886+0000 7facf6000700 1 -- 172.21.15.112:0/1698864957 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7facfc005400 con 0x55ae194dfa50 2024-05-08T05:56:32.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.887+0000 7fad1d2dfb80 10 client.15150 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:32.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.887+0000 7fad1d2dfb80 10 client.15150 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T05:56:32.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.887+0000 7fad1d2dfb80 20 client.15150 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:32.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.887+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/1698864957 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ae19501820 0x55ae19503c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.887+0000 7fad1d2dfb80 4 client.15150 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:32.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.887+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/1698864957 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae194ac140 con 0x55ae19501820 2024-05-08T05:56:32.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.887+0000 7fad0b400700 1 --2- 172.21.15.112:0/1698864957 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ae19501820 0x55ae19503c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.888+0000 7fad0b400700 1 --2- 172.21.15.112:0/1698864957 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ae19501820 0x55ae19503c10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf86be9b708010cf server_cookie=da0e7bab717bf23c in_seq=0 out_seq=0 2024-05-08T05:56:32.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.888+0000 7facf6000700 10 client.15150 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T05:56:32.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.889+0000 7facf6000700 1 -- 172.21.15.112:0/1698864957 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ae194ac140 con 0x55ae19501820 2024-05-08T05:56:32.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.889+0000 7facf6000700 10 client.15150 handle_command_reply: tid=0 2024-05-08T05:56:32.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.889+0000 7fad1d2dfb80 1 client.15150 shutdown 2024-05-08T05:56:32.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.890+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/1698864957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194dfa50 msgr2=0x55ae194eb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:32.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.890+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/1698864957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194dfa50 0x55ae194eb410 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7facfc01c040 tx=0x7facfc012730 comp rx=0 tx=0).stop 2024-05-08T05:56:32.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.890+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/1698864957 shutdown_connections 2024-05-08T05:56:32.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.890+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/1698864957 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ae19501820 0x55ae19503c10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.890+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/1698864957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e56c0 0x55ae194e8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.890+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/1698864957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194dfa50 0x55ae194eb410 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.890+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/1698864957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194e4da0 0x55ae194e87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.890+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/1698864957 >> 172.21.15.112:0/1698864957 conn(0x55ae194576d0 msgr2=0x55ae194d4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.890+0000 7facf6000700 0 client.15150 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T05:56:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.891+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/1698864957 shutdown_connections 2024-05-08T05:56:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.891+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/1698864957 wait complete. 2024-05-08T05:56:32.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.891+0000 7fad1d2dfb80 20 client.15150 trim_cache size 0 max 16384 2024-05-08T05:56:32.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.921+0000 7fad1d2dfb80 1 Processor -- start 2024-05-08T05:56:32.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.921+0000 7fad1d2dfb80 1 -- start start 2024-05-08T05:56:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.921+0000 7fad1d2dfb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194dfa50 0x55ae194df430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.922+0000 7fad1d2dfb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae194d4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.922+0000 7fad1d2dfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194e56c0 0x55ae194e7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.922+0000 7fad1d2dfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1b6900 con 0x55ae194e56c0 2024-05-08T05:56:32.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.922+0000 7fad1d2dfb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1b4190 con 0x55ae194dfa50 2024-05-08T05:56:32.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.922+0000 7fad1d2dfb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c06a190 con 0x55ae194e4da0 2024-05-08T05:56:32.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.922+0000 7fad10a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194e56c0 0x55ae194e7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.922+0000 7fad0be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194dfa50 0x55ae194df430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.922+0000 7fad0be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194dfa50 0x55ae194df430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:54902/0 (socket says 172.21.15.112:54902) 2024-05-08T05:56:32.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.922+0000 7fad0be00700 1 -- 172.21.15.112:0/3609125947 learned_addr learned my addr 172.21.15.112:0/3609125947 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:32.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.922+0000 7fad0b400700 1 --2- 172.21.15.112:0/3609125947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae194d4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.923+0000 7fad0be00700 1 -- 172.21.15.112:0/3609125947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 msgr2=0x55ae194d4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:32.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.923+0000 7fad0be00700 1 --2- 172.21.15.112:0/3609125947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae194d4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.923+0000 7fad0be00700 1 -- 172.21.15.112:0/3609125947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194e56c0 msgr2=0x55ae194e7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:32.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.923+0000 7fad0be00700 1 --2- 172.21.15.112:0/3609125947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194e56c0 0x55ae194e7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.923+0000 7fad0be00700 1 -- 172.21.15.112:0/3609125947 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae194d68a0 con 0x55ae194dfa50 2024-05-08T05:56:32.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.923+0000 7fad10a00700 1 --2- 172.21.15.112:0/3609125947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194e56c0 0x55ae194e7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:32.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.923+0000 7fad0be00700 1 --2- 172.21.15.112:0/3609125947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194dfa50 0x55ae194df430 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7facfc005d60 tx=0x7facfc009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74eb2b35a3189e23 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:32.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.924+0000 7fad00c00700 1 -- 172.21.15.112:0/3609125947 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc029040 con 0x55ae194dfa50 2024-05-08T05:56:32.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.924+0000 7fad00c00700 1 -- 172.21.15.112:0/3609125947 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facfc00f8a0 con 0x55ae194dfa50 2024-05-08T05:56:32.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.924+0000 7fad00c00700 1 -- 172.21.15.112:0/3609125947 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc005040 con 0x55ae194dfa50 2024-05-08T05:56:32.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.925+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/3609125947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194dfa50 msgr2=0x55ae194df430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:32.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.925+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/3609125947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194dfa50 0x55ae194df430 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7facfc005d60 tx=0x7facfc009750 comp rx=0 tx=0).stop 2024-05-08T05:56:32.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.925+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/3609125947 shutdown_connections 2024-05-08T05:56:32.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.925+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/3609125947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194e56c0 0x55ae194e7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.925+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/3609125947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194dfa50 0x55ae194df430 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.925+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/3609125947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae194d4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.925+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/3609125947 >> 172.21.15.112:0/3609125947 conn(0x55ae194576d0 msgr2=0x55ae194e7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:32.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.925+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/3609125947 shutdown_connections 2024-05-08T05:56:32.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.926+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/3609125947 wait complete. 2024-05-08T05:56:32.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.926+0000 7fad1d2dfb80 1 -- start start 2024-05-08T05:56:32.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.926+0000 7fad1d2dfb80 1 Processor -- start 2024-05-08T05:56:32.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.927+0000 7fad1d2dfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194dfa50 0x55ae19564110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.927+0000 7fad1d2dfb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae19564650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.927+0000 7fad1d2dfb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194e56c0 0x55ae195040d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.927+0000 7fad1d2dfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c112fe0 con 0x55ae194dfa50 2024-05-08T05:56:32.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.927+0000 7fad1d2dfb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1b3ac0 con 0x55ae194e56c0 2024-05-08T05:56:32.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.927+0000 7fad1d2dfb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1b2d40 con 0x55ae194e4da0 2024-05-08T05:56:32.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.927+0000 7fad1d2dfb80 10 client.0 _ll_register_callbacks cb 0x55ae193d8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.928+0000 7fad0b400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae19564650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.928+0000 7fad0b400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae19564650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:47250/0 (socket says 172.21.15.112:47250) 2024-05-08T05:56:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.928+0000 7fad0b400700 1 -- 172.21.15.112:0/4205042606 learned_addr learned my addr 172.21.15.112:0/4205042606 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.928+0000 7fad0be00700 1 --2- 172.21.15.112:0/4205042606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194dfa50 0x55ae19564110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.928+0000 7fad10a00700 1 --2- 172.21.15.112:0/4205042606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194e56c0 0x55ae195040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.928+0000 7fad0b400700 1 -- 172.21.15.112:0/4205042606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194e56c0 msgr2=0x55ae195040d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:32.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.929+0000 7fad0b400700 1 --2- 172.21.15.112:0/4205042606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194e56c0 0x55ae195040d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.929+0000 7fad0b400700 1 -- 172.21.15.112:0/4205042606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194dfa50 msgr2=0x55ae19564110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:32.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.929+0000 7fad0b400700 1 --2- 172.21.15.112:0/4205042606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194dfa50 0x55ae19564110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.929+0000 7fad0b400700 1 -- 172.21.15.112:0/4205042606 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae195047d0 con 0x55ae194e4da0 2024-05-08T05:56:32.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.930+0000 7fad0b400700 1 --2- 172.21.15.112:0/4205042606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae19564650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7facf80059d0 tx=0x7facf80a60b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b795a9fea7436a37 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:32.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.930+0000 7facf6000700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:56:32.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.933+0000 7facf6000700 1 -- 172.21.15.112:0/4205042606 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8003e20 con 0x55ae194e4da0 2024-05-08T05:56:32.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.933+0000 7facf6000700 1 -- 172.21.15.112:0/4205042606 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7facf80a66c0 con 0x55ae194e4da0 2024-05-08T05:56:32.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.933+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/4205042606 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facfc00f8a0 con 0x55ae194e4da0 2024-05-08T05:56:32.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.933+0000 7facf6000700 1 -- 172.21.15.112:0/4205042606 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facf8020370 con 0x55ae194e4da0 2024-05-08T05:56:32.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.934+0000 7fad1d2dfb80 10 client.25285 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:32.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.934+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/4205042606 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae1956b980 con 0x55ae194e4da0 2024-05-08T05:56:32.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.934+0000 7facf6000700 1 -- 172.21.15.112:0/4205042606 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facfc00f8a0 con 0x55ae194e4da0 2024-05-08T05:56:32.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.934+0000 7facf6000700 1 -- 172.21.15.112:0/4205042606 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7facf80cc040 con 0x55ae194e4da0 2024-05-08T05:56:32.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.934+0000 7fad1d2dfb80 10 client.25285 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:32.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.934+0000 7fad1d2dfb80 10 client.25285 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T05:56:32.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.934+0000 7fad1d2dfb80 20 client.25285 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:32.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.934+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/4205042606 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ae1956c030 0x55ae1956e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:32.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.934+0000 7fad1d2dfb80 4 client.25285 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:56:32.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.934+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/4205042606 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae1943f400 con 0x55ae1956c030 2024-05-08T05:56:32.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.935+0000 7fad0be00700 1 --2- 172.21.15.112:0/4205042606 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ae1956c030 0x55ae1956e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:32.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.936+0000 7facf6000700 10 client.25285 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T05:56:32.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.936+0000 7fad0be00700 1 --2- 172.21.15.112:0/4205042606 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ae1956c030 0x55ae1956e420 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9771280cae36a1a5 server_cookie=9150b2874cb15245 in_seq=0 out_seq=0 2024-05-08T05:56:32.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.936+0000 7facf6000700 1 -- 172.21.15.112:0/4205042606 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae1943f400 con 0x55ae1956c030 2024-05-08T05:56:32.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.936+0000 7facf6000700 10 client.25285 handle_command_reply: tid=0 2024-05-08T05:56:32.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.936+0000 7fad1d2dfb80 1 client.25285 shutdown 2024-05-08T05:56:32.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.936+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/4205042606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 msgr2=0x55ae19564650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:32.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.936+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/4205042606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae19564650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7facf80059d0 tx=0x7facf80a60b0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.936+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/4205042606 shutdown_connections 2024-05-08T05:56:32.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.936+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/4205042606 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ae1956c030 0x55ae1956e420 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.936+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/4205042606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae194e56c0 0x55ae195040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.936+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/4205042606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae194dfa50 0x55ae19564110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.936+0000 7facf6000700 0 client.25285 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T05:56:32.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.936+0000 7fad1d2dfb80 1 --2- 172.21.15.112:0/4205042606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ae194e4da0 0x55ae19564650 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.936+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/4205042606 >> 172.21.15.112:0/4205042606 conn(0x55ae194576d0 msgr2=0x55ae194e4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:32.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.937+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/4205042606 shutdown_connections 2024-05-08T05:56:32.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.937+0000 7fad1d2dfb80 1 -- 172.21.15.112:0/4205042606 wait complete. 2024-05-08T05:56:32.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.937+0000 7fad1d2dfb80 20 client.25285 trim_cache size 0 max 16384 2024-05-08T05:56:32.937 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:56:32.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.938+0000 7fad11400700 1 -- 172.21.15.112:0/2993137476 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7facf00b2900 msgr2=0x7facf00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:32.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.938+0000 7fad11400700 1 --2- 172.21.15.112:0/2993137476 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7facf00b2900 0x7facf00b4dc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55ae194e0410 tx=0x7fad0400b040 comp rx=0 tx=0).stop 2024-05-08T05:56:32.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.938+0000 7fad11400700 1 -- 172.21.15.112:0/2993137476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad0c10adc0 msgr2=0x7fad0c1a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:32.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.938+0000 7fad11400700 1 --2- 172.21.15.112:0/2993137476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad0c10adc0 0x7fad0c1a4970 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7facf8006010 tx=0x7facf8005330 comp rx=0 tx=0).stop 2024-05-08T05:56:32.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.938+0000 7fad11400700 1 -- 172.21.15.112:0/2993137476 shutdown_connections 2024-05-08T05:56:32.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.938+0000 7fad11400700 1 --2- 172.21.15.112:0/2993137476 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7facf00b2900 0x7facf00b4dc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.938+0000 7fad11400700 1 --2- 172.21.15.112:0/2993137476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad0c1165f0 0x7fad0c1a8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.938+0000 7fad11400700 1 --2- 172.21.15.112:0/2993137476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad0c10a4a0 0x7fad0c1a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.938+0000 7fad11400700 1 --2- 172.21.15.112:0/2993137476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad0c10adc0 0x7fad0c1a4970 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:32.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.938+0000 7fad11400700 1 -- 172.21.15.112:0/2993137476 >> 172.21.15.112:0/2993137476 conn(0x7fad0c06e480 msgr2=0x7fad0c10be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:32.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.939+0000 7fad11400700 1 -- 172.21.15.112:0/2993137476 shutdown_connections 2024-05-08T05:56:32.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:32.939+0000 7fad11400700 1 -- 172.21.15.112:0/2993137476 wait complete. 2024-05-08T05:56:33.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:32 smithi150 ceph-mon[34315]: pgmap v478: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:33.248 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:32 smithi112 ceph-mon[30003]: pgmap v478: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:33.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:32 smithi162 ceph-mon[34800]: pgmap v478: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:33.736 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-08T05:56:34.189 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:35.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:34 smithi112 ceph-mon[30003]: pgmap v479: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:56:35.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:34 smithi150 ceph-mon[34315]: pgmap v479: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:56:35.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:34 smithi162 ceph-mon[34800]: pgmap v479: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T05:56:36.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.544+0000 7fb8ed800700 1 -- 172.21.15.112:0/834696432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8e8103d90 msgr2=0x7fb8e81102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:36.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.544+0000 7fb8ed800700 1 --2- 172.21.15.112:0/834696432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8e8103d90 0x7fb8e81102a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000b0a0 tx=0x7fb8e000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.544+0000 7fb8ed800700 1 -- 172.21.15.112:0/834696432 shutdown_connections 2024-05-08T05:56:36.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.544+0000 7fb8ed800700 1 --2- 172.21.15.112:0/834696432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8e8103d90 0x7fb8e81102a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.544+0000 7fb8ed800700 1 --2- 172.21.15.112:0/834696432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8e8109460 0x7fb8e8109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.544+0000 7fb8ed800700 1 --2- 172.21.15.112:0/834696432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e81033d0 0x7fb8e8103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.544+0000 7fb8ed800700 1 -- 172.21.15.112:0/834696432 >> 172.21.15.112:0/834696432 conn(0x7fb8e80fef20 msgr2=0x7fb8e8101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:36.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.544+0000 7fb8ed800700 1 -- 172.21.15.112:0/834696432 shutdown_connections 2024-05-08T05:56:36.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.544+0000 7fb8ed800700 1 -- 172.21.15.112:0/834696432 wait complete. 2024-05-08T05:56:36.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.546+0000 7fb8ed800700 1 Processor -- start 2024-05-08T05:56:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.546+0000 7fb8ed800700 1 -- start start 2024-05-08T05:56:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.547+0000 7fb8ed800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8e81033d0 0x7fb8e819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.547+0000 7fb8ed800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8e8103d90 0x7fb8e819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.547+0000 7fb8ed800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8109460 0x7fb8e8199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.547+0000 7fb8ed800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81135c0 con 0x7fb8e8109460 2024-05-08T05:56:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.547+0000 7fb8ed800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e8113440 con 0x7fb8e8103d90 2024-05-08T05:56:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.547+0000 7fb8ed800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e8113740 con 0x7fb8e81033d0 2024-05-08T05:56:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.547+0000 7fb8e7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8e81033d0 0x7fb8e819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.547+0000 7fb8e7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8e81033d0 0x7fb8e819f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:52892/0 (socket says 172.21.15.112:52892) 2024-05-08T05:56:36.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.547+0000 7fb8e7e00700 1 -- 172.21.15.112:0/3513190361 learned_addr learned my addr 172.21.15.112:0/3513190361 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:36.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.547+0000 7fb8e7400700 1 --2- 172.21.15.112:0/3513190361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8e8103d90 0x7fb8e819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.548+0000 7fb8ece00700 1 --2- 172.21.15.112:0/3513190361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8109460 0x7fb8e8199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.548+0000 7fb8e7400700 1 -- 172.21.15.112:0/3513190361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8e81033d0 msgr2=0x7fb8e819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:36.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.548+0000 7fb8e7400700 1 --2- 172.21.15.112:0/3513190361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8e81033d0 0x7fb8e819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.548+0000 7fb8e7400700 1 -- 172.21.15.112:0/3513190361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8109460 msgr2=0x7fb8e8199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:36.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.548+0000 7fb8e7400700 1 --2- 172.21.15.112:0/3513190361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8109460 0x7fb8e8199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.548+0000 7fb8e7400700 1 -- 172.21.15.112:0/3513190361 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e819a020 con 0x7fb8e8103d90 2024-05-08T05:56:36.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.548+0000 7fb8ece00700 1 --2- 172.21.15.112:0/3513190361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8109460 0x7fb8e8199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:36.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.548+0000 7fb8e7400700 1 --2- 172.21.15.112:0/3513190361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8e8103d90 0x7fb8e819fc00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400e9f0 tx=0x7fb8d400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:36.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.549+0000 7fb8e4c00700 1 -- 172.21.15.112:0/3513190361 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d400cbd0 con 0x7fb8e8103d90 2024-05-08T05:56:36.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.549+0000 7fb8e4c00700 1 -- 172.21.15.112:0/3513190361 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8d400cd70 con 0x7fb8e8103d90 2024-05-08T05:56:36.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.549+0000 7fb8e4c00700 1 -- 172.21.15.112:0/3513190361 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4018680 con 0x7fb8e8103d90 2024-05-08T05:56:36.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.549+0000 7fb8ed800700 1 -- 172.21.15.112:0/3513190361 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e819a310 con 0x7fb8e8103d90 2024-05-08T05:56:36.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.549+0000 7fb8ed800700 1 -- 172.21.15.112:0/3513190361 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e81a6520 con 0x7fb8e8103d90 2024-05-08T05:56:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.555+0000 7fb8f988bb80 1 Processor -- start 2024-05-08T05:56:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.556+0000 7fb8f988bb80 1 -- start start 2024-05-08T05:56:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.556+0000 7fb8f988bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6bb3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.556+0000 7fb8f988bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bb3770 0x5650c6bad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.556+0000 7fb8f988bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bada10 0x5650c6bade90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.556+0000 7fb8f988bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5650c6afba20 con 0x5650c6bb2e50 2024-05-08T05:56:36.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.556+0000 7fb8f988bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5650c6a99760 con 0x5650c6bada10 2024-05-08T05:56:36.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.556+0000 7fb8f988bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5650c6af79e0 con 0x5650c6bb3770 2024-05-08T05:56:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.557+0000 7fb8e4c00700 1 -- 172.21.15.112:0/3513190361 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb8d4010430 con 0x7fb8e8103d90 2024-05-08T05:56:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.557+0000 7fb8e7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bb3770 0x5650c6bad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.557+0000 7fb8e7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bb3770 0x5650c6bad3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:52898/0 (socket says 172.21.15.112:52898) 2024-05-08T05:56:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.557+0000 7fb8e7e00700 1 -- 172.21.15.112:0/3428559215 learned_addr learned my addr 172.21.15.112:0/3428559215 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.557+0000 7fb8e7e00700 1 -- 172.21.15.112:0/3428559215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bada10 msgr2=0x5650c6bade90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:56:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.557+0000 7fb8ece00700 1 --2- 172.21.15.112:0/3428559215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6bb3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.557+0000 7fb8e7400700 1 --2- 172.21.15.112:0/3428559215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bada10 0x5650c6bade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.557+0000 7fb8e7e00700 1 --2- 172.21.15.112:0/3428559215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bada10 0x5650c6bade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.557+0000 7fb8e7e00700 1 -- 172.21.15.112:0/3428559215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 msgr2=0x5650c6bb3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:36.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.557+0000 7fb8e7e00700 1 --2- 172.21.15.112:0/3428559215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6bb3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.557+0000 7fb8e7e00700 1 -- 172.21.15.112:0/3428559215 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650c6a488d0 con 0x5650c6bb3770 2024-05-08T05:56:36.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.557+0000 7fb8ece00700 1 --2- 172.21.15.112:0/3428559215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6bb3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:36.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.558+0000 7fb8e7e00700 1 --2- 172.21.15.112:0/3428559215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bb3770 0x5650c6bad3f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009dd0 tx=0x7fb8d8011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa2df240d4431c8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:36.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.559+0000 7fb8dd600700 1 -- 172.21.15.112:0/3428559215 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8004a30 con 0x5650c6bb3770 2024-05-08T05:56:36.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.559+0000 7fb8dd600700 1 -- 172.21.15.112:0/3428559215 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8d80054d0 con 0x5650c6bb3770 2024-05-08T05:56:36.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.559+0000 7fb8dd600700 1 -- 172.21.15.112:0/3428559215 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8019890 con 0x5650c6bb3770 2024-05-08T05:56:36.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.559+0000 7fb8e4c00700 1 --2- 172.21.15.112:0/3513190361 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb8cc0b2980 0x7fb8cc0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.560+0000 7fb8e4c00700 1 -- 172.21.15.112:0/3513190361 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fb8d40e23c0 con 0x7fb8e8103d90 2024-05-08T05:56:36.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.560+0000 7fb8f988bb80 1 -- 172.21.15.112:0/3428559215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bb3770 msgr2=0x5650c6bad3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:36.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.560+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/3428559215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bb3770 0x5650c6bad3f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009dd0 tx=0x7fb8d8011720 comp rx=0 tx=0).stop 2024-05-08T05:56:36.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.560+0000 7fb8ece00700 1 --2- 172.21.15.112:0/3513190361 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb8cc0b2980 0x7fb8cc0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.561+0000 7fb8f988bb80 1 -- 172.21.15.112:0/3428559215 shutdown_connections 2024-05-08T05:56:36.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.561+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/3428559215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bada10 0x5650c6bade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.561+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/3428559215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6bb3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.561+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/3428559215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bb3770 0x5650c6bad3f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.561+0000 7fb8f988bb80 1 -- 172.21.15.112:0/3428559215 >> 172.21.15.112:0/3428559215 conn(0x5650c6ad7b60 msgr2=0x5650c6ba39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:36.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.561+0000 7fb8f988bb80 1 -- 172.21.15.112:0/3428559215 shutdown_connections 2024-05-08T05:56:36.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.561+0000 7fb8f988bb80 1 -- 172.21.15.112:0/3428559215 wait complete. 2024-05-08T05:56:36.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.562+0000 7fb8ece00700 1 --2- 172.21.15.112:0/3513190361 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb8cc0b2980 0x7fb8cc0b4e40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000b070 tx=0x7fb8e0009460 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:36.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.562+0000 7fb8f988bb80 1 -- start start 2024-05-08T05:56:36.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.563+0000 7fb8f988bb80 1 Processor -- start 2024-05-08T05:56:36.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.563+0000 7fb8f988bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bada10 0x5650c6bb9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.563+0000 7fb8f988bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6bb66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.564+0000 7fb8f988bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bb6c00 0x5650c6bb7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.564+0000 7fb8f988bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5650c6bba210 con 0x5650c6bb2e50 2024-05-08T05:56:36.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.564+0000 7fb8f988bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5650c6a61b20 con 0x5650c6bb6c00 2024-05-08T05:56:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.564+0000 7fb8f988bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5650c6bba390 con 0x5650c6bada10 2024-05-08T05:56:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.564+0000 7fb8f988bb80 10 client.0 _ll_register_callbacks cb 0x5650c6aa2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.564+0000 7fb8e7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bada10 0x5650c6bb9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.564+0000 7fb8e7400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6bb66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.564+0000 7fb8e7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bada10 0x5650c6bb9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:52912/0 (socket says 172.21.15.112:52912) 2024-05-08T05:56:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.564+0000 7fb8e7e00700 1 -- 172.21.15.112:0/6800344 learned_addr learned my addr 172.21.15.112:0/6800344 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.565+0000 7fb8ece00700 1 --2- 172.21.15.112:0/6800344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bb6c00 0x5650c6bb7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.565+0000 7fb8e7e00700 1 -- 172.21.15.112:0/6800344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bb6c00 msgr2=0x5650c6bb7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:36.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.565+0000 7fb8e7e00700 1 --2- 172.21.15.112:0/6800344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bb6c00 0x5650c6bb7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.565+0000 7fb8e7e00700 1 -- 172.21.15.112:0/6800344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 msgr2=0x5650c6bb66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:36.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.565+0000 7fb8e7e00700 1 --2- 172.21.15.112:0/6800344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6bb66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.565+0000 7fb8e7e00700 1 -- 172.21.15.112:0/6800344 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650c6ac1420 con 0x5650c6bada10 2024-05-08T05:56:36.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.565+0000 7fb8e7400700 1 --2- 172.21.15.112:0/6800344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6bb66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:36.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.566+0000 7fb8e7e00700 1 --2- 172.21.15.112:0/6800344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bada10 0x5650c6bb9320 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8019470 tx=0x7fb8d8004980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2901e4f5d2b67cd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:36.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.566+0000 7fb8d2000700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:56:36.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.567+0000 7fb8d2000700 1 -- 172.21.15.112:0/6800344 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8004450 con 0x5650c6bada10 2024-05-08T05:56:36.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.567+0000 7fb8d2000700 1 -- 172.21.15.112:0/6800344 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fb8d8003680 con 0x5650c6bada10 2024-05-08T05:56:36.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.567+0000 7fb8f988bb80 1 -- 172.21.15.112:0/6800344 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d80054d0 con 0x5650c6bada10 2024-05-08T05:56:36.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.567+0000 7fb8d2000700 1 -- 172.21.15.112:0/6800344 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb8d8030030 con 0x5650c6bada10 2024-05-08T05:56:36.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.567+0000 7fb8f988bb80 10 client.25297 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:36.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.567+0000 7fb8f988bb80 1 -- 172.21.15.112:0/6800344 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650c6bc4d10 con 0x5650c6bada10 2024-05-08T05:56:36.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.568+0000 7fb8d2000700 1 -- 172.21.15.112:0/6800344 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8d80054d0 con 0x5650c6bada10 2024-05-08T05:56:36.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.568+0000 7fb8d2000700 1 -- 172.21.15.112:0/6800344 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fb8d80194d0 con 0x5650c6bada10 2024-05-08T05:56:36.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.568+0000 7fb8f988bb80 10 client.25297 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:36.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.568+0000 7fb8f988bb80 10 client.25297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:56:36.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.569+0000 7fb8f988bb80 20 client.25297 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:36.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.569+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/6800344 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5650c6bc7350 0x5650c6bc9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.569+0000 7fb8f988bb80 4 client.25297 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:36.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.569+0000 7fb8f988bb80 1 -- 172.21.15.112:0/6800344 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650c6b760a0 con 0x5650c6bc7350 2024-05-08T05:56:36.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.569+0000 7fb8e7400700 1 --2- 172.21.15.112:0/6800344 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5650c6bc7350 0x5650c6bc9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.570+0000 7fb8d2000700 10 client.25297 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:36.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.570+0000 7fb8e7400700 1 --2- 172.21.15.112:0/6800344 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5650c6bc7350 0x5650c6bc9740 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d44966996cbb9f1d server_cookie=5db441b753bd4c42 in_seq=0 out_seq=0 2024-05-08T05:56:36.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.571+0000 7fb8d2000700 1 -- 172.21.15.112:0/6800344 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5650c6b760a0 con 0x5650c6bc7350 2024-05-08T05:56:36.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.571+0000 7fb8d2000700 10 client.25297 handle_command_reply: tid=0 2024-05-08T05:56:36.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.571+0000 7fb8f988bb80 1 client.25297 shutdown 2024-05-08T05:56:36.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.572+0000 7fb8f988bb80 1 -- 172.21.15.112:0/6800344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bada10 msgr2=0x5650c6bb9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:36.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.572+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/6800344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bada10 0x5650c6bb9320 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8019470 tx=0x7fb8d8004980 comp rx=0 tx=0).stop 2024-05-08T05:56:36.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.572+0000 7fb8f988bb80 1 -- 172.21.15.112:0/6800344 shutdown_connections 2024-05-08T05:56:36.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.572+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/6800344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bb6c00 0x5650c6bb7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.572+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/6800344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bada10 0x5650c6bb9320 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.572+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/6800344 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5650c6bc7350 0x5650c6bc9740 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.572+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/6800344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6bb66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.572+0000 7fb8f988bb80 1 -- 172.21.15.112:0/6800344 >> 172.21.15.112:0/6800344 conn(0x5650c6ad7b60 msgr2=0x5650c6b9e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:36.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.572+0000 7fb8d2000700 0 client.25297 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:36.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.573+0000 7fb8f988bb80 1 -- 172.21.15.112:0/6800344 shutdown_connections 2024-05-08T05:56:36.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.573+0000 7fb8f988bb80 1 -- 172.21.15.112:0/6800344 wait complete. 2024-05-08T05:56:36.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.573+0000 7fb8f988bb80 20 client.25297 trim_cache size 0 max 16384 2024-05-08T05:56:36.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.603+0000 7fb8f988bb80 1 Processor -- start 2024-05-08T05:56:36.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.603+0000 7fb8f988bb80 1 -- start start 2024-05-08T05:56:36.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.604+0000 7fb8f988bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bada10 0x5650c6bc9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.604+0000 7fb8f988bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6b9d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.604+0000 7fb8f988bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bb6c00 0x5650c6bc3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.604+0000 7fb8f988bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81ae260 con 0x5650c6bb2e50 2024-05-08T05:56:36.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.604+0000 7fb8f988bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81abaf0 con 0x5650c6bb6c00 2024-05-08T05:56:36.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.604+0000 7fb8f988bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81a9940 con 0x5650c6bada10 2024-05-08T05:56:36.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.604+0000 7fb8e7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bada10 0x5650c6bc9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.604+0000 7fb8e7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bada10 0x5650c6bc9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:52920/0 (socket says 172.21.15.112:52920) 2024-05-08T05:56:36.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.604+0000 7fb8e7e00700 1 -- 172.21.15.112:0/1257405647 learned_addr learned my addr 172.21.15.112:0/1257405647 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:36.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.605+0000 7fb8e7400700 1 --2- 172.21.15.112:0/1257405647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6b9d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.605+0000 7fb8ece00700 1 --2- 172.21.15.112:0/1257405647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bb6c00 0x5650c6bc3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.605+0000 7fb8e7e00700 1 -- 172.21.15.112:0/1257405647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bb6c00 msgr2=0x5650c6bc3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:36.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.605+0000 7fb8e7e00700 1 --2- 172.21.15.112:0/1257405647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bb6c00 0x5650c6bc3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.605+0000 7fb8e7e00700 1 -- 172.21.15.112:0/1257405647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 msgr2=0x5650c6b9d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:36.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.605+0000 7fb8e7e00700 1 --2- 172.21.15.112:0/1257405647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6b9d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.605+0000 7fb8e7e00700 1 -- 172.21.15.112:0/1257405647 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650c6ba48a0 con 0x5650c6bada10 2024-05-08T05:56:36.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.605+0000 7fb8ece00700 1 --2- 172.21.15.112:0/1257405647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bb6c00 0x5650c6bc3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:36.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.605+0000 7fb8e7e00700 1 --2- 172.21.15.112:0/1257405647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bada10 0x5650c6bc9740 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009930 tx=0x7fb8d80196b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=433665d5c3d6fbc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:36.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.605+0000 7fb8e7400700 1 --2- 172.21.15.112:0/1257405647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6b9d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.606+0000 7fb8dcc00700 1 -- 172.21.15.112:0/1257405647 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8019b10 con 0x5650c6bada10 2024-05-08T05:56:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.606+0000 7fb8dcc00700 1 -- 172.21.15.112:0/1257405647 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8d8019cb0 con 0x5650c6bada10 2024-05-08T05:56:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.606+0000 7fb8dcc00700 1 -- 172.21.15.112:0/1257405647 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8003680 con 0x5650c6bada10 2024-05-08T05:56:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.607+0000 7fb8f988bb80 1 -- 172.21.15.112:0/1257405647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bada10 msgr2=0x5650c6bc9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.607+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/1257405647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bada10 0x5650c6bc9740 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009930 tx=0x7fb8d80196b0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.607+0000 7fb8f988bb80 1 -- 172.21.15.112:0/1257405647 shutdown_connections 2024-05-08T05:56:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.607+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/1257405647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bb6c00 0x5650c6bc3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.607+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/1257405647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bada10 0x5650c6bc9740 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.607+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/1257405647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6b9d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.607+0000 7fb8f988bb80 1 -- 172.21.15.112:0/1257405647 >> 172.21.15.112:0/1257405647 conn(0x5650c6ad7b60 msgr2=0x5650c6ba3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.607+0000 7fb8f988bb80 1 -- 172.21.15.112:0/1257405647 shutdown_connections 2024-05-08T05:56:36.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.607+0000 7fb8f988bb80 1 -- 172.21.15.112:0/1257405647 wait complete. 2024-05-08T05:56:36.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.608+0000 7fb8f988bb80 1 -- start start 2024-05-08T05:56:36.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.608+0000 7fb8f988bb80 1 Processor -- start 2024-05-08T05:56:36.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.609+0000 7fb8f988bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bada10 0x5650c6bbfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.609+0000 7fb8f988bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6bce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.609+0000 7fb8f988bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bb6c00 0x5650c6bcea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.609+0000 7fb8f988bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81a9fb0 con 0x5650c6bb2e50 2024-05-08T05:56:36.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.609+0000 7fb8f988bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81ab420 con 0x5650c6bada10 2024-05-08T05:56:36.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.609+0000 7fb8f988bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81aa680 con 0x5650c6bb6c00 2024-05-08T05:56:36.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.609+0000 7fb8f988bb80 10 client.0 _ll_register_callbacks cb 0x5650c6aa2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:36.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.609+0000 7fb8e7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bada10 0x5650c6bbfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.609+0000 7fb8e7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bada10 0x5650c6bbfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:46196/0 (socket says 172.21.15.112:46196) 2024-05-08T05:56:36.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.609+0000 7fb8e7e00700 1 -- 172.21.15.112:0/3168045174 learned_addr learned my addr 172.21.15.112:0/3168045174 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:36.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.610+0000 7fb8ece00700 1 --2- 172.21.15.112:0/3168045174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bb6c00 0x5650c6bcea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.610+0000 7fb8e7400700 1 --2- 172.21.15.112:0/3168045174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6bce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.610+0000 7fb8e7e00700 1 -- 172.21.15.112:0/3168045174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bb6c00 msgr2=0x5650c6bcea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.610+0000 7fb8e7e00700 1 --2- 172.21.15.112:0/3168045174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bb6c00 0x5650c6bcea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.610+0000 7fb8e7e00700 1 -- 172.21.15.112:0/3168045174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 msgr2=0x5650c6bce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.610+0000 7fb8e7e00700 1 --2- 172.21.15.112:0/3168045174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6bce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.610+0000 7fb8e7e00700 1 -- 172.21.15.112:0/3168045174 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650c6c309b0 con 0x5650c6bada10 2024-05-08T05:56:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.611+0000 7fb8e7e00700 1 --2- 172.21.15.112:0/3168045174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bada10 0x5650c6bbfce0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8019920 tx=0x7fb8d8005680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5f3f0490c7c2ba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.611+0000 7fb8d2000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:56:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.611+0000 7fb8d2000700 1 -- 172.21.15.112:0/3168045174 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8019e60 con 0x5650c6bada10 2024-05-08T05:56:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.611+0000 7fb8d2000700 1 -- 172.21.15.112:0/3168045174 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fb8d8004460 con 0x5650c6bada10 2024-05-08T05:56:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.611+0000 7fb8f988bb80 1 -- 172.21.15.112:0/3168045174 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d8019cb0 con 0x5650c6bada10 2024-05-08T05:56:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.612+0000 7fb8d2000700 1 -- 172.21.15.112:0/3168045174 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb8d8019b10 con 0x5650c6bada10 2024-05-08T05:56:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.612+0000 7fb8f988bb80 10 client.25280 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:36.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.612+0000 7fb8f988bb80 1 -- 172.21.15.112:0/3168045174 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650c6c31a70 con 0x5650c6bada10 2024-05-08T05:56:36.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.612+0000 7fb8d2000700 1 -- 172.21.15.112:0/3168045174 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8d8019cb0 con 0x5650c6bada10 2024-05-08T05:56:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.613+0000 7fb8d2000700 1 -- 172.21.15.112:0/3168045174 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fb8d804f070 con 0x5650c6bada10 2024-05-08T05:56:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.613+0000 7fb8f988bb80 10 client.25280 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.613+0000 7fb8f988bb80 10 client.25280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:56:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.613+0000 7fb8f988bb80 20 client.25280 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.613+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/3168045174 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5650c6c32060 0x5650c6c34480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.613+0000 7fb8f988bb80 4 client.25280 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:56:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.613+0000 7fb8f988bb80 1 -- 172.21.15.112:0/3168045174 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650c6b09400 con 0x5650c6c32060 2024-05-08T05:56:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.614+0000 7fb8e7400700 1 --2- 172.21.15.112:0/3168045174 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5650c6c32060 0x5650c6c34480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.614+0000 7fb8d2000700 10 client.25280 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.614+0000 7fb8e7400700 1 --2- 172.21.15.112:0/3168045174 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5650c6c32060 0x5650c6c34480 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2edfa6ec7f11f93e server_cookie=9c5c7fc3882064f1 in_seq=0 out_seq=0 2024-05-08T05:56:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.615+0000 7fb8d2000700 1 -- 172.21.15.112:0/3168045174 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650c6b09400 con 0x5650c6c32060 2024-05-08T05:56:36.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.615+0000 7fb8d2000700 10 client.25280 handle_command_reply: tid=0 2024-05-08T05:56:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.615+0000 7fb8f988bb80 1 client.25280 shutdown 2024-05-08T05:56:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.615+0000 7fb8f988bb80 1 -- 172.21.15.112:0/3168045174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bada10 msgr2=0x5650c6bbfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.615+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/3168045174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bada10 0x5650c6bbfce0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8019920 tx=0x7fb8d8005680 comp rx=0 tx=0).stop 2024-05-08T05:56:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.615+0000 7fb8f988bb80 1 -- 172.21.15.112:0/3168045174 shutdown_connections 2024-05-08T05:56:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.615+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/3168045174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650c6bb6c00 0x5650c6bcea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.615+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/3168045174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5650c6bada10 0x5650c6bbfce0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.615+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/3168045174 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5650c6c32060 0x5650c6c34480 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.615+0000 7fb8f988bb80 1 --2- 172.21.15.112:0/3168045174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5650c6bb2e50 0x5650c6bce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.615+0000 7fb8f988bb80 1 -- 172.21.15.112:0/3168045174 >> 172.21.15.112:0/3168045174 conn(0x5650c6ad7b60 msgr2=0x5650c6b9d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.615+0000 7fb8d2000700 0 client.25280 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.616+0000 7fb8f988bb80 1 -- 172.21.15.112:0/3168045174 shutdown_connections 2024-05-08T05:56:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.616+0000 7fb8f988bb80 1 -- 172.21.15.112:0/3168045174 wait complete. 2024-05-08T05:56:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.616+0000 7fb8f988bb80 20 client.25280 trim_cache size 0 max 16384 2024-05-08T05:56:36.618 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:56:36.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.617+0000 7fb8ed800700 1 -- 172.21.15.112:0/3513190361 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb8cc0b2980 msgr2=0x7fb8cc0b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:36.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.617+0000 7fb8ed800700 1 --2- 172.21.15.112:0/3513190361 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb8cc0b2980 0x7fb8cc0b4e40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000b070 tx=0x7fb8e0009460 comp rx=0 tx=0).stop 2024-05-08T05:56:36.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.617+0000 7fb8ed800700 1 -- 172.21.15.112:0/3513190361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8e8103d90 msgr2=0x7fb8e819fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:36.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.617+0000 7fb8ed800700 1 --2- 172.21.15.112:0/3513190361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8e8103d90 0x7fb8e819fc00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400e9f0 tx=0x7fb8d400ed00 comp rx=0 tx=0).stop 2024-05-08T05:56:36.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.618+0000 7fb8ed800700 1 -- 172.21.15.112:0/3513190361 shutdown_connections 2024-05-08T05:56:36.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.618+0000 7fb8ed800700 1 --2- 172.21.15.112:0/3513190361 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb8cc0b2980 0x7fb8cc0b4e40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.618+0000 7fb8ed800700 1 --2- 172.21.15.112:0/3513190361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8109460 0x7fb8e8199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.618+0000 7fb8ed800700 1 --2- 172.21.15.112:0/3513190361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8e81033d0 0x7fb8e819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.618+0000 7fb8ed800700 1 --2- 172.21.15.112:0/3513190361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8e8103d90 0x7fb8e819fc00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.618+0000 7fb8ed800700 1 -- 172.21.15.112:0/3513190361 >> 172.21.15.112:0/3513190361 conn(0x7fb8e80fef20 msgr2=0x7fb8e80ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:36.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.618+0000 7fb8ed800700 1 -- 172.21.15.112:0/3513190361 shutdown_connections 2024-05-08T05:56:36.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:36.618+0000 7fb8ed800700 1 -- 172.21.15.112:0/3513190361 wait complete. 2024-05-08T05:56:37.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:36 smithi162 ceph-mon[34800]: pgmap v480: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:56:37.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:36 smithi112 ceph-mon[30003]: pgmap v480: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:56:37.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:36 smithi150 ceph-mon[34315]: pgmap v480: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:56:38.482 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T05:56:38.482 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T05:56:38.888 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:39.213 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:38 smithi112 ceph-mon[30003]: pgmap v481: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:39.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:38 smithi162 ceph-mon[34800]: pgmap v481: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:39.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:38 smithi150 ceph-mon[34315]: pgmap v481: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:41.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.105+0000 7fadeb200700 1 -- 172.21.15.112:0/200486826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fade41143e0 msgr2=0x7fade41167d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.105+0000 7fadeb200700 1 --2- 172.21.15.112:0/200486826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fade41143e0 0x7fade41167d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fade000b0a0 tx=0x7fade000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.105+0000 7fadeb200700 1 -- 172.21.15.112:0/200486826 shutdown_connections 2024-05-08T05:56:41.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.105+0000 7fadeb200700 1 --2- 172.21.15.112:0/200486826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fade41143e0 0x7fade41167d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.105+0000 7fadeb200700 1 --2- 172.21.15.112:0/200486826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade4075840 0x7fade4075c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.105+0000 7fadeb200700 1 --2- 172.21.15.112:0/200486826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fade4076160 0x7fade4113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.105+0000 7fadeb200700 1 -- 172.21.15.112:0/200486826 >> 172.21.15.112:0/200486826 conn(0x7fade40fef20 msgr2=0x7fade4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:41.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.106+0000 7fadeb200700 1 -- 172.21.15.112:0/200486826 shutdown_connections 2024-05-08T05:56:41.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.106+0000 7fadeb200700 1 -- 172.21.15.112:0/200486826 wait complete. 2024-05-08T05:56:41.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.107+0000 7fadeb200700 1 Processor -- start 2024-05-08T05:56:41.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.107+0000 7fadeb200700 1 -- start start 2024-05-08T05:56:41.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.108+0000 7fadeb200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade4075840 0x7fade41a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.108+0000 7fadeb200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fade4076160 0x7fade41a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.108+0000 7fadeb200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fade41143e0 0x7fade41a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.108+0000 7fadeb200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fade4119bf0 con 0x7fade4075840 2024-05-08T05:56:41.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.108+0000 7fadeb200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fade4119a70 con 0x7fade4076160 2024-05-08T05:56:41.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.108+0000 7fadeb200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fade4119d70 con 0x7fade41143e0 2024-05-08T05:56:41.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.108+0000 7fade9400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fade4076160 0x7fade41a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.108+0000 7fade9400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fade4076160 0x7fade41a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:46204/0 (socket says 172.21.15.112:46204) 2024-05-08T05:56:41.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.108+0000 7fade9400700 1 -- 172.21.15.112:0/524799742 learned_addr learned my addr 172.21.15.112:0/524799742 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:41.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.108+0000 7fade9e00700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade4075840 0x7fade41a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.108+0000 7fadea800700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fade41143e0 0x7fade41a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.109+0000 7fade9400700 1 -- 172.21.15.112:0/524799742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fade41143e0 msgr2=0x7fade41a4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.109+0000 7fade9400700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fade41143e0 0x7fade41a4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.109+0000 7fade9400700 1 -- 172.21.15.112:0/524799742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade4075840 msgr2=0x7fade41a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.109+0000 7fade9400700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade4075840 0x7fade41a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.109+0000 7fade9400700 1 -- 172.21.15.112:0/524799742 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade41a50d0 con 0x7fade4076160 2024-05-08T05:56:41.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.109+0000 7fadea800700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fade41143e0 0x7fade41a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:41.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.109+0000 7fade9e00700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade4075840 0x7fade41a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:41.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.109+0000 7fade9400700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fade4076160 0x7fade41a0600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fadd400e9c0 tx=0x7fadd400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:41.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.110+0000 7faddea00700 1 -- 172.21.15.112:0/524799742 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd400cbd0 con 0x7fade4076160 2024-05-08T05:56:41.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.110+0000 7fadeb200700 1 -- 172.21.15.112:0/524799742 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade41a53c0 con 0x7fade4076160 2024-05-08T05:56:41.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.110+0000 7faddea00700 1 -- 172.21.15.112:0/524799742 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd400ee60 con 0x7fade4076160 2024-05-08T05:56:41.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.110+0000 7fadeb200700 1 -- 172.21.15.112:0/524799742 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade41acc40 con 0x7fade4076160 2024-05-08T05:56:41.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.110+0000 7faddea00700 1 -- 172.21.15.112:0/524799742 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4005560 con 0x7fade4076160 2024-05-08T05:56:41.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.114+0000 7fadf70fdb80 1 Processor -- start 2024-05-08T05:56:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.115+0000 7fadf70fdb80 1 -- start start 2024-05-08T05:56:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.115+0000 7fadf70fdb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3915e20 0x55fca3916200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.115+0000 7fadf70fdb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3916740 0x55fca39104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.115+0000 7fadf70fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 0x55fca3910f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.115+0000 7fadf70fdb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fca3862aa0 con 0x55fca3916740 2024-05-08T05:56:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.115+0000 7fadf70fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fca38007e0 con 0x55fca3910ad0 2024-05-08T05:56:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.115+0000 7fadf70fdb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55fca385ea60 con 0x55fca3915e20 2024-05-08T05:56:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.115+0000 7fadea800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3915e20 0x55fca3916200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.115+0000 7fadea800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3915e20 0x55fca3916200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:52952/0 (socket says 172.21.15.112:52952) 2024-05-08T05:56:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.115+0000 7fadea800700 1 -- 172.21.15.112:0/2974857398 learned_addr learned my addr 172.21.15.112:0/2974857398 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:41.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.116+0000 7fadea800700 1 -- 172.21.15.112:0/2974857398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 msgr2=0x55fca3910f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:56:41.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.116+0000 7fadea800700 1 --2- 172.21.15.112:0/2974857398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 0x55fca3910f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.116+0000 7fadea800700 1 -- 172.21.15.112:0/2974857398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3916740 msgr2=0x55fca39104b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:56:41.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.116+0000 7fade9e00700 1 --2- 172.21.15.112:0/2974857398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3916740 0x55fca39104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.116+0000 7faddea00700 1 -- 172.21.15.112:0/524799742 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fadd4004750 con 0x7fade4076160 2024-05-08T05:56:41.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.116+0000 7fadea800700 1 --2- 172.21.15.112:0/2974857398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3916740 0x55fca39104b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.116+0000 7fadea800700 1 -- 172.21.15.112:0/2974857398 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca37af950 con 0x55fca3915e20 2024-05-08T05:56:41.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.116+0000 7fade9e00700 1 --2- 172.21.15.112:0/2974857398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3916740 0x55fca39104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:41.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.117+0000 7fadea800700 1 --2- 172.21.15.112:0/2974857398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3915e20 0x55fca3916200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fade000b0a0 tx=0x7fade0009d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af4da68fe8b096a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:41.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.118+0000 7fadd3400700 1 -- 172.21.15.112:0/2974857398 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0021070 con 0x55fca3915e20 2024-05-08T05:56:41.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.118+0000 7fadd3400700 1 -- 172.21.15.112:0/2974857398 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade00074e0 con 0x55fca3915e20 2024-05-08T05:56:41.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.118+0000 7fadd3400700 1 -- 172.21.15.112:0/2974857398 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0005240 con 0x55fca3915e20 2024-05-08T05:56:41.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.118+0000 7faddea00700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fadcc0b2880 0x7fadcc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.118+0000 7faddea00700 1 -- 172.21.15.112:0/524799742 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fadd40e24b0 con 0x7fade4076160 2024-05-08T05:56:41.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.118+0000 7fadf70fdb80 1 -- 172.21.15.112:0/2974857398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3915e20 msgr2=0x55fca3916200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.118+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/2974857398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3915e20 0x55fca3916200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fade000b0a0 tx=0x7fade0009d50 comp rx=0 tx=0).stop 2024-05-08T05:56:41.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.119+0000 7fade9e00700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fadcc0b2880 0x7fadcc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.119+0000 7fadf70fdb80 1 -- 172.21.15.112:0/2974857398 shutdown_connections 2024-05-08T05:56:41.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.119+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/2974857398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 0x55fca3910f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.119+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/2974857398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3915e20 0x55fca3916200 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.119+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/2974857398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3916740 0x55fca39104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.119+0000 7fadf70fdb80 1 -- 172.21.15.112:0/2974857398 >> 172.21.15.112:0/2974857398 conn(0x55fca3888750 msgr2=0x55fca390ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:41.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.119+0000 7fadf70fdb80 1 -- 172.21.15.112:0/2974857398 shutdown_connections 2024-05-08T05:56:41.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.120+0000 7fadf70fdb80 1 -- 172.21.15.112:0/2974857398 wait complete. 2024-05-08T05:56:41.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.120+0000 7fade9e00700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fadcc0b2880 0x7fadcc0b4d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55fca390c9f0 tx=0x7fadd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:41.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.120+0000 7fadf70fdb80 1 -- start start 2024-05-08T05:56:41.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.121+0000 7fadf70fdb80 1 Processor -- start 2024-05-08T05:56:41.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.121+0000 7fadf70fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 0x55fca391c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.121+0000 7fadf70fdb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3915e20 0x55fca3919830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.122+0000 7fadf70fdb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3916740 0x55fca3919d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.122+0000 7fadf70fdb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fca391d170 con 0x55fca3915e20 2024-05-08T05:56:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.122+0000 7fadf70fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fca37c8ba0 con 0x55fca3910ad0 2024-05-08T05:56:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.122+0000 7fadf70fdb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55fca391d2f0 con 0x55fca3916740 2024-05-08T05:56:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.122+0000 7fade9e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 0x55fca391c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.122+0000 7fadf70fdb80 10 client.0 _ll_register_callbacks cb 0x55fca3809cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.122+0000 7fade9e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 0x55fca391c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:46212/0 (socket says 172.21.15.112:46212) 2024-05-08T05:56:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.122+0000 7fade9e00700 1 -- 172.21.15.112:0/261120450 learned_addr learned my addr 172.21.15.112:0/261120450 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.122+0000 7fade9400700 1 --2- 172.21.15.112:0/261120450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3915e20 0x55fca3919830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.122+0000 7fade9e00700 1 -- 172.21.15.112:0/261120450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3916740 msgr2=0x55fca3919d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:41.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.122+0000 7fade9e00700 1 --2- 172.21.15.112:0/261120450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3916740 0x55fca3919d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.122+0000 7fade9e00700 1 -- 172.21.15.112:0/261120450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3915e20 msgr2=0x55fca3919830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.122+0000 7fade9e00700 1 --2- 172.21.15.112:0/261120450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3915e20 0x55fca3919830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.122+0000 7fade9e00700 1 -- 172.21.15.112:0/261120450 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca38284a0 con 0x55fca3910ad0 2024-05-08T05:56:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.123+0000 7fade9e00700 1 --2- 172.21.15.112:0/261120450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 0x55fca391c440 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fadd8011da0 tx=0x7fadd8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=298dd0c514d3d5de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.123+0000 7fadc7e00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:56:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.123+0000 7fadc7e00700 1 -- 172.21.15.112:0/261120450 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd802b060 con 0x55fca3910ad0 2024-05-08T05:56:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.123+0000 7fadc7e00700 1 -- 172.21.15.112:0/261120450 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fadd801ec20 con 0x55fca3910ad0 2024-05-08T05:56:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.123+0000 7fadf70fdb80 1 -- 172.21.15.112:0/261120450 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fade00074e0 con 0x55fca3910ad0 2024-05-08T05:56:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.124+0000 7fadc7e00700 1 -- 172.21.15.112:0/261120450 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fadd801ee20 con 0x55fca3910ad0 2024-05-08T05:56:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.124+0000 7fadf70fdb80 10 client.25289 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.124+0000 7fadf70fdb80 1 -- 172.21.15.112:0/261120450 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fca3927980 con 0x55fca3910ad0 2024-05-08T05:56:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.124+0000 7fadc7e00700 1 -- 172.21.15.112:0/261120450 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade00074e0 con 0x55fca3910ad0 2024-05-08T05:56:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.124+0000 7fadc7e00700 1 -- 172.21.15.112:0/261120450 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fadd801a040 con 0x55fca3910ad0 2024-05-08T05:56:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.124+0000 7fadf70fdb80 10 client.25289 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:41.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.124+0000 7fadf70fdb80 10 client.25289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:56:41.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.124+0000 7fadf70fdb80 20 client.25289 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:41.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.124+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/261120450 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fca3929f10 0x55fca392c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.124+0000 7fadf70fdb80 4 client.25289 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:41.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.124+0000 7fadf70fdb80 1 -- 172.21.15.112:0/261120450 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fca38dd1c0 con 0x55fca3929f10 2024-05-08T05:56:41.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.125+0000 7fade9400700 1 --2- 172.21.15.112:0/261120450 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fca3929f10 0x55fca392c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.126+0000 7fade9400700 1 --2- 172.21.15.112:0/261120450 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fca3929f10 0x55fca392c360 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a568d1927399503 server_cookie=eab5037cf91fbc05 in_seq=0 out_seq=0 2024-05-08T05:56:41.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.126+0000 7fadc7e00700 10 client.25289 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:41.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.127+0000 7fadc7e00700 1 -- 172.21.15.112:0/261120450 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fca38dd1c0 con 0x55fca3929f10 2024-05-08T05:56:41.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.127+0000 7fadc7e00700 10 client.25289 handle_command_reply: tid=0 2024-05-08T05:56:41.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.127+0000 7fadf70fdb80 1 client.25289 shutdown 2024-05-08T05:56:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.127+0000 7fadf70fdb80 1 -- 172.21.15.112:0/261120450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 msgr2=0x55fca391c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.127+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/261120450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 0x55fca391c440 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fadd8011da0 tx=0x7fadd8004750 comp rx=0 tx=0).stop 2024-05-08T05:56:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.127+0000 7fadf70fdb80 1 -- 172.21.15.112:0/261120450 shutdown_connections 2024-05-08T05:56:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.127+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/261120450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3916740 0x55fca3919d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.127+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/261120450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 0x55fca391c440 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.127+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/261120450 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fca3929f10 0x55fca392c360 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.127+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/261120450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3915e20 0x55fca3919830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.127+0000 7fadf70fdb80 1 -- 172.21.15.112:0/261120450 >> 172.21.15.112:0/261120450 conn(0x55fca3888750 msgr2=0x55fca39057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:41.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.127+0000 7fadc7e00700 0 client.25289 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:41.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.128+0000 7fadf70fdb80 1 -- 172.21.15.112:0/261120450 shutdown_connections 2024-05-08T05:56:41.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.128+0000 7fadf70fdb80 1 -- 172.21.15.112:0/261120450 wait complete. 2024-05-08T05:56:41.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.128+0000 7fadf70fdb80 20 client.25289 trim_cache size 0 max 16384 2024-05-08T05:56:41.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.149+0000 7fadf70fdb80 1 Processor -- start 2024-05-08T05:56:41.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.149+0000 7fadf70fdb80 1 -- start start 2024-05-08T05:56:41.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.150+0000 7fadf70fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 0x55fca3926200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.150+0000 7fadf70fdb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3915e20 0x55fca39173e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.150+0000 7fadf70fdb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3916740 0x55fca39058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.150+0000 7fadf70fdb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fade41b6890 con 0x55fca3916740 2024-05-08T05:56:41.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.150+0000 7fadf70fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fade41b4120 con 0x55fca3910ad0 2024-05-08T05:56:41.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.150+0000 7fadf70fdb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fade41b1f70 con 0x55fca3915e20 2024-05-08T05:56:41.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.150+0000 7fade9e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 0x55fca3926200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.150+0000 7fade9400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3915e20 0x55fca39173e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.150+0000 7fade9e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 0x55fca3926200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:46214/0 (socket says 172.21.15.112:46214) 2024-05-08T05:56:41.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.150+0000 7fade9e00700 1 -- 172.21.15.112:0/521304713 learned_addr learned my addr 172.21.15.112:0/521304713 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:41.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.150+0000 7fadea800700 1 --2- 172.21.15.112:0/521304713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3916740 0x55fca39058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.151+0000 7fade9400700 1 -- 172.21.15.112:0/521304713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 msgr2=0x55fca3926200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:41.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.151+0000 7fade9400700 1 --2- 172.21.15.112:0/521304713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 0x55fca3926200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.151+0000 7fade9400700 1 -- 172.21.15.112:0/521304713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3916740 msgr2=0x55fca39058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:41.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.151+0000 7fade9400700 1 --2- 172.21.15.112:0/521304713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3916740 0x55fca39058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.151+0000 7fade9400700 1 -- 172.21.15.112:0/521304713 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca3907920 con 0x55fca3915e20 2024-05-08T05:56:41.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.151+0000 7fade9e00700 1 --2- 172.21.15.112:0/521304713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 0x55fca3926200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:41.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.151+0000 7fade9400700 1 --2- 172.21.15.112:0/521304713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3915e20 0x55fca39173e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55fca3911490 tx=0x7fadd4009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c92bdc50986e3f9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:41.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.151+0000 7fadd2a00700 1 -- 172.21.15.112:0/521304713 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd408e070 con 0x55fca3915e20 2024-05-08T05:56:41.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.151+0000 7fadd2a00700 1 -- 172.21.15.112:0/521304713 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd40a9550 con 0x55fca3915e20 2024-05-08T05:56:41.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.151+0000 7fadd2a00700 1 -- 172.21.15.112:0/521304713 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd40b2040 con 0x55fca3915e20 2024-05-08T05:56:41.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.152+0000 7fadf70fdb80 1 -- 172.21.15.112:0/521304713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3915e20 msgr2=0x55fca39173e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.152+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/521304713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3915e20 0x55fca39173e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55fca3911490 tx=0x7fadd4009640 comp rx=0 tx=0).stop 2024-05-08T05:56:41.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.153+0000 7fadf70fdb80 1 -- 172.21.15.112:0/521304713 shutdown_connections 2024-05-08T05:56:41.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.153+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/521304713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3916740 0x55fca39058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.153+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/521304713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3910ad0 0x55fca3926200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.153+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/521304713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3915e20 0x55fca39173e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.153+0000 7fadf70fdb80 1 -- 172.21.15.112:0/521304713 >> 172.21.15.112:0/521304713 conn(0x55fca3888750 msgr2=0x55fca391b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:41.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.153+0000 7fadf70fdb80 1 -- 172.21.15.112:0/521304713 shutdown_connections 2024-05-08T05:56:41.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.153+0000 7fadf70fdb80 1 -- 172.21.15.112:0/521304713 wait complete. 2024-05-08T05:56:41.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.153+0000 7fadf70fdb80 1 -- start start 2024-05-08T05:56:41.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.154+0000 7fadf70fdb80 1 Processor -- start 2024-05-08T05:56:41.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.154+0000 7fadf70fdb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3910ad0 0x55fca391f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.154+0000 7fadf70fdb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3915e20 0x55fca391fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.154+0000 7fadf70fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3916740 0x55fca391ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.154+0000 7fadf70fdb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fade41b25e0 con 0x55fca3915e20 2024-05-08T05:56:41.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.154+0000 7fadf70fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fade41b3a50 con 0x55fca3916740 2024-05-08T05:56:41.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.154+0000 7fadf70fdb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fade41b2cb0 con 0x55fca3910ad0 2024-05-08T05:56:41.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.154+0000 7fadf70fdb80 10 client.0 _ll_register_callbacks cb 0x55fca3809cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:41.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.155+0000 7fade9e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3910ad0 0x55fca391f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.155+0000 7fade9e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3910ad0 0x55fca391f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:52982/0 (socket says 172.21.15.112:52982) 2024-05-08T05:56:41.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.155+0000 7fade9e00700 1 -- 172.21.15.112:0/2147588028 learned_addr learned my addr 172.21.15.112:0/2147588028 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:41.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.155+0000 7fade9400700 1 --2- 172.21.15.112:0/2147588028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3915e20 0x55fca391fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.155+0000 7fadea800700 1 --2- 172.21.15.112:0/2147588028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3916740 0x55fca391ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.155+0000 7fade9e00700 1 -- 172.21.15.112:0/2147588028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3916740 msgr2=0x55fca391ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.155+0000 7fade9e00700 1 --2- 172.21.15.112:0/2147588028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3916740 0x55fca391ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.155+0000 7fade9e00700 1 -- 172.21.15.112:0/2147588028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3915e20 msgr2=0x55fca391fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.155+0000 7fade9e00700 1 --2- 172.21.15.112:0/2147588028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3915e20 0x55fca391fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.155+0000 7fade9e00700 1 -- 172.21.15.112:0/2147588028 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca3994990 con 0x55fca3910ad0 2024-05-08T05:56:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.156+0000 7fade9e00700 1 --2- 172.21.15.112:0/2147588028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3910ad0 0x55fca391f550 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fadd8005c10 tx=0x7fadd8005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=759e38a5f6bb12e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:41.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.156+0000 7fadc7e00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:56:41.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.156+0000 7fadc7e00700 1 -- 172.21.15.112:0/2147588028 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8035080 con 0x55fca3910ad0 2024-05-08T05:56:41.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.156+0000 7fadc7e00700 1 -- 172.21.15.112:0/2147588028 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fadd8010ca0 con 0x55fca3910ad0 2024-05-08T05:56:41.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.156+0000 7fadf70fdb80 1 -- 172.21.15.112:0/2147588028 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd40a9550 con 0x55fca3910ad0 2024-05-08T05:56:41.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.156+0000 7fadc7e00700 1 -- 172.21.15.112:0/2147588028 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fadd8041050 con 0x55fca3910ad0 2024-05-08T05:56:41.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.156+0000 7fadf70fdb80 10 client.25318 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:41.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.156+0000 7fadf70fdb80 1 -- 172.21.15.112:0/2147588028 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fca3995c90 con 0x55fca3910ad0 2024-05-08T05:56:41.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.157+0000 7fadc7e00700 1 -- 172.21.15.112:0/2147588028 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd40a9550 con 0x55fca3910ad0 2024-05-08T05:56:41.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.157+0000 7fadc7e00700 1 -- 172.21.15.112:0/2147588028 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fadd801a040 con 0x55fca3910ad0 2024-05-08T05:56:41.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.157+0000 7fadf70fdb80 10 client.25318 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:41.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.157+0000 7fadf70fdb80 10 client.25318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:56:41.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.157+0000 7fadf70fdb80 20 client.25318 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:41.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.157+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/2147588028 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fca3996340 0x55fca3998760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.157+0000 7fadf70fdb80 4 client.25318 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T05:56:41.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.157+0000 7fadf70fdb80 1 -- 172.21.15.112:0/2147588028 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fca3870480 con 0x55fca3996340 2024-05-08T05:56:41.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.157+0000 7fade9400700 1 --2- 172.21.15.112:0/2147588028 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fca3996340 0x55fca3998760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.158+0000 7fade9400700 1 --2- 172.21.15.112:0/2147588028 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fca3996340 0x55fca3998760 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe508e843c3ad6cd server_cookie=f56db5f695d78aa3 in_seq=0 out_seq=0 2024-05-08T05:56:41.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.158+0000 7fadc7e00700 10 client.25318 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:41.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.160+0000 7fadc7e00700 1 -- 172.21.15.112:0/2147588028 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fca3870480 con 0x55fca3996340 2024-05-08T05:56:41.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.160+0000 7fadc7e00700 10 client.25318 handle_command_reply: tid=0 2024-05-08T05:56:41.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.160+0000 7fadf70fdb80 1 client.25318 shutdown 2024-05-08T05:56:41.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.161+0000 7fadf70fdb80 1 -- 172.21.15.112:0/2147588028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3910ad0 msgr2=0x55fca391f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.161+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/2147588028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3910ad0 0x55fca391f550 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fadd8005c10 tx=0x7fadd8005ec0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.162+0000 7fadf70fdb80 1 -- 172.21.15.112:0/2147588028 shutdown_connections 2024-05-08T05:56:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.162+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/2147588028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fca3916740 0x55fca391ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.162+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/2147588028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca3910ad0 0x55fca391f550 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.162+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/2147588028 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fca3996340 0x55fca3998760 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.162+0000 7fadf70fdb80 1 --2- 172.21.15.112:0/2147588028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fca3915e20 0x55fca391fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.162+0000 7fadf70fdb80 1 -- 172.21.15.112:0/2147588028 >> 172.21.15.112:0/2147588028 conn(0x55fca3888750 msgr2=0x55fca391ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.162+0000 7fadc7e00700 0 client.25318 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.162+0000 7fadf70fdb80 1 -- 172.21.15.112:0/2147588028 shutdown_connections 2024-05-08T05:56:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.162+0000 7fadf70fdb80 1 -- 172.21.15.112:0/2147588028 wait complete. 2024-05-08T05:56:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.163+0000 7fadf70fdb80 20 client.25318 trim_cache size 0 max 16384 2024-05-08T05:56:41.164 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:56:41.164 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-08T05:56:41.164 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "10764047-7f24-4c12-b652-716b54739123", 2024-05-08T05:56:41.164 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-08T05:56:41.164 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:56:41.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.164+0000 7fadeb200700 1 -- 172.21.15.112:0/524799742 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fadcc0b2880 msgr2=0x7fadcc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.164+0000 7fadeb200700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fadcc0b2880 0x7fadcc0b4d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55fca390c9f0 tx=0x7fadd8009260 comp rx=0 tx=0).stop 2024-05-08T05:56:41.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.164+0000 7fadeb200700 1 -- 172.21.15.112:0/524799742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fade4076160 msgr2=0x7fade41a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.164+0000 7fadeb200700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fade4076160 0x7fade41a0600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fadd400e9c0 tx=0x7fadd400ecd0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.164+0000 7fadeb200700 1 -- 172.21.15.112:0/524799742 shutdown_connections 2024-05-08T05:56:41.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.164+0000 7fadeb200700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fadcc0b2880 0x7fadcc0b4d40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.164+0000 7fadeb200700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fade41143e0 0x7fade41a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.164+0000 7fadeb200700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade4075840 0x7fade41a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.164+0000 7fadeb200700 1 --2- 172.21.15.112:0/524799742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fade4076160 0x7fade41a0600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.164+0000 7fadeb200700 1 -- 172.21.15.112:0/524799742 >> 172.21.15.112:0/524799742 conn(0x7fade40fef20 msgr2=0x7fade40ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:41.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.165+0000 7fadeb200700 1 -- 172.21.15.112:0/524799742 shutdown_connections 2024-05-08T05:56:41.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:41.165+0000 7fadeb200700 1 -- 172.21.15.112:0/524799742 wait complete. 2024-05-08T05:56:41.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:40 smithi112 ceph-mon[30003]: pgmap v482: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:56:41.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:40 smithi162 ceph-mon[34800]: pgmap v482: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:56:41.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:40 smithi150 ceph-mon[34315]: pgmap v482: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T05:56:41.956 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:41 smithi112 ceph-mon[30003]: scrub queued for path: / 2024-05-08T05:56:41.956 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:41 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:56:41.956 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:41 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T05:56:41.956 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:41 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:56:42.210 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T05:56:42.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:41 smithi162 ceph-mon[34800]: scrub queued for path: / 2024-05-08T05:56:42.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:41 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:56:42.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:41 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T05:56:42.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:41 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:56:42.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:41 smithi150 ceph-mon[34315]: scrub queued for path: / 2024-05-08T05:56:42.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:41 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:56:42.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:41 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T05:56:42.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:41 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:56:42.658 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:43.163 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:42 smithi112 ceph-mon[30003]: pgmap v483: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:43.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:42 smithi162 ceph-mon[34800]: pgmap v483: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:43.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:42 smithi150 ceph-mon[34315]: pgmap v483: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T05:56:44.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:43 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T05:56:44.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:43 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T05:56:44.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:43 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T05:56:44.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.896+0000 7fd8a7e00700 1 -- 172.21.15.112:0/4111886346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8a8103d90 msgr2=0x7fd8a81102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:44.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.896+0000 7fd8a7e00700 1 --2- 172.21.15.112:0/4111886346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8a8103d90 0x7fd8a81102a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009f90 tx=0x7fd8a000b330 comp rx=0 tx=0).stop 2024-05-08T05:56:44.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.896+0000 7fd8a7e00700 1 -- 172.21.15.112:0/4111886346 shutdown_connections 2024-05-08T05:56:44.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.896+0000 7fd8a7e00700 1 --2- 172.21.15.112:0/4111886346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8a8103d90 0x7fd8a81102a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.896+0000 7fd8a7e00700 1 --2- 172.21.15.112:0/4111886346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8a8109460 0x7fd8a8109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.896+0000 7fd8a7e00700 1 --2- 172.21.15.112:0/4111886346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a81033d0 0x7fd8a8103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.896+0000 7fd8a7e00700 1 -- 172.21.15.112:0/4111886346 >> 172.21.15.112:0/4111886346 conn(0x7fd8a80fef20 msgr2=0x7fd8a8101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:44.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.896+0000 7fd8a7e00700 1 -- 172.21.15.112:0/4111886346 shutdown_connections 2024-05-08T05:56:44.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.896+0000 7fd8a7e00700 1 -- 172.21.15.112:0/4111886346 wait complete. 2024-05-08T05:56:44.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.898+0000 7fd8a7e00700 1 Processor -- start 2024-05-08T05:56:44.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.898+0000 7fd8a7e00700 1 -- start start 2024-05-08T05:56:44.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.898+0000 7fd8a7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8a81033d0 0x7fd8a81a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.899+0000 7fd8a7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8a8103d90 0x7fd8a81a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.899+0000 7fd8a7e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a8109460 0x7fd8a819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.899+0000 7fd8a7e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81135e0 con 0x7fd8a8109460 2024-05-08T05:56:44.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.899+0000 7fd8a7e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a8113460 con 0x7fd8a8103d90 2024-05-08T05:56:44.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.899+0000 7fd8a7e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a8113760 con 0x7fd8a81033d0 2024-05-08T05:56:44.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.899+0000 7fd8a6a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8a81033d0 0x7fd8a81a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.899+0000 7fd8a6a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8a81033d0 0x7fd8a81a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:41700/0 (socket says 172.21.15.112:41700) 2024-05-08T05:56:44.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.899+0000 7fd8a6a00700 1 -- 172.21.15.112:0/545128633 learned_addr learned my addr 172.21.15.112:0/545128633 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:44.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.899+0000 7fd8a6000700 1 --2- 172.21.15.112:0/545128633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8a8103d90 0x7fd8a81a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.899+0000 7fd8a7400700 1 --2- 172.21.15.112:0/545128633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a8109460 0x7fd8a819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.900+0000 7fd8a6000700 1 -- 172.21.15.112:0/545128633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8a81033d0 msgr2=0x7fd8a81a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:44.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.900+0000 7fd8a6000700 1 --2- 172.21.15.112:0/545128633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8a81033d0 0x7fd8a81a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.900+0000 7fd8a6000700 1 -- 172.21.15.112:0/545128633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a8109460 msgr2=0x7fd8a819ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:44.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.900+0000 7fd8a6000700 1 --2- 172.21.15.112:0/545128633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a8109460 0x7fd8a819ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.900+0000 7fd8a6000700 1 -- 172.21.15.112:0/545128633 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a819c1f0 con 0x7fd8a8103d90 2024-05-08T05:56:44.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.900+0000 7fd8a6000700 1 --2- 172.21.15.112:0/545128633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8a8103d90 0x7fd8a81a1e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd89400e9c0 tx=0x7fd89400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:44.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.900+0000 7fd8aca00700 1 -- 172.21.15.112:0/545128633 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89400c9c0 con 0x7fd8a8103d90 2024-05-08T05:56:44.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.900+0000 7fd8aca00700 1 -- 172.21.15.112:0/545128633 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd89400cb60 con 0x7fd8a8103d90 2024-05-08T05:56:44.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.900+0000 7fd8aca00700 1 -- 172.21.15.112:0/545128633 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd894018420 con 0x7fd8a8103d90 2024-05-08T05:56:44.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.901+0000 7fd8a7e00700 1 -- 172.21.15.112:0/545128633 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a819c4e0 con 0x7fd8a8103d90 2024-05-08T05:56:44.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.901+0000 7fd8a7e00700 1 -- 172.21.15.112:0/545128633 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a81a87a0 con 0x7fd8a8103d90 2024-05-08T05:56:44.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.906+0000 7fd8aca00700 1 -- 172.21.15.112:0/545128633 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd894010430 con 0x7fd8a8103d90 2024-05-08T05:56:44.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.907+0000 7fd8b878bb80 1 Processor -- start 2024-05-08T05:56:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.907+0000 7fd8b878bb80 1 -- start start 2024-05-08T05:56:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.908+0000 7fd8b878bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a6da0 0x555db67a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.908+0000 7fd8b878bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 0x555db67a1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.908+0000 7fd8b878bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a1a50 0x555db67a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.908+0000 7fd8b878bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555db66f3a20 con 0x555db67a1a50 2024-05-08T05:56:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.908+0000 7fd8b878bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555db6691760 con 0x555db67a76c0 2024-05-08T05:56:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.908+0000 7fd8b878bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x555db66ef9e0 con 0x555db67a6da0 2024-05-08T05:56:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.908+0000 7fd8a6a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 0x555db67a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.908+0000 7fd8a6a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 0x555db67a1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:55114/0 (socket says 172.21.15.112:55114) 2024-05-08T05:56:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.908+0000 7fd8a6a00700 1 -- 172.21.15.112:0/3022956039 learned_addr learned my addr 172.21.15.112:0/3022956039 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:44.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.908+0000 7fd8a7400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a6da0 0x555db67a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.909+0000 7fd8a6a00700 1 -- 172.21.15.112:0/3022956039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a6da0 msgr2=0x555db67a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.909+0000 7fd8a6a00700 1 --2- 172.21.15.112:0/3022956039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a6da0 0x555db67a7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.909+0000 7fd8a6a00700 1 -- 172.21.15.112:0/3022956039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a1a50 msgr2=0x555db67a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.909+0000 7fd8a6000700 1 --2- 172.21.15.112:0/3022956039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a1a50 0x555db67a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.909+0000 7fd8a6a00700 1 --2- 172.21.15.112:0/3022956039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a1a50 0x555db67a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.909+0000 7fd8a6a00700 1 -- 172.21.15.112:0/3022956039 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db66408d0 con 0x555db67a76c0 2024-05-08T05:56:44.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.909+0000 7fd8a6000700 1 --2- 172.21.15.112:0/3022956039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a1a50 0x555db67a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:44.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.910+0000 7fd8a6a00700 1 --2- 172.21.15.112:0/3022956039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 0x555db67a1430 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd898009d90 tx=0x7fd898011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbef858704940fb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:44.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.910+0000 7fd89cc00700 1 -- 172.21.15.112:0/3022956039 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89801e070 con 0x555db67a76c0 2024-05-08T05:56:44.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.910+0000 7fd89cc00700 1 -- 172.21.15.112:0/3022956039 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd898019ae0 con 0x555db67a76c0 2024-05-08T05:56:44.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.910+0000 7fd89cc00700 1 -- 172.21.15.112:0/3022956039 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89800fac0 con 0x555db67a76c0 2024-05-08T05:56:44.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.911+0000 7fd8aca00700 1 --2- 172.21.15.112:0/545128633 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd88c0b2900 0x7fd88c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.911+0000 7fd8b878bb80 1 -- 172.21.15.112:0/3022956039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 msgr2=0x555db67a1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:44.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.911+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/3022956039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 0x555db67a1430 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd898009d90 tx=0x7fd898011720 comp rx=0 tx=0).stop 2024-05-08T05:56:44.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.911+0000 7fd8aca00700 1 -- 172.21.15.112:0/545128633 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd8940e2400 con 0x7fd8a8103d90 2024-05-08T05:56:44.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.912+0000 7fd8a7400700 1 --2- 172.21.15.112:0/545128633 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd88c0b2900 0x7fd88c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.912+0000 7fd8b878bb80 1 -- 172.21.15.112:0/3022956039 shutdown_connections 2024-05-08T05:56:44.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.912+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/3022956039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a1a50 0x555db67a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.912+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/3022956039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a6da0 0x555db67a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.912+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/3022956039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 0x555db67a1430 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.912+0000 7fd8b878bb80 1 -- 172.21.15.112:0/3022956039 >> 172.21.15.112:0/3022956039 conn(0x555db67196d0 msgr2=0x555db679bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:44.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.912+0000 7fd8b878bb80 1 -- 172.21.15.112:0/3022956039 shutdown_connections 2024-05-08T05:56:44.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.913+0000 7fd8b878bb80 1 -- 172.21.15.112:0/3022956039 wait complete. 2024-05-08T05:56:44.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.913+0000 7fd8a7400700 1 --2- 172.21.15.112:0/545128633 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd88c0b2900 0x7fd88c0b4dc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009da0 tx=0x7fd8a0009700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:44.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.914+0000 7fd8b878bb80 1 -- start start 2024-05-08T05:56:44.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.915+0000 7fd8b878bb80 1 Processor -- start 2024-05-08T05:56:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.915+0000 7fd8b878bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a1a50 0x555db67ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.915+0000 7fd8b878bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a6da0 0x555db67aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.916+0000 7fd8b878bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 0x555db67aac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.916+0000 7fd8b878bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555db67ae150 con 0x555db67a6da0 2024-05-08T05:56:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.916+0000 7fd8b878bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555db6659b20 con 0x555db67a76c0 2024-05-08T05:56:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.916+0000 7fd8b878bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x555db67ae2d0 con 0x555db67a1a50 2024-05-08T05:56:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.916+0000 7fd8b878bb80 10 client.0 _ll_register_callbacks cb 0x555db669ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.916+0000 7fd8a6a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a1a50 0x555db67ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.916+0000 7fd8a6a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a1a50 0x555db67ad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:41718/0 (socket says 172.21.15.112:41718) 2024-05-08T05:56:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.916+0000 7fd8a6a00700 1 -- 172.21.15.112:0/588331524 learned_addr learned my addr 172.21.15.112:0/588331524 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.917+0000 7fd8a6000700 1 --2- 172.21.15.112:0/588331524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a6da0 0x555db67aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.917+0000 7fd8a6a00700 1 -- 172.21.15.112:0/588331524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 msgr2=0x555db67aac30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:56:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.917+0000 7fd8a6a00700 1 --2- 172.21.15.112:0/588331524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 0x555db67aac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.917+0000 7fd8a6a00700 1 -- 172.21.15.112:0/588331524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a6da0 msgr2=0x555db67aa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:44.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.917+0000 7fd8a6a00700 1 --2- 172.21.15.112:0/588331524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a6da0 0x555db67aa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.917+0000 7fd8a6a00700 1 -- 172.21.15.112:0/588331524 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db66b9420 con 0x555db67a1a50 2024-05-08T05:56:44.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.918+0000 7fd8b878bb80 1 -- 172.21.15.112:0/588331524 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd898019ae0 con 0x555db67a1a50 2024-05-08T05:56:44.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.918+0000 7fd8a6a00700 1 --2- 172.21.15.112:0/588331524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a1a50 0x555db67ad350 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd89801b000 tx=0x7fd898005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c06450f2f28fe82 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:44.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.918+0000 7fd891600700 10 client.25330 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:56:44.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.918+0000 7fd891600700 1 -- 172.21.15.112:0/588331524 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898026020 con 0x555db67a1a50 2024-05-08T05:56:44.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.919+0000 7fd891600700 1 -- 172.21.15.112:0/588331524 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd898005030 con 0x555db67a1a50 2024-05-08T05:56:44.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.919+0000 7fd891600700 1 -- 172.21.15.112:0/588331524 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd89801e040 con 0x555db67a1a50 2024-05-08T05:56:44.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.919+0000 7fd8b878bb80 10 client.25330 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:44.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.919+0000 7fd8b878bb80 1 -- 172.21.15.112:0/588331524 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({fsmap=25}) -- 0x555db67b87d0 con 0x555db67a1a50 2024-05-08T05:56:44.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.920+0000 7fd891600700 1 -- 172.21.15.112:0/588331524 <== mon.1 v2:172.21.15.162:3300/0 4 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd898005270 con 0x555db67a1a50 2024-05-08T05:56:44.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.920+0000 7fd8b878bb80 10 client.25330 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:44.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.920+0000 7fd8b878bb80 10 client.25330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:56:44.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.920+0000 7fd8b878bb80 20 client.25330 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:44.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.920+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/588331524 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x555db67baca0 0x555db67bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.920+0000 7fd8b878bb80 4 client.25330 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:44.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.920+0000 7fd8b878bb80 1 -- 172.21.15.112:0/588331524 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555db676e140 con 0x555db67baca0 2024-05-08T05:56:44.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.921+0000 7fd8a6000700 1 --2- 172.21.15.112:0/588331524 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x555db67baca0 0x555db67bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.922+0000 7fd8a6000700 1 --2- 172.21.15.112:0/588331524 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x555db67baca0 0x555db67bd090 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec05b0d7232eda8 server_cookie=d0db0959751ce718 in_seq=0 out_seq=0 2024-05-08T05:56:44.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.922+0000 7fd891600700 10 client.25330 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:44.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.923+0000 7fd891600700 1 -- 172.21.15.112:0/588331524 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555db676e140 con 0x555db67baca0 2024-05-08T05:56:44.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.923+0000 7fd891600700 10 client.25330 handle_command_reply: tid=0 2024-05-08T05:56:44.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.923+0000 7fd8b878bb80 1 client.25330 shutdown 2024-05-08T05:56:44.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.924+0000 7fd8b878bb80 1 -- 172.21.15.112:0/588331524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a1a50 msgr2=0x555db67ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:44.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.924+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/588331524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a1a50 0x555db67ad350 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd89801b000 tx=0x7fd898005d90 comp rx=0 tx=0).stop 2024-05-08T05:56:44.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.924+0000 7fd8b878bb80 1 -- 172.21.15.112:0/588331524 shutdown_connections 2024-05-08T05:56:44.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.924+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/588331524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 0x555db67aac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.924+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/588331524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a1a50 0x555db67ad350 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.924+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/588331524 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x555db67baca0 0x555db67bd090 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.924+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/588331524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a6da0 0x555db67aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.924+0000 7fd8b878bb80 1 -- 172.21.15.112:0/588331524 >> 172.21.15.112:0/588331524 conn(0x555db67196d0 msgr2=0x555db6796750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:44.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.924+0000 7fd891600700 0 client.25330 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:44.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.925+0000 7fd8b878bb80 1 -- 172.21.15.112:0/588331524 shutdown_connections 2024-05-08T05:56:44.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.925+0000 7fd8b878bb80 1 -- 172.21.15.112:0/588331524 wait complete. 2024-05-08T05:56:44.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.925+0000 7fd8b878bb80 20 client.25330 trim_cache size 0 max 16384 2024-05-08T05:56:44.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.966+0000 7fd8b878bb80 1 Processor -- start 2024-05-08T05:56:44.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.966+0000 7fd8b878bb80 1 -- start start 2024-05-08T05:56:44.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.967+0000 7fd8b878bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a1a50 0x555db67a1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.967+0000 7fd8b878bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a6da0 0x555db67b7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.967+0000 7fd8b878bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a76c0 0x555db6796860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.967+0000 7fd8b878bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81b25a0 con 0x555db67a1a50 2024-05-08T05:56:44.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.967+0000 7fd8b878bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81afe30 con 0x555db67a6da0 2024-05-08T05:56:44.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.967+0000 7fd8b878bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81adc80 con 0x555db67a76c0 2024-05-08T05:56:44.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.967+0000 7fd8a6000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a6da0 0x555db67b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.967+0000 7fd8a6000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a6da0 0x555db67b7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:55136/0 (socket says 172.21.15.112:55136) 2024-05-08T05:56:44.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.967+0000 7fd8a6000700 1 -- 172.21.15.112:0/892140079 learned_addr learned my addr 172.21.15.112:0/892140079 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:44.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.967+0000 7fd8a7400700 1 --2- 172.21.15.112:0/892140079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a76c0 0x555db6796860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.967+0000 7fd8a6a00700 1 --2- 172.21.15.112:0/892140079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a1a50 0x555db67a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.968+0000 7fd8a6000700 1 -- 172.21.15.112:0/892140079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a76c0 msgr2=0x555db6796860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:44.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.968+0000 7fd8a6000700 1 --2- 172.21.15.112:0/892140079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a76c0 0x555db6796860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.968+0000 7fd8a6000700 1 -- 172.21.15.112:0/892140079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a1a50 msgr2=0x555db67a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:44.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.968+0000 7fd8a6000700 1 --2- 172.21.15.112:0/892140079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a1a50 0x555db67a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.968+0000 7fd8a6000700 1 -- 172.21.15.112:0/892140079 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db67ae2f0 con 0x555db67a6da0 2024-05-08T05:56:44.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.969+0000 7fd8a7400700 1 --2- 172.21.15.112:0/892140079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a76c0 0x555db6796860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:44.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.969+0000 7fd8a6a00700 1 --2- 172.21.15.112:0/892140079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a1a50 0x555db67a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:44.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.969+0000 7fd8a6000700 1 --2- 172.21.15.112:0/892140079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a6da0 0x555db67b7210 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd89400c5b0 tx=0x7fd894091040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96651127310091bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:44.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.969+0000 7fd893e00700 1 -- 172.21.15.112:0/892140079 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8940a9990 con 0x555db67a6da0 2024-05-08T05:56:44.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.969+0000 7fd893e00700 1 -- 172.21.15.112:0/892140079 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8940add40 con 0x555db67a6da0 2024-05-08T05:56:44.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.969+0000 7fd893e00700 1 -- 172.21.15.112:0/892140079 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8940cc0b0 con 0x555db67a6da0 2024-05-08T05:56:44.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.970+0000 7fd8b878bb80 1 -- 172.21.15.112:0/892140079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a6da0 msgr2=0x555db67b7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:44.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.970+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/892140079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a6da0 0x555db67b7210 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd89400c5b0 tx=0x7fd894091040 comp rx=0 tx=0).stop 2024-05-08T05:56:44.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.970+0000 7fd8b878bb80 1 -- 172.21.15.112:0/892140079 shutdown_connections 2024-05-08T05:56:44.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.970+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/892140079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a76c0 0x555db6796860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.970+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/892140079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a1a50 0x555db67a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.970+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/892140079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a6da0 0x555db67b7210 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.970+0000 7fd8b878bb80 1 -- 172.21.15.112:0/892140079 >> 172.21.15.112:0/892140079 conn(0x555db67196d0 msgr2=0x555db67abcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.971+0000 7fd8b878bb80 1 -- 172.21.15.112:0/892140079 shutdown_connections 2024-05-08T05:56:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.971+0000 7fd8b878bb80 1 -- 172.21.15.112:0/892140079 wait complete. 2024-05-08T05:56:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.971+0000 7fd8b878bb80 1 -- start start 2024-05-08T05:56:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.972+0000 7fd8b878bb80 1 Processor -- start 2024-05-08T05:56:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.972+0000 7fd8b878bb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a1a50 0x555db67b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.972+0000 7fd8b878bb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a6da0 0x555db67b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.972+0000 7fd8b878bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 0x555db67b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.972+0000 7fd8b878bb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81ae2f0 con 0x555db67a6da0 2024-05-08T05:56:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.972+0000 7fd8b878bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81af760 con 0x555db67a76c0 2024-05-08T05:56:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.972+0000 7fd8b878bb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81ae9c0 con 0x555db67a1a50 2024-05-08T05:56:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.972+0000 7fd8b878bb80 10 client.0 _ll_register_callbacks cb 0x555db669ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:44.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.973+0000 7fd8a6a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a1a50 0x555db67b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.973+0000 7fd8a6a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a1a50 0x555db67b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:41724/0 (socket says 172.21.15.112:41724) 2024-05-08T05:56:44.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.973+0000 7fd8a6a00700 1 -- 172.21.15.112:0/1077645017 learned_addr learned my addr 172.21.15.112:0/1077645017 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:44.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.973+0000 7fd8a6000700 1 --2- 172.21.15.112:0/1077645017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a6da0 0x555db67b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.973+0000 7fd8a7400700 1 --2- 172.21.15.112:0/1077645017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 0x555db67b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.973+0000 7fd8a6a00700 1 -- 172.21.15.112:0/1077645017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 msgr2=0x555db67b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:44.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.973+0000 7fd8a6a00700 1 --2- 172.21.15.112:0/1077645017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 0x555db67b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.973+0000 7fd8a6a00700 1 -- 172.21.15.112:0/1077645017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a6da0 msgr2=0x555db67b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:44.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.973+0000 7fd8a6a00700 1 --2- 172.21.15.112:0/1077645017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a6da0 0x555db67b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.974+0000 7fd8a6a00700 1 -- 172.21.15.112:0/1077645017 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db67988a0 con 0x555db67a1a50 2024-05-08T05:56:44.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.974+0000 7fd8a6000700 1 --2- 172.21.15.112:0/1077645017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a6da0 0x555db67b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:44.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.974+0000 7fd8a7400700 1 --2- 172.21.15.112:0/1077645017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 0x555db67b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:44.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.974+0000 7fd8a6a00700 1 --2- 172.21.15.112:0/1077645017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a1a50 0x555db67b3590 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd898009230 tx=0x7fd898009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6417363ccfaa11c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:44.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.974+0000 7fd891600700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:56:44.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.974+0000 7fd891600700 1 -- 172.21.15.112:0/1077645017 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89803a070 con 0x555db67a1a50 2024-05-08T05:56:44.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.975+0000 7fd8b878bb80 1 -- 172.21.15.112:0/1077645017 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8940add40 con 0x555db67a1a50 2024-05-08T05:56:44.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.975+0000 7fd891600700 1 -- 172.21.15.112:0/1077645017 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd89802a7b0 con 0x555db67a1a50 2024-05-08T05:56:44.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.975+0000 7fd891600700 1 -- 172.21.15.112:0/1077645017 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd898044050 con 0x555db67a1a50 2024-05-08T05:56:44.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.975+0000 7fd8b878bb80 10 client.25336 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:44.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.975+0000 7fd8b878bb80 1 -- 172.21.15.112:0/1077645017 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555db68258c0 con 0x555db67a1a50 2024-05-08T05:56:44.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.976+0000 7fd891600700 1 -- 172.21.15.112:0/1077645017 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8940add40 con 0x555db67a1a50 2024-05-08T05:56:44.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.976+0000 7fd891600700 1 -- 172.21.15.112:0/1077645017 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd898036c80 con 0x555db67a1a50 2024-05-08T05:56:44.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.976+0000 7fd8b878bb80 10 client.25336 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:44.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.976+0000 7fd8b878bb80 10 client.25336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:56:44.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.976+0000 7fd8b878bb80 20 client.25336 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:44.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.976+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/1077645017 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x555db6825f60 0x555db6828350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.976+0000 7fd8b878bb80 4 client.25336 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:56:44.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.976+0000 7fd8b878bb80 1 -- 172.21.15.112:0/1077645017 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555db6701400 con 0x555db6825f60 2024-05-08T05:56:44.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.977+0000 7fd8a6000700 1 --2- 172.21.15.112:0/1077645017 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x555db6825f60 0x555db6828350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.978+0000 7fd8a6000700 1 --2- 172.21.15.112:0/1077645017 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x555db6825f60 0x555db6828350 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a950de5b5347c78 server_cookie=7453e94f10a69d67 in_seq=0 out_seq=0 2024-05-08T05:56:44.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.978+0000 7fd891600700 10 client.25336 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:44.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.978+0000 7fd891600700 1 -- 172.21.15.112:0/1077645017 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555db6701400 con 0x555db6825f60 2024-05-08T05:56:44.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.978+0000 7fd891600700 10 client.25336 handle_command_reply: tid=0 2024-05-08T05:56:44.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.978+0000 7fd8b878bb80 1 client.25336 shutdown 2024-05-08T05:56:44.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.978+0000 7fd8b878bb80 1 -- 172.21.15.112:0/1077645017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a1a50 msgr2=0x555db67b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:44.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.978+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/1077645017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a1a50 0x555db67b3590 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd898009230 tx=0x7fd898009260 comp rx=0 tx=0).stop 2024-05-08T05:56:44.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.979+0000 7fd8b878bb80 1 -- 172.21.15.112:0/1077645017 shutdown_connections 2024-05-08T05:56:44.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.979+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/1077645017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555db67a76c0 0x555db67b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.979+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/1077645017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555db67a1a50 0x555db67b3590 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.979+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/1077645017 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x555db6825f60 0x555db6828350 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.979+0000 7fd8b878bb80 1 --2- 172.21.15.112:0/1077645017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555db67a6da0 0x555db67b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.979+0000 7fd891600700 0 client.25336 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:56:44.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.979+0000 7fd8b878bb80 1 -- 172.21.15.112:0/1077645017 >> 172.21.15.112:0/1077645017 conn(0x555db67196d0 msgr2=0x555db67aa800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:44.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.979+0000 7fd8b878bb80 1 -- 172.21.15.112:0/1077645017 shutdown_connections 2024-05-08T05:56:44.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.979+0000 7fd8b878bb80 1 -- 172.21.15.112:0/1077645017 wait complete. 2024-05-08T05:56:44.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.979+0000 7fd8b878bb80 20 client.25336 trim_cache size 0 max 16384 2024-05-08T05:56:44.985 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:56:44.985 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-08T05:56:44.985 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-08T05:56:44.986 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:56:44.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.980+0000 7fd8a7e00700 1 -- 172.21.15.112:0/545128633 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd88c0b2900 msgr2=0x7fd88c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:44.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.980+0000 7fd8a7e00700 1 --2- 172.21.15.112:0/545128633 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd88c0b2900 0x7fd88c0b4dc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009da0 tx=0x7fd8a0009700 comp rx=0 tx=0).stop 2024-05-08T05:56:44.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.980+0000 7fd8a7e00700 1 -- 172.21.15.112:0/545128633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8a8103d90 msgr2=0x7fd8a81a1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:44.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.980+0000 7fd8a7e00700 1 --2- 172.21.15.112:0/545128633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8a8103d90 0x7fd8a81a1e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd89400e9c0 tx=0x7fd89400ecd0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.981+0000 7fd8a7e00700 1 -- 172.21.15.112:0/545128633 shutdown_connections 2024-05-08T05:56:44.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.981+0000 7fd8a7e00700 1 --2- 172.21.15.112:0/545128633 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd88c0b2900 0x7fd88c0b4dc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.981+0000 7fd8a7e00700 1 --2- 172.21.15.112:0/545128633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a8109460 0x7fd8a819ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.981+0000 7fd8a7e00700 1 --2- 172.21.15.112:0/545128633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8a81033d0 0x7fd8a81a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.981+0000 7fd8a7e00700 1 --2- 172.21.15.112:0/545128633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8a8103d90 0x7fd8a81a1e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.981+0000 7fd8a7e00700 1 -- 172.21.15.112:0/545128633 >> 172.21.15.112:0/545128633 conn(0x7fd8a80fef20 msgr2=0x7fd8a810ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:44.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.981+0000 7fd8a7e00700 1 -- 172.21.15.112:0/545128633 shutdown_connections 2024-05-08T05:56:44.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:44.981+0000 7fd8a7e00700 1 -- 172.21.15.112:0/545128633 wait complete. 2024-05-08T05:56:45.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:44 smithi112 ceph-mon[30003]: pgmap v484: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-08T05:56:45.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:44 smithi162 ceph-mon[34800]: pgmap v484: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-08T05:56:45.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:44 smithi150 ceph-mon[34315]: pgmap v484: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-08T05:56:46.115 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:56:46.115 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:56:46.551 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:47.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:46 smithi112 ceph-mon[30003]: pgmap v485: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:56:47.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:46 smithi162 ceph-mon[34800]: pgmap v485: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:56:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:46 smithi150 ceph-mon[34315]: pgmap v485: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:56:49.075 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:48 smithi112 ceph-mon[30003]: pgmap v486: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-08T05:56:49.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.088+0000 7f0cb0600700 1 -- 172.21.15.112:0/3876784258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ca8108850 msgr2=0x7f0ca81103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:49.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.088+0000 7f0cb0600700 1 --2- 172.21.15.112:0/3876784258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ca8108850 0x7f0ca81103d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000b0d0 tx=0x7f0ca000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.088+0000 7f0cb0600700 1 -- 172.21.15.112:0/3876784258 shutdown_connections 2024-05-08T05:56:49.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.088+0000 7f0cb0600700 1 --2- 172.21.15.112:0/3876784258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ca8108850 0x7f0ca81103d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.088+0000 7f0cb0600700 1 --2- 172.21.15.112:0/3876784258 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ca8103750 0x7f0ca8103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.088+0000 7f0cb0600700 1 --2- 172.21.15.112:0/3876784258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca8104100 0x7f0ca8108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.088+0000 7f0cb0600700 1 -- 172.21.15.112:0/3876784258 >> 172.21.15.112:0/3876784258 conn(0x7f0ca807a600 msgr2=0x7f0ca807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:49.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.088+0000 7f0cb0600700 1 -- 172.21.15.112:0/3876784258 shutdown_connections 2024-05-08T05:56:49.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.088+0000 7f0cb0600700 1 -- 172.21.15.112:0/3876784258 wait complete. 2024-05-08T05:56:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.089+0000 7f0cb0600700 1 Processor -- start 2024-05-08T05:56:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.089+0000 7f0cb0600700 1 -- start start 2024-05-08T05:56:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cb0600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca8103750 0x7f0ca819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cb0600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ca8104100 0x7f0ca819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cb0600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ca8108850 0x7f0ca8199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:49.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cb0600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca8113690 con 0x7f0ca8108850 2024-05-08T05:56:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cb0600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca8113510 con 0x7f0ca8103750 2024-05-08T05:56:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cb0600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca8113810 con 0x7f0ca8104100 2024-05-08T05:56:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cadc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca8103750 0x7f0ca819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cadc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca8103750 0x7f0ca819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:55176/0 (socket says 172.21.15.112:55176) 2024-05-08T05:56:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cadc00700 1 -- 172.21.15.112:0/1543836674 learned_addr learned my addr 172.21.15.112:0/1543836674 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cad200700 1 --2- 172.21.15.112:0/1543836674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ca8104100 0x7f0ca819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:49.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cae600700 1 --2- 172.21.15.112:0/1543836674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ca8108850 0x7f0ca8199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:49.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cad200700 1 -- 172.21.15.112:0/1543836674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca8103750 msgr2=0x7f0ca819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:49.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cad200700 1 --2- 172.21.15.112:0/1543836674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca8103750 0x7f0ca819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cad200700 1 -- 172.21.15.112:0/1543836674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ca8108850 msgr2=0x7f0ca8199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:49.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cad200700 1 --2- 172.21.15.112:0/1543836674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ca8108850 0x7f0ca8199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cad200700 1 -- 172.21.15.112:0/1543836674 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca819a070 con 0x7f0ca8104100 2024-05-08T05:56:49.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.090+0000 7f0cadc00700 1 --2- 172.21.15.112:0/1543836674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca8103750 0x7f0ca819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:49.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.091+0000 7f0cad200700 1 --2- 172.21.15.112:0/1543836674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ca8104100 0x7f0ca819fc90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400e9f0 tx=0x7f0c9400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:49.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.091+0000 7f0ca6000700 1 -- 172.21.15.112:0/1543836674 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9400c9c0 con 0x7f0ca8104100 2024-05-08T05:56:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.091+0000 7f0ca6000700 1 -- 172.21.15.112:0/1543836674 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c9400cb60 con 0x7f0ca8104100 2024-05-08T05:56:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.091+0000 7f0cb0600700 1 -- 172.21.15.112:0/1543836674 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca819a360 con 0x7f0ca8104100 2024-05-08T05:56:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.091+0000 7f0ca6000700 1 -- 172.21.15.112:0/1543836674 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c94018680 con 0x7f0ca8104100 2024-05-08T05:56:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.091+0000 7f0cb0600700 1 -- 172.21.15.112:0/1543836674 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca81a6620 con 0x7f0ca8104100 2024-05-08T05:56:49.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.092+0000 7f0cb0600700 1 -- 172.21.15.112:0/1543836674 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca810ba20 con 0x7f0ca8104100 2024-05-08T05:56:49.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.094+0000 7f0ca6000700 1 -- 172.21.15.112:0/1543836674 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0c94010430 con 0x7f0ca8104100 2024-05-08T05:56:49.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.096+0000 7f0ca6000700 1 --2- 172.21.15.112:0/1543836674 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0c900b2880 0x7f0c900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:49.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.096+0000 7f0ca6000700 1 -- 172.21.15.112:0/1543836674 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f0c940e23f0 con 0x7f0ca8104100 2024-05-08T05:56:49.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.096+0000 7f0cadc00700 1 --2- 172.21.15.112:0/1543836674 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0c900b2880 0x7f0c900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:49.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.097+0000 7f0cadc00700 1 --2- 172.21.15.112:0/1543836674 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0c900b2880 0x7f0c900b4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0ca819aa50 tx=0x7f0c9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:49.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.100+0000 7f0ca6000700 1 -- 172.21.15.112:0/1543836674 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0c940aa080 con 0x7f0ca8104100 2024-05-08T05:56:49.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:48 smithi162 ceph-mon[34800]: pgmap v486: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-08T05:56:49.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.378+0000 7f0cb0600700 1 -- 172.21.15.112:0/1543836674 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ca8104be0 con 0x7f0ca8104100 2024-05-08T05:56:49.381 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:56:49.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.380+0000 7f0ca6000700 1 -- 172.21.15.112:0/1543836674 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11891 (secure 0 0 0) 0x7f0c940b2100 con 0x7f0ca8104100 2024-05-08T05:56:49.381 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T05:54:58.248303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24518,24511]},"id":1}]} 2024-05-08T05:56:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.384+0000 7f0cb0600700 1 -- 172.21.15.112:0/1543836674 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0c900b2880 msgr2=0x7f0c900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.384+0000 7f0cb0600700 1 --2- 172.21.15.112:0/1543836674 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0c900b2880 0x7f0c900b4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0ca819aa50 tx=0x7f0c9c009260 comp rx=0 tx=0).stop 2024-05-08T05:56:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.384+0000 7f0cb0600700 1 -- 172.21.15.112:0/1543836674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ca8104100 msgr2=0x7f0ca819fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.384+0000 7f0cb0600700 1 --2- 172.21.15.112:0/1543836674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ca8104100 0x7f0ca819fc90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400e9f0 tx=0x7f0c9400ed00 comp rx=0 tx=0).stop 2024-05-08T05:56:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.384+0000 7f0cb0600700 1 -- 172.21.15.112:0/1543836674 shutdown_connections 2024-05-08T05:56:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.384+0000 7f0cb0600700 1 --2- 172.21.15.112:0/1543836674 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f0c900b2880 0x7f0c900b4d40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.384+0000 7f0cb0600700 1 --2- 172.21.15.112:0/1543836674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ca8108850 0x7f0ca8199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.384+0000 7f0cb0600700 1 --2- 172.21.15.112:0/1543836674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca8103750 0x7f0ca819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.384+0000 7f0cb0600700 1 --2- 172.21.15.112:0/1543836674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ca8104100 0x7f0ca819fc90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.384+0000 7f0cb0600700 1 -- 172.21.15.112:0/1543836674 >> 172.21.15.112:0/1543836674 conn(0x7f0ca807a600 msgr2=0x7f0ca8101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.384+0000 7f0cb0600700 1 -- 172.21.15.112:0/1543836674 shutdown_connections 2024-05-08T05:56:49.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:49.385+0000 7f0cb0600700 1 -- 172.21.15.112:0/1543836674 wait complete. 2024-05-08T05:56:49.386 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-05-08T05:56:49.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:48 smithi150 ceph-mon[34315]: pgmap v486: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-08T05:56:49.964 INFO:tasks.workunit.client.0.smithi112.stderr:Updating files: 58% (7139/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 96% (11846/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-08T05:56:50.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:49 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1543836674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:56:50.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:49 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1543836674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:56:50.423 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-08T05:56:50.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:49 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1543836674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:56:50.860 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:51.081 INFO:tasks.workunit.client.0.smithi112.stderr:Note: switching to 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a'. 2024-05-08T05:56:51.081 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-05-08T05:56:51.081 INFO:tasks.workunit.client.0.smithi112.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-08T05:56:51.081 INFO:tasks.workunit.client.0.smithi112.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-08T05:56:51.081 INFO:tasks.workunit.client.0.smithi112.stderr:state without impacting any branches by switching back to a branch. 2024-05-08T05:56:51.081 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-05-08T05:56:51.081 INFO:tasks.workunit.client.0.smithi112.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-08T05:56:51.081 INFO:tasks.workunit.client.0.smithi112.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-08T05:56:51.082 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-05-08T05:56:51.082 INFO:tasks.workunit.client.0.smithi112.stderr: git switch -c 2024-05-08T05:56:51.082 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-05-08T05:56:51.082 INFO:tasks.workunit.client.0.smithi112.stderr:Or undo this operation with: 2024-05-08T05:56:51.082 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-05-08T05:56:51.082 INFO:tasks.workunit.client.0.smithi112.stderr: git switch - 2024-05-08T05:56:51.082 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-05-08T05:56:51.082 INFO:tasks.workunit.client.0.smithi112.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-08T05:56:51.082 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-05-08T05:56:51.082 INFO:tasks.workunit.client.0.smithi112.stderr:HEAD is now at bee299f4ac3 mds: relax divergent backtrace scrub failures for replicated ancestor inodes 2024-05-08T05:56:51.096 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-08T05:56:51.196 INFO:tasks.workunit.client.0.smithi112.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-08T05:56:51.204 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T05:56:51.204 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-08T05:56:51.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:50 smithi112 ceph-mon[30003]: pgmap v487: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:56:51.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:50 smithi162 ceph-mon[34800]: pgmap v487: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:56:51.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:50 smithi150 ceph-mon[34315]: pgmap v487: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-08T05:56:53.177 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-08T05:56:53.238 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:52 smithi112 ceph-mon[30003]: pgmap v488: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-08T05:56:53.291 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-08T05:56:53.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:52 smithi162 ceph-mon[34800]: pgmap v488: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-08T05:56:53.370 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T05:56:53.375 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T05:56:53.375 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-08T05:56:53.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:52 smithi150 ceph-mon[34315]: pgmap v488: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-08T05:56:53.453 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T05:56:53.461 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T05:56:53.461 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-08T05:56:53.505 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-05-08T05:56:53.506 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-05-08T05:56:53.507 DEBUG:teuthology.orchestra.run.smithi112:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-05-08T05:56:53.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.555+0000 7f25db000700 1 -- 172.21.15.112:0/1100514995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25d4105ee0 msgr2=0x7f25d4106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:53.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.555+0000 7f25db000700 1 --2- 172.21.15.112:0/1100514995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25d4105ee0 0x7f25d4106370 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00b0a0 tx=0x7f25cc00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.555+0000 7f25db000700 1 -- 172.21.15.112:0/1100514995 shutdown_connections 2024-05-08T05:56:53.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.555+0000 7f25db000700 1 --2- 172.21.15.112:0/1100514995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25d4105ee0 0x7f25d4106370 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.555+0000 7f25db000700 1 --2- 172.21.15.112:0/1100514995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d41083d0 0x7f25d406b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.555+0000 7f25db000700 1 --2- 172.21.15.112:0/1100514995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d406b6f0 0x7f25d406bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.555+0000 7f25db000700 1 -- 172.21.15.112:0/1100514995 >> 172.21.15.112:0/1100514995 conn(0x7f25d4079350 msgr2=0x7f25d4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:53.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.555+0000 7f25db000700 1 -- 172.21.15.112:0/1100514995 shutdown_connections 2024-05-08T05:56:53.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.555+0000 7f25db000700 1 -- 172.21.15.112:0/1100514995 wait complete. 2024-05-08T05:56:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.556+0000 7f25db000700 1 Processor -- start 2024-05-08T05:56:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.556+0000 7f25db000700 1 -- start start 2024-05-08T05:56:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.557+0000 7f25db000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d406b6f0 0x7f25d41a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.557+0000 7f25db000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25d4105ee0 0x7f25d41a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.557+0000 7f25db000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d41083d0 0x7f25d41a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.557+0000 7f25db000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25d4119c30 con 0x7f25d41083d0 2024-05-08T05:56:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.557+0000 7f25db000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f25d4119ab0 con 0x7f25d406b6f0 2024-05-08T05:56:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.557+0000 7f25db000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f25d4119db0 con 0x7f25d4105ee0 2024-05-08T05:56:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.557+0000 7f25d3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d406b6f0 0x7f25d41a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.557+0000 7f25d3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d406b6f0 0x7f25d41a0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:55202/0 (socket says 172.21.15.112:55202) 2024-05-08T05:56:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.557+0000 7f25d3e00700 1 -- 172.21.15.112:0/2240599730 learned_addr learned my addr 172.21.15.112:0/2240599730 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:53.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.557+0000 7f25d3400700 1 --2- 172.21.15.112:0/2240599730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25d4105ee0 0x7f25d41a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.557+0000 7f25d9000700 1 --2- 172.21.15.112:0/2240599730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d41083d0 0x7f25d41a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.558+0000 7f25d3e00700 1 -- 172.21.15.112:0/2240599730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25d4105ee0 msgr2=0x7f25d41a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:53.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.558+0000 7f25d3e00700 1 --2- 172.21.15.112:0/2240599730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25d4105ee0 0x7f25d41a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.558+0000 7f25d3e00700 1 -- 172.21.15.112:0/2240599730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d41083d0 msgr2=0x7f25d41a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:53.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.558+0000 7f25d3e00700 1 --2- 172.21.15.112:0/2240599730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d41083d0 0x7f25d41a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.558+0000 7f25d3e00700 1 -- 172.21.15.112:0/2240599730 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d41a5150 con 0x7f25d406b6f0 2024-05-08T05:56:53.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.558+0000 7f25d3e00700 1 --2- 172.21.15.112:0/2240599730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d406b6f0 0x7f25d41a0140 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f25c400eb30 tx=0x7f25c400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:53.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.558+0000 7f25d0c00700 1 -- 172.21.15.112:0/2240599730 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c400cbd0 con 0x7f25d406b6f0 2024-05-08T05:56:53.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.559+0000 7f25d0c00700 1 -- 172.21.15.112:0/2240599730 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25c40044d0 con 0x7f25d406b6f0 2024-05-08T05:56:53.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.559+0000 7f25db000700 1 -- 172.21.15.112:0/2240599730 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d41a5440 con 0x7f25d406b6f0 2024-05-08T05:56:53.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.559+0000 7f25d0c00700 1 -- 172.21.15.112:0/2240599730 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4010430 con 0x7f25d406b6f0 2024-05-08T05:56:53.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.559+0000 7f25db000700 1 -- 172.21.15.112:0/2240599730 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d41accc0 con 0x7f25d406b6f0 2024-05-08T05:56:53.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.564+0000 7f25e59a7b80 1 Processor -- start 2024-05-08T05:56:53.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.564+0000 7f25e59a7b80 1 -- start start 2024-05-08T05:56:53.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.564+0000 7f25d0c00700 1 -- 172.21.15.112:0/2240599730 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f25c4003ed0 con 0x7f25d406b6f0 2024-05-08T05:56:53.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.564+0000 7f25e59a7b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af37d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.565+0000 7f25e59a7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af37d980 0x5558af381490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.565+0000 7f25e59a7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af3819d0 0x5558af381e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.565+0000 7f25e59a7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5558af2cfa20 con 0x5558af3819d0 2024-05-08T05:56:53.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.565+0000 7f25e59a7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5558af26d760 con 0x5558af37d980 2024-05-08T05:56:53.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.565+0000 7f25e59a7b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5558af2cb9e0 con 0x5558af37d060 2024-05-08T05:56:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.565+0000 7f25d3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af37d980 0x5558af381490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.565+0000 7f25d3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af37d980 0x5558af381490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:55214/0 (socket says 172.21.15.112:55214) 2024-05-08T05:56:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.565+0000 7f25d3e00700 1 -- 172.21.15.112:0/2464895919 learned_addr learned my addr 172.21.15.112:0/2464895919 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.565+0000 7f25d9000700 1 --2- 172.21.15.112:0/2464895919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af37d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.565+0000 7f25d3400700 1 --2- 172.21.15.112:0/2464895919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af3819d0 0x5558af381e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.566+0000 7f25d3e00700 1 -- 172.21.15.112:0/2464895919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 msgr2=0x5558af37d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.566+0000 7f25d3e00700 1 --2- 172.21.15.112:0/2464895919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af37d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.566+0000 7f25d3e00700 1 -- 172.21.15.112:0/2464895919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af3819d0 msgr2=0x5558af381e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:53.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.566+0000 7f25d3e00700 1 --2- 172.21.15.112:0/2464895919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af3819d0 0x5558af381e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.566+0000 7f25d3e00700 1 -- 172.21.15.112:0/2464895919 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558af21c8d0 con 0x5558af37d980 2024-05-08T05:56:53.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.566+0000 7f25d9000700 1 --2- 172.21.15.112:0/2464895919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af37d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:53.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.566+0000 7f25d3400700 1 --2- 172.21.15.112:0/2464895919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af3819d0 0x5558af381e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:53.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.566+0000 7f25d3e00700 1 --2- 172.21.15.112:0/2464895919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af37d980 0x5558af381490 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f25c40ea400 tx=0x7f25c40ea710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bb30c49a10dd6a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:53.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.566+0000 7f25d0c00700 1 --2- 172.21.15.112:0/2240599730 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f25b80b2900 0x7f25b80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.567+0000 7f25c9600700 1 -- 172.21.15.112:0/2464895919 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c40f6070 con 0x5558af37d980 2024-05-08T05:56:53.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.567+0000 7f25c9600700 1 -- 172.21.15.112:0/2464895919 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25c40a9510 con 0x5558af37d980 2024-05-08T05:56:53.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.567+0000 7f25c9600700 1 -- 172.21.15.112:0/2464895919 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c40a9170 con 0x5558af37d980 2024-05-08T05:56:53.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.567+0000 7f25d0c00700 1 -- 172.21.15.112:0/2240599730 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f25c40e2470 con 0x7f25d406b6f0 2024-05-08T05:56:53.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.567+0000 7f25d9000700 1 --2- 172.21.15.112:0/2240599730 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f25b80b2900 0x7f25b80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.568+0000 7f25e59a7b80 1 -- 172.21.15.112:0/2464895919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af37d980 msgr2=0x5558af381490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:53.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.568+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/2464895919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af37d980 0x5558af381490 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f25c40ea400 tx=0x7f25c40ea710 comp rx=0 tx=0).stop 2024-05-08T05:56:53.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.568+0000 7f25d9000700 1 --2- 172.21.15.112:0/2240599730 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f25b80b2900 0x7f25b80b4dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009e60 tx=0x7f25cc00b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:53.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.568+0000 7f25e59a7b80 1 -- 172.21.15.112:0/2464895919 shutdown_connections 2024-05-08T05:56:53.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.568+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/2464895919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af3819d0 0x5558af381e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.568+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/2464895919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af37d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.568+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/2464895919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af37d980 0x5558af381490 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.568+0000 7f25e59a7b80 1 -- 172.21.15.112:0/2464895919 >> 172.21.15.112:0/2464895919 conn(0x5558af2abb60 msgr2=0x5558af3777a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:53.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.568+0000 7f25e59a7b80 1 -- 172.21.15.112:0/2464895919 shutdown_connections 2024-05-08T05:56:53.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.568+0000 7f25e59a7b80 1 -- 172.21.15.112:0/2464895919 wait complete. 2024-05-08T05:56:53.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.569+0000 7f25e59a7b80 1 -- start start 2024-05-08T05:56:53.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.569+0000 7f25e59a7b80 1 Processor -- start 2024-05-08T05:56:53.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.569+0000 7f25e59a7b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af388e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25e59a7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 0x5558af3861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25e59a7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af3819d0 0x5558af386720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25e59a7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5558af389be0 con 0x5558af37d980 2024-05-08T05:56:53.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25e59a7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5558af235b20 con 0x5558af3819d0 2024-05-08T05:56:53.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25e59a7b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5558af389d60 con 0x5558af37d060 2024-05-08T05:56:53.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25e59a7b80 10 client.0 _ll_register_callbacks cb 0x5558af276c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:53.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25d3e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af388e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25d3e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af388e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:41778/0 (socket says 172.21.15.112:41778) 2024-05-08T05:56:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25d3e00700 1 -- 172.21.15.112:0/3788118788 learned_addr learned my addr 172.21.15.112:0/3788118788 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25d3400700 1 --2- 172.21.15.112:0/3788118788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 0x5558af3861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25d9000700 1 --2- 172.21.15.112:0/3788118788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af3819d0 0x5558af386720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25d3e00700 1 -- 172.21.15.112:0/3788118788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af3819d0 msgr2=0x5558af386720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25d3e00700 1 --2- 172.21.15.112:0/3788118788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af3819d0 0x5558af386720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25d3e00700 1 -- 172.21.15.112:0/3788118788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 msgr2=0x5558af3861e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25d3e00700 1 --2- 172.21.15.112:0/3788118788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 0x5558af3861e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.570+0000 7f25d3e00700 1 -- 172.21.15.112:0/3788118788 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558af295420 con 0x5558af37d060 2024-05-08T05:56:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.571+0000 7f25d3400700 1 --2- 172.21.15.112:0/3788118788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 0x5558af3861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.571+0000 7f25d3e00700 1 --2- 172.21.15.112:0/3788118788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af388e40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f25c400c5b0 tx=0x7f25c4003870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42441cbf8267986d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:53.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.571+0000 7f25be000700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:56:53.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.571+0000 7f25be000700 1 -- 172.21.15.112:0/3788118788 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c40a9170 con 0x5558af37d060 2024-05-08T05:56:53.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.571+0000 7f25be000700 1 -- 172.21.15.112:0/3788118788 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f25c4073960 con 0x5558af37d060 2024-05-08T05:56:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.571+0000 7f25e59a7b80 1 -- 172.21.15.112:0/3788118788 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c40a9510 con 0x5558af37d060 2024-05-08T05:56:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.571+0000 7f25be000700 1 -- 172.21.15.112:0/3788118788 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f25c40a5190 con 0x5558af37d060 2024-05-08T05:56:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.571+0000 7f25e59a7b80 10 client.25354 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.571+0000 7f25e59a7b80 1 -- 172.21.15.112:0/3788118788 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5558af389be0 con 0x5558af37d060 2024-05-08T05:56:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.572+0000 7f25be000700 1 -- 172.21.15.112:0/3788118788 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25c40a9510 con 0x5558af37d060 2024-05-08T05:56:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.572+0000 7f25be000700 1 -- 172.21.15.112:0/3788118788 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f25c40a5c90 con 0x5558af37d060 2024-05-08T05:56:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.572+0000 7f25e59a7b80 10 client.25354 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.572+0000 7f25e59a7b80 10 client.25354 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T05:56:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.572+0000 7f25e59a7b80 20 client.25354 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.572+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/3788118788 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5558af396770 0x5558af398b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.572+0000 7f25e59a7b80 4 client.25354 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.572+0000 7f25e59a7b80 1 -- 172.21.15.112:0/3788118788 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558af34a0a0 con 0x5558af396770 2024-05-08T05:56:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.572+0000 7f25d3400700 1 --2- 172.21.15.112:0/3788118788 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5558af396770 0x5558af398b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.573+0000 7f25d3400700 1 --2- 172.21.15.112:0/3788118788 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5558af396770 0x5558af398b60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0564e37a2fbc49c server_cookie=ab94d9ae446db37c in_seq=0 out_seq=0 2024-05-08T05:56:53.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.573+0000 7f25be000700 10 client.25354 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:56:53.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.574+0000 7f25be000700 1 -- 172.21.15.112:0/3788118788 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5558af34a0a0 con 0x5558af396770 2024-05-08T05:56:53.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.574+0000 7f25be000700 10 client.25354 handle_command_reply: tid=0 2024-05-08T05:56:53.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.574+0000 7f25e59a7b80 1 client.25354 shutdown 2024-05-08T05:56:53.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.575+0000 7f25e59a7b80 1 -- 172.21.15.112:0/3788118788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 msgr2=0x5558af388e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:53.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.575+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/3788118788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af388e40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f25c400c5b0 tx=0x7f25c4003870 comp rx=0 tx=0).stop 2024-05-08T05:56:53.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.575+0000 7f25e59a7b80 1 -- 172.21.15.112:0/3788118788 shutdown_connections 2024-05-08T05:56:53.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.575+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/3788118788 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5558af396770 0x5558af398b60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.575+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/3788118788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af3819d0 0x5558af386720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.575+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/3788118788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af388e40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.575+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/3788118788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 0x5558af3861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.575+0000 7f25e59a7b80 1 -- 172.21.15.112:0/3788118788 >> 172.21.15.112:0/3788118788 conn(0x5558af2abb60 msgr2=0x5558af3722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:53.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.575+0000 7f25be000700 0 client.25354 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:56:53.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.575+0000 7f25e59a7b80 1 -- 172.21.15.112:0/3788118788 shutdown_connections 2024-05-08T05:56:53.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.575+0000 7f25e59a7b80 1 -- 172.21.15.112:0/3788118788 wait complete. 2024-05-08T05:56:53.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.576+0000 7f25e59a7b80 20 client.25354 trim_cache size 0 max 16384 2024-05-08T05:56:53.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.595+0000 7f25e59a7b80 1 Processor -- start 2024-05-08T05:56:53.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.595+0000 7f25e59a7b80 1 -- start start 2024-05-08T05:56:53.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.596+0000 7f25e59a7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af37d060 0x5558af386810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.596+0000 7f25e59a7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 0x5558af392ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.596+0000 7f25e59a7b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af3819d0 0x5558af372130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.596+0000 7f25e59a7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25d41b6b80 con 0x5558af37d980 2024-05-08T05:56:53.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.596+0000 7f25e59a7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f25d41b4410 con 0x5558af37d060 2024-05-08T05:56:53.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.596+0000 7f25e59a7b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f25d41b2260 con 0x5558af3819d0 2024-05-08T05:56:53.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.596+0000 7f25d3400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 0x5558af392ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.596+0000 7f25d9000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af3819d0 0x5558af372130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.596+0000 7f25d3400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 0x5558af392ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52592/0 (socket says 172.21.15.112:52592) 2024-05-08T05:56:53.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.596+0000 7f25d3400700 1 -- 172.21.15.112:0/879946083 learned_addr learned my addr 172.21.15.112:0/879946083 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:53.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.596+0000 7f25d3e00700 1 --2- 172.21.15.112:0/879946083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af37d060 0x5558af386810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.597+0000 7f25d9000700 1 -- 172.21.15.112:0/879946083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af37d060 msgr2=0x5558af386810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:53.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.597+0000 7f25d9000700 1 --2- 172.21.15.112:0/879946083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af37d060 0x5558af386810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.597+0000 7f25d9000700 1 -- 172.21.15.112:0/879946083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 msgr2=0x5558af392ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:53.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.597+0000 7f25d9000700 1 --2- 172.21.15.112:0/879946083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 0x5558af392ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.597+0000 7f25d9000700 1 -- 172.21.15.112:0/879946083 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558af3743f0 con 0x5558af3819d0 2024-05-08T05:56:53.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.597+0000 7f25d9000700 1 --2- 172.21.15.112:0/879946083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af3819d0 0x5558af372130 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00d010 tx=0x7f25cc01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=893210e2471edf99 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:53.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.597+0000 7f25c8c00700 1 -- 172.21.15.112:0/879946083 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc003c00 con 0x5558af3819d0 2024-05-08T05:56:53.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.598+0000 7f25c8c00700 1 -- 172.21.15.112:0/879946083 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25cc003da0 con 0x5558af3819d0 2024-05-08T05:56:53.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.598+0000 7f25c8c00700 1 -- 172.21.15.112:0/879946083 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0327b0 con 0x5558af3819d0 2024-05-08T05:56:53.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.598+0000 7f25e59a7b80 1 -- 172.21.15.112:0/879946083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af3819d0 msgr2=0x5558af372130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:53.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.598+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/879946083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af3819d0 0x5558af372130 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00d010 tx=0x7f25cc01f000 comp rx=0 tx=0).stop 2024-05-08T05:56:53.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.598+0000 7f25e59a7b80 1 -- 172.21.15.112:0/879946083 shutdown_connections 2024-05-08T05:56:53.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.599+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/879946083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af3819d0 0x5558af372130 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.599+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/879946083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af37d060 0x5558af386810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.599+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/879946083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 0x5558af392ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.599+0000 7f25e59a7b80 1 -- 172.21.15.112:0/879946083 >> 172.21.15.112:0/879946083 conn(0x5558af2abb60 msgr2=0x5558af371a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:53.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.599+0000 7f25e59a7b80 1 -- 172.21.15.112:0/879946083 shutdown_connections 2024-05-08T05:56:53.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.599+0000 7f25e59a7b80 1 -- 172.21.15.112:0/879946083 wait complete. 2024-05-08T05:56:53.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.599+0000 7f25e59a7b80 1 -- start start 2024-05-08T05:56:53.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.600+0000 7f25e59a7b80 1 Processor -- start 2024-05-08T05:56:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.600+0000 7f25e59a7b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af38f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.600+0000 7f25e59a7b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 0x5558af38c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.600+0000 7f25e59a7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af3819d0 0x5558af38ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.600+0000 7f25e59a7b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25d41b28d0 con 0x5558af37d980 2024-05-08T05:56:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.600+0000 7f25e59a7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f25d41b3d40 con 0x5558af3819d0 2024-05-08T05:56:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.600+0000 7f25e59a7b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f25d41b2fa0 con 0x5558af37d060 2024-05-08T05:56:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.600+0000 7f25e59a7b80 10 client.0 _ll_register_callbacks cb 0x5558af276c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.600+0000 7f25d3e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af38f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.600+0000 7f25d9000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af3819d0 0x5558af38ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.600+0000 7f25d3e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af38f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:41794/0 (socket says 172.21.15.112:41794) 2024-05-08T05:56:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.600+0000 7f25d3e00700 1 -- 172.21.15.112:0/1387349358 learned_addr learned my addr 172.21.15.112:0/1387349358 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.600+0000 7f25d3400700 1 --2- 172.21.15.112:0/1387349358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 0x5558af38c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.601+0000 7f25d3e00700 1 -- 172.21.15.112:0/1387349358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af3819d0 msgr2=0x5558af38ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.601+0000 7f25d3e00700 1 --2- 172.21.15.112:0/1387349358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af3819d0 0x5558af38ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.601+0000 7f25d3e00700 1 -- 172.21.15.112:0/1387349358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 msgr2=0x5558af38c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.601+0000 7f25d3e00700 1 --2- 172.21.15.112:0/1387349358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 0x5558af38c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.601+0000 7f25d3e00700 1 -- 172.21.15.112:0/1387349358 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558af38d660 con 0x5558af37d060 2024-05-08T05:56:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.601+0000 7f25d9000700 1 --2- 172.21.15.112:0/1387349358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af3819d0 0x5558af38ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.601+0000 7f25d3e00700 1 --2- 172.21.15.112:0/1387349358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af38f570 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f25c40a6c70 tx=0x7f25c40a5f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a50ebc95c0c5ba36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:53.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.601+0000 7f25be000700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:56:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.602+0000 7f25be000700 1 -- 172.21.15.112:0/1387349358 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c40a9350 con 0x5558af37d060 2024-05-08T05:56:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.602+0000 7f25be000700 1 -- 172.21.15.112:0/1387349358 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f25c40a5ff0 con 0x5558af37d060 2024-05-08T05:56:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.602+0000 7f25e59a7b80 1 -- 172.21.15.112:0/1387349358 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25cc003da0 con 0x5558af37d060 2024-05-08T05:56:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.602+0000 7f25be000700 1 -- 172.21.15.112:0/1387349358 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f25c40a97d0 con 0x5558af37d060 2024-05-08T05:56:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.602+0000 7f25e59a7b80 10 client.25360 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.602+0000 7f25e59a7b80 1 -- 172.21.15.112:0/1387349358 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5558af400d60 con 0x5558af37d060 2024-05-08T05:56:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.602+0000 7f25be000700 1 -- 172.21.15.112:0/1387349358 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25cc003da0 con 0x5558af37d060 2024-05-08T05:56:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.603+0000 7f25be000700 1 -- 172.21.15.112:0/1387349358 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f25c40b23d0 con 0x5558af37d060 2024-05-08T05:56:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.603+0000 7f25e59a7b80 10 client.25360 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:53.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.603+0000 7f25e59a7b80 10 client.25360 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T05:56:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.603+0000 7f25e59a7b80 20 client.25360 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.603+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/1387349358 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5558af401170 0x5558af403560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.603+0000 7f25e59a7b80 4 client.25360 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:56:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.603+0000 7f25e59a7b80 1 -- 172.21.15.112:0/1387349358 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558af2dd400 con 0x5558af401170 2024-05-08T05:56:53.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.603+0000 7f25d3400700 1 --2- 172.21.15.112:0/1387349358 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5558af401170 0x5558af403560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:53.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.604+0000 7f25be000700 10 client.25360 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:56:53.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.604+0000 7f25d3400700 1 --2- 172.21.15.112:0/1387349358 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5558af401170 0x5558af403560 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=800d1c073923fdea server_cookie=d353f9a40f087eaa in_seq=0 out_seq=0 2024-05-08T05:56:53.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.604+0000 7f25be000700 1 -- 172.21.15.112:0/1387349358 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558af2dd400 con 0x5558af401170 2024-05-08T05:56:53.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.604+0000 7f25be000700 10 client.25360 handle_command_reply: tid=0 2024-05-08T05:56:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.604+0000 7f25e59a7b80 1 client.25360 shutdown 2024-05-08T05:56:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.605+0000 7f25e59a7b80 1 -- 172.21.15.112:0/1387349358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 msgr2=0x5558af38f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.605+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/1387349358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af38f570 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f25c40a6c70 tx=0x7f25c40a5f40 comp rx=0 tx=0).stop 2024-05-08T05:56:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.605+0000 7f25e59a7b80 1 -- 172.21.15.112:0/1387349358 shutdown_connections 2024-05-08T05:56:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.605+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/1387349358 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5558af401170 0x5558af403560 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.605+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/1387349358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558af3819d0 0x5558af38ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.605+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/1387349358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5558af37d060 0x5558af38f570 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.605+0000 7f25e59a7b80 1 --2- 172.21.15.112:0/1387349358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558af37d980 0x5558af38c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.605+0000 7f25e59a7b80 1 -- 172.21.15.112:0/1387349358 >> 172.21.15.112:0/1387349358 conn(0x5558af2abb60 msgr2=0x5558af3722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.605+0000 7f25be000700 0 client.25360 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:56:53.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.605+0000 7f25e59a7b80 1 -- 172.21.15.112:0/1387349358 shutdown_connections 2024-05-08T05:56:53.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.605+0000 7f25e59a7b80 1 -- 172.21.15.112:0/1387349358 wait complete. 2024-05-08T05:56:53.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.605+0000 7f25e59a7b80 20 client.25360 trim_cache size 0 max 16384 2024-05-08T05:56:53.610 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:56:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.606+0000 7f25db000700 1 -- 172.21.15.112:0/2240599730 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f25b80b2900 msgr2=0x7f25b80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.606+0000 7f25db000700 1 --2- 172.21.15.112:0/2240599730 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f25b80b2900 0x7f25b80b4dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009e60 tx=0x7f25cc00b720 comp rx=0 tx=0).stop 2024-05-08T05:56:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.606+0000 7f25db000700 1 -- 172.21.15.112:0/2240599730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d406b6f0 msgr2=0x7f25d41a0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.606+0000 7f25db000700 1 --2- 172.21.15.112:0/2240599730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d406b6f0 0x7f25d41a0140 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f25c400eb30 tx=0x7f25c400ee40 comp rx=0 tx=0).stop 2024-05-08T05:56:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.606+0000 7f25db000700 1 -- 172.21.15.112:0/2240599730 shutdown_connections 2024-05-08T05:56:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.606+0000 7f25db000700 1 --2- 172.21.15.112:0/2240599730 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f25b80b2900 0x7f25b80b4dc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.606+0000 7f25db000700 1 --2- 172.21.15.112:0/2240599730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d41083d0 0x7f25d41a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.606+0000 7f25db000700 1 --2- 172.21.15.112:0/2240599730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25d406b6f0 0x7f25d41a0140 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.606+0000 7f25db000700 1 --2- 172.21.15.112:0/2240599730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25d4105ee0 0x7f25d41a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:53.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.606+0000 7f25db000700 1 -- 172.21.15.112:0/2240599730 >> 172.21.15.112:0/2240599730 conn(0x7f25d4079350 msgr2=0x7f25d4104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:53.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.607+0000 7f25db000700 1 -- 172.21.15.112:0/2240599730 shutdown_connections 2024-05-08T05:56:53.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:53.607+0000 7f25db000700 1 -- 172.21.15.112:0/2240599730 wait complete. 2024-05-08T05:56:53.612 INFO:tasks.workunit.client.0.smithi112.stderr:+ set -e 2024-05-08T05:56:53.612 INFO:tasks.workunit.client.0.smithi112.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-05-08T05:56:53.614 INFO:tasks.workunit.client.0.smithi112.stderr:Cloning into 'xfstests-dev'... 2024-05-08T05:56:54.644 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-08T05:56:55.131 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:55.289 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:54 smithi112 ceph-mon[30003]: pgmap v489: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T05:56:55.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:54 smithi162 ceph-mon[34800]: pgmap v489: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T05:56:55.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:54 smithi150 ceph-mon[34315]: pgmap v489: 97 pgs: 97 active+clean; 658 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T05:56:57.063 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:56 smithi112 ceph-mon[30003]: pgmap v490: 97 pgs: 97 active+clean; 789 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-08T05:56:57.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:56 smithi162 ceph-mon[34800]: pgmap v490: 97 pgs: 97 active+clean; 789 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-08T05:56:57.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:56 smithi150 ceph-mon[34315]: pgmap v490: 97 pgs: 97 active+clean; 789 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-08T05:56:57.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.634+0000 7f1c3b600700 1 -- 172.21.15.112:0/567140235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c3410a490 msgr2=0x7f1c341169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:57.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.634+0000 7f1c3b600700 1 --2- 172.21.15.112:0/567140235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c3410a490 0x7f1c341169a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c009f90 tx=0x7f1c2c00b330 comp rx=0 tx=0).stop 2024-05-08T05:56:57.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.634+0000 7f1c3b600700 1 -- 172.21.15.112:0/567140235 shutdown_connections 2024-05-08T05:56:57.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.634+0000 7f1c3b600700 1 --2- 172.21.15.112:0/567140235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c3410a490 0x7f1c341169a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.634+0000 7f1c3b600700 1 --2- 172.21.15.112:0/567140235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c3410fb60 0x7f1c3410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.634+0000 7f1c3b600700 1 --2- 172.21.15.112:0/567140235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c34109ad0 0x7f1c34109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.634+0000 7f1c3b600700 1 -- 172.21.15.112:0/567140235 >> 172.21.15.112:0/567140235 conn(0x7f1c34076c70 msgr2=0x7f1c341082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:57.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.634+0000 7f1c3b600700 1 -- 172.21.15.112:0/567140235 shutdown_connections 2024-05-08T05:56:57.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.634+0000 7f1c3b600700 1 -- 172.21.15.112:0/567140235 wait complete. 2024-05-08T05:56:57.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.635+0000 7f1c3b600700 1 Processor -- start 2024-05-08T05:56:57.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.636+0000 7f1c3b600700 1 -- start start 2024-05-08T05:56:57.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.636+0000 7f1c3b600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c34109ad0 0x7f1c341a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.636+0000 7f1c3b600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c3410a490 0x7f1c341a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.636+0000 7f1c3b600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c3410fb60 0x7f1c341a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.636+0000 7f1c3b600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c34119cc0 con 0x7f1c34109ad0 2024-05-08T05:56:57.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.636+0000 7f1c3b600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1c34119b40 con 0x7f1c3410a490 2024-05-08T05:56:57.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.636+0000 7f1c3b600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1c34119e40 con 0x7f1c3410fb60 2024-05-08T05:56:57.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.636+0000 7f1c38c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c34109ad0 0x7f1c341a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.636+0000 7f1c38c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c34109ad0 0x7f1c341a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55532/0 (socket says 172.21.15.112:55532) 2024-05-08T05:56:57.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.636+0000 7f1c38c00700 1 -- 172.21.15.112:0/2569451225 learned_addr learned my addr 172.21.15.112:0/2569451225 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:57.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.636+0000 7f1c33e00700 1 --2- 172.21.15.112:0/2569451225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c3410a490 0x7f1c341a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.637+0000 7f1c39600700 1 --2- 172.21.15.112:0/2569451225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c3410fb60 0x7f1c341a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.637+0000 7f1c38c00700 1 -- 172.21.15.112:0/2569451225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c3410fb60 msgr2=0x7f1c341a49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:57.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.637+0000 7f1c38c00700 1 --2- 172.21.15.112:0/2569451225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c3410fb60 0x7f1c341a49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.637+0000 7f1c38c00700 1 -- 172.21.15.112:0/2569451225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c3410a490 msgr2=0x7f1c341a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:57.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.637+0000 7f1c38c00700 1 --2- 172.21.15.112:0/2569451225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c3410a490 0x7f1c341a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.637+0000 7f1c38c00700 1 -- 172.21.15.112:0/2569451225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c341a5100 con 0x7f1c34109ad0 2024-05-08T05:56:57.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.637+0000 7f1c38c00700 1 --2- 172.21.15.112:0/2569451225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c34109ad0 0x7f1c341a00f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800ec10 tx=0x7f1c2800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:57.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.637+0000 7f1c31600700 1 -- 172.21.15.112:0/2569451225 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2800cc90 con 0x7f1c34109ad0 2024-05-08T05:56:57.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.638+0000 7f1c3b600700 1 -- 172.21.15.112:0/2569451225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c341a53f0 con 0x7f1c34109ad0 2024-05-08T05:56:57.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.638+0000 7f1c31600700 1 -- 172.21.15.112:0/2569451225 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c2800ce30 con 0x7f1c34109ad0 2024-05-08T05:56:57.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.638+0000 7f1c31600700 1 -- 172.21.15.112:0/2569451225 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28010430 con 0x7f1c34109ad0 2024-05-08T05:56:57.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.638+0000 7f1c3b600700 1 -- 172.21.15.112:0/2569451225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c341acc70 con 0x7f1c34109ad0 2024-05-08T05:56:57.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.642+0000 7f1c31600700 1 -- 172.21.15.112:0/2569451225 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1c28004000 con 0x7f1c34109ad0 2024-05-08T05:56:57.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.643+0000 7f1c45fc5b80 1 Processor -- start 2024-05-08T05:56:57.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.643+0000 7f1c45fc5b80 1 -- start start 2024-05-08T05:56:57.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.643+0000 7f1c31600700 1 --2- 172.21.15.112:0/2569451225 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1c1c0b2900 0x7f1c1c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.643+0000 7f1c45fc5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a5fa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.643+0000 7f1c45fc5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fa980 0x561c2a5fe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c45fc5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fe9d0 0x561c2a5fee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c45fc5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561c2a54ca20 con 0x561c2a5fa060 2024-05-08T05:56:57.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c45fc5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561c2a4ea760 con 0x561c2a5fe9d0 2024-05-08T05:56:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c45fc5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561c2a5489e0 con 0x561c2a5fa980 2024-05-08T05:56:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c31600700 1 -- 172.21.15.112:0/2569451225 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f1c280e2440 con 0x7f1c34109ad0 2024-05-08T05:56:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c38c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a5fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c39600700 1 --2- 172.21.15.112:0/2569451225 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1c1c0b2900 0x7f1c1c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c38c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a5fa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55546/0 (socket says 172.21.15.112:55546) 2024-05-08T05:56:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c38c00700 1 -- 172.21.15.112:0/1569926230 learned_addr learned my addr 172.21.15.112:0/1569926230 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c33e00700 1 --2- 172.21.15.112:0/1569926230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fa980 0x561c2a5fe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c39600700 1 --2- 172.21.15.112:0/1569926230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fe9d0 0x561c2a5fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c38c00700 1 -- 172.21.15.112:0/1569926230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fa980 msgr2=0x561c2a5fe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c38c00700 1 --2- 172.21.15.112:0/1569926230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fa980 0x561c2a5fe490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c38c00700 1 -- 172.21.15.112:0/1569926230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fe9d0 msgr2=0x561c2a5fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c38c00700 1 --2- 172.21.15.112:0/1569926230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fe9d0 0x561c2a5fee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.644+0000 7f1c38c00700 1 -- 172.21.15.112:0/1569926230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c2a4998d0 con 0x561c2a5fa060 2024-05-08T05:56:57.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.645+0000 7f1c38c00700 1 --2- 172.21.15.112:0/1569926230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a5fa440 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1c280a6c40 tx=0x7f1c280a9820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4d6c8d56f3ee4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.645+0000 7f1c39600700 1 --2- 172.21.15.112:0/2569451225 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1c1c0b2900 0x7f1c1c0b4dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00b6c0 tx=0x7f1c2c003bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.645+0000 7f1c26000700 1 -- 172.21.15.112:0/1569926230 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28097510 con 0x561c2a5fa060 2024-05-08T05:56:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.645+0000 7f1c26000700 1 -- 172.21.15.112:0/1569926230 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c280b2180 con 0x561c2a5fa060 2024-05-08T05:56:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.645+0000 7f1c26000700 1 -- 172.21.15.112:0/1569926230 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c280a5160 con 0x561c2a5fa060 2024-05-08T05:56:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.646+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/1569926230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 msgr2=0x561c2a5fa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.646+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/1569926230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a5fa440 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1c280a6c40 tx=0x7f1c280a9820 comp rx=0 tx=0).stop 2024-05-08T05:56:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.646+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/1569926230 shutdown_connections 2024-05-08T05:56:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.646+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/1569926230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fe9d0 0x561c2a5fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.646+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/1569926230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a5fa440 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.646+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/1569926230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fa980 0x561c2a5fe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.646+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/1569926230 >> 172.21.15.112:0/1569926230 conn(0x561c2a528b60 msgr2=0x561c2a5f47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.646+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/1569926230 shutdown_connections 2024-05-08T05:56:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.646+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/1569926230 wait complete. 2024-05-08T05:56:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.647+0000 7f1c45fc5b80 1 -- start start 2024-05-08T05:56:57.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.647+0000 7f1c45fc5b80 1 Processor -- start 2024-05-08T05:56:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c45fc5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a605e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c45fc5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fa980 0x561c2a6031c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c45fc5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fe9d0 0x561c2a603700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c45fc5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561c2a606b20 con 0x561c2a5fa060 2024-05-08T05:56:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c45fc5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561c2a4b2b20 con 0x561c2a5fa980 2024-05-08T05:56:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c45fc5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561c2a606ca0 con 0x561c2a5fe9d0 2024-05-08T05:56:57.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c45fc5b80 10 client.0 _ll_register_callbacks cb 0x561c2a4f3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c38c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a605e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c38c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a605e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55560/0 (socket says 172.21.15.112:55560) 2024-05-08T05:56:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c38c00700 1 -- 172.21.15.112:0/661321683 learned_addr learned my addr 172.21.15.112:0/661321683 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c33e00700 1 --2- 172.21.15.112:0/661321683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fa980 0x561c2a6031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c39600700 1 --2- 172.21.15.112:0/661321683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fe9d0 0x561c2a603700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c38c00700 1 -- 172.21.15.112:0/661321683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fe9d0 msgr2=0x561c2a603700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c38c00700 1 --2- 172.21.15.112:0/661321683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fe9d0 0x561c2a603700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c38c00700 1 -- 172.21.15.112:0/661321683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fa980 msgr2=0x561c2a6031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.648+0000 7f1c38c00700 1 --2- 172.21.15.112:0/661321683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fa980 0x561c2a6031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.649+0000 7f1c38c00700 1 -- 172.21.15.112:0/661321683 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c2a512420 con 0x561c2a5fa060 2024-05-08T05:56:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.649+0000 7f1c38c00700 1 --2- 172.21.15.112:0/661321683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a605e20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1c28003fd0 tx=0x7f1c280038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527f9598a26fa2cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.649+0000 7f1c1aa00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:56:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.649+0000 7f1c1aa00700 1 -- 172.21.15.112:0/661321683 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28004500 con 0x561c2a5fa060 2024-05-08T05:56:57.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.649+0000 7f1c1aa00700 1 -- 172.21.15.112:0/661321683 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f1c280972a0 con 0x561c2a5fa060 2024-05-08T05:56:57.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.649+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/661321683 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c280b2180 con 0x561c2a5fa060 2024-05-08T05:56:57.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.650+0000 7f1c1aa00700 1 -- 172.21.15.112:0/661321683 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c280a99d0 con 0x561c2a5fa060 2024-05-08T05:56:57.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.650+0000 7f1c45fc5b80 10 client.15225 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:57.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.650+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/661321683 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c2a611360 con 0x561c2a5fa060 2024-05-08T05:56:57.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.650+0000 7f1c1aa00700 1 -- 172.21.15.112:0/661321683 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c280b2180 con 0x561c2a5fa060 2024-05-08T05:56:57.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.650+0000 7f1c1aa00700 1 -- 172.21.15.112:0/661321683 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f1c280a9c30 con 0x561c2a5fa060 2024-05-08T05:56:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.650+0000 7f1c45fc5b80 10 client.15225 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.650+0000 7f1c45fc5b80 10 client.15225 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T05:56:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.650+0000 7f1c45fc5b80 20 client.15225 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.650+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/661321683 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561c2a613830 0x561c2a615c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.650+0000 7f1c45fc5b80 4 client.15225 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.650+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/661321683 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c2a5c70a0 con 0x561c2a613830 2024-05-08T05:56:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.651+0000 7f1c33e00700 1 --2- 172.21.15.112:0/661321683 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561c2a613830 0x561c2a615c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.651+0000 7f1c33e00700 1 --2- 172.21.15.112:0/661321683 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561c2a613830 0x561c2a615c20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2458b6b7e725601f server_cookie=5486c75366f6b5f0 in_seq=0 out_seq=0 2024-05-08T05:56:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.651+0000 7f1c1aa00700 10 client.15225 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T05:56:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.652+0000 7f1c1aa00700 1 -- 172.21.15.112:0/661321683 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561c2a5c70a0 con 0x561c2a613830 2024-05-08T05:56:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.652+0000 7f1c1aa00700 10 client.15225 handle_command_reply: tid=0 2024-05-08T05:56:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.652+0000 7f1c45fc5b80 1 client.15225 shutdown 2024-05-08T05:56:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.652+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/661321683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 msgr2=0x561c2a605e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:57.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.652+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/661321683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a605e20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1c28003fd0 tx=0x7f1c280038b0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.652+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/661321683 shutdown_connections 2024-05-08T05:56:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.652+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/661321683 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561c2a613830 0x561c2a615c20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.652+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/661321683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fe9d0 0x561c2a603700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.652+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/661321683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a605e20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.652+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/661321683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fa980 0x561c2a6031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.652+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/661321683 >> 172.21.15.112:0/661321683 conn(0x561c2a528b60 msgr2=0x561c2a5ef2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.652+0000 7f1c1aa00700 0 client.15225 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T05:56:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.653+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/661321683 shutdown_connections 2024-05-08T05:56:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.653+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/661321683 wait complete. 2024-05-08T05:56:57.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.653+0000 7f1c45fc5b80 20 client.15225 trim_cache size 0 max 16384 2024-05-08T05:56:57.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.669+0000 7f1c45fc5b80 1 Processor -- start 2024-05-08T05:56:57.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.669+0000 7f1c45fc5b80 1 -- start start 2024-05-08T05:56:57.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.669+0000 7f1c45fc5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fa060 0x561c2a600d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.669+0000 7f1c45fc5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fa980 0x561c2a618c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.669+0000 7f1c45fc5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fe9d0 0x561c2a619180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.669+0000 7f1c45fc5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c341b6b30 con 0x561c2a5fe9d0 2024-05-08T05:56:57.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.669+0000 7f1c45fc5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1c341b43c0 con 0x561c2a5fa980 2024-05-08T05:56:57.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.669+0000 7f1c45fc5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1c341b2210 con 0x561c2a5fa060 2024-05-08T05:56:57.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.670+0000 7f1c33e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fa980 0x561c2a618c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.670+0000 7f1c33e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fa980 0x561c2a618c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:34308/0 (socket says 172.21.15.112:34308) 2024-05-08T05:56:57.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.670+0000 7f1c33e00700 1 -- 172.21.15.112:0/1293153039 learned_addr learned my addr 172.21.15.112:0/1293153039 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:57.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.670+0000 7f1c38c00700 1 --2- 172.21.15.112:0/1293153039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fa060 0x561c2a600d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.670+0000 7f1c39600700 1 --2- 172.21.15.112:0/1293153039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fe9d0 0x561c2a619180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.670+0000 7f1c33e00700 1 -- 172.21.15.112:0/1293153039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fa060 msgr2=0x561c2a600d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.670+0000 7f1c33e00700 1 --2- 172.21.15.112:0/1293153039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fa060 0x561c2a600d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.670+0000 7f1c33e00700 1 -- 172.21.15.112:0/1293153039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fe9d0 msgr2=0x561c2a619180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.670+0000 7f1c33e00700 1 --2- 172.21.15.112:0/1293153039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fe9d0 0x561c2a619180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.670+0000 7f1c33e00700 1 -- 172.21.15.112:0/1293153039 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c2a606bc0 con 0x561c2a5fa980 2024-05-08T05:56:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.670+0000 7f1c33e00700 1 --2- 172.21.15.112:0/1293153039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fa980 0x561c2a618c40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x561c2a606df0 tx=0x7f1c20009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef30375d5048a103 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.671+0000 7f1c25600700 1 -- 172.21.15.112:0/1293153039 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20004050 con 0x561c2a5fa980 2024-05-08T05:56:57.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.671+0000 7f1c25600700 1 -- 172.21.15.112:0/1293153039 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c20024070 con 0x561c2a5fa980 2024-05-08T05:56:57.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.671+0000 7f1c25600700 1 -- 172.21.15.112:0/1293153039 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2001e420 con 0x561c2a5fa980 2024-05-08T05:56:57.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.672+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/1293153039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fa980 msgr2=0x561c2a618c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:57.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.672+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/1293153039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fa980 0x561c2a618c40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x561c2a606df0 tx=0x7f1c20009260 comp rx=0 tx=0).stop 2024-05-08T05:56:57.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.672+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/1293153039 shutdown_connections 2024-05-08T05:56:57.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.672+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/1293153039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fe9d0 0x561c2a619180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.672+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/1293153039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fa060 0x561c2a600d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.672+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/1293153039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a5fa980 0x561c2a618c40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.672+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/1293153039 >> 172.21.15.112:0/1293153039 conn(0x561c2a528b60 msgr2=0x561c2a600620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:57.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.672+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/1293153039 shutdown_connections 2024-05-08T05:56:57.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.672+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/1293153039 wait complete. 2024-05-08T05:56:57.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.672+0000 7f1c45fc5b80 1 -- start start 2024-05-08T05:56:57.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.672+0000 7f1c45fc5b80 1 Processor -- start 2024-05-08T05:56:57.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.673+0000 7f1c45fc5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a616ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.673+0000 7f1c45fc5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fe9d0 0x561c2a6170e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.673+0000 7f1c45fc5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a617780 0x561c2a67cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.673+0000 7f1c45fc5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c341b2880 con 0x561c2a5fa060 2024-05-08T05:56:57.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.673+0000 7f1c45fc5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1c341b3cf0 con 0x561c2a617780 2024-05-08T05:56:57.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.673+0000 7f1c45fc5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1c341b2f50 con 0x561c2a5fe9d0 2024-05-08T05:56:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.673+0000 7f1c45fc5b80 10 client.0 _ll_register_callbacks cb 0x561c2a4f3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.673+0000 7f1c33e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fe9d0 0x561c2a6170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.673+0000 7f1c33e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fe9d0 0x561c2a6170e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:34024/0 (socket says 172.21.15.112:34024) 2024-05-08T05:56:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.673+0000 7f1c33e00700 1 -- 172.21.15.112:0/841478835 learned_addr learned my addr 172.21.15.112:0/841478835 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:56:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.673+0000 7f1c38c00700 1 --2- 172.21.15.112:0/841478835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a616ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.673+0000 7f1c39600700 1 --2- 172.21.15.112:0/841478835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a617780 0x561c2a67cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.674+0000 7f1c33e00700 1 -- 172.21.15.112:0/841478835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a617780 msgr2=0x561c2a67cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.674+0000 7f1c33e00700 1 --2- 172.21.15.112:0/841478835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a617780 0x561c2a67cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.674+0000 7f1c33e00700 1 -- 172.21.15.112:0/841478835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 msgr2=0x561c2a616ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.674+0000 7f1c33e00700 1 --2- 172.21.15.112:0/841478835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a616ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.674+0000 7f1c33e00700 1 -- 172.21.15.112:0/841478835 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c2a5f13f0 con 0x561c2a5fe9d0 2024-05-08T05:56:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.674+0000 7f1c38c00700 1 --2- 172.21.15.112:0/841478835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a616ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.674+0000 7f1c33e00700 1 --2- 172.21.15.112:0/841478835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fe9d0 0x561c2a6170e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x561c2a605930 tx=0x7f1c20009c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33b8d35f8797b075 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:57.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.674+0000 7f1c1aa00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:56:57.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.674+0000 7f1c1aa00700 1 -- 172.21.15.112:0/841478835 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20003680 con 0x561c2a5fe9d0 2024-05-08T05:56:57.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.674+0000 7f1c1aa00700 1 -- 172.21.15.112:0/841478835 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f1c20003820 con 0x561c2a5fe9d0 2024-05-08T05:56:57.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.674+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/841478835 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c20024070 con 0x561c2a5fe9d0 2024-05-08T05:56:57.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.675+0000 7f1c1aa00700 1 -- 172.21.15.112:0/841478835 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c20039050 con 0x561c2a5fe9d0 2024-05-08T05:56:57.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.675+0000 7f1c45fc5b80 10 client.25378 fetch_fsmap learned FSMap version 25 2024-05-08T05:56:57.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.675+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/841478835 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c2a67e6f0 con 0x561c2a5fe9d0 2024-05-08T05:56:57.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.675+0000 7f1c1aa00700 1 -- 172.21.15.112:0/841478835 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c20024070 con 0x561c2a5fe9d0 2024-05-08T05:56:57.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.675+0000 7f1c1aa00700 1 -- 172.21.15.112:0/841478835 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f1c2001e800 con 0x561c2a5fe9d0 2024-05-08T05:56:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.675+0000 7f1c45fc5b80 10 client.25378 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:56:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.675+0000 7f1c45fc5b80 10 client.25378 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T05:56:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.675+0000 7f1c45fc5b80 20 client.25378 populate_metadata read hostname 'smithi112' 2024-05-08T05:56:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.675+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/841478835 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561c2a67ede0 0x561c2a6811d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.675+0000 7f1c45fc5b80 4 client.25378 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:56:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.675+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/841478835 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c2a55a400 con 0x561c2a67ede0 2024-05-08T05:56:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.676+0000 7f1c38c00700 1 --2- 172.21.15.112:0/841478835 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561c2a67ede0 0x561c2a6811d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.676+0000 7f1c38c00700 1 --2- 172.21.15.112:0/841478835 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561c2a67ede0 0x561c2a6811d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=248d7a1e7663911c server_cookie=822949cb40b486bc in_seq=0 out_seq=0 2024-05-08T05:56:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.676+0000 7f1c1aa00700 10 client.25378 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T05:56:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.676+0000 7f1c1aa00700 1 -- 172.21.15.112:0/841478835 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c2a55a400 con 0x561c2a67ede0 2024-05-08T05:56:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.676+0000 7f1c1aa00700 10 client.25378 handle_command_reply: tid=0 2024-05-08T05:56:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.676+0000 7f1c45fc5b80 1 client.25378 shutdown 2024-05-08T05:56:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.677+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/841478835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fe9d0 msgr2=0x561c2a6170e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:57.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.677+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/841478835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fe9d0 0x561c2a6170e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x561c2a605930 tx=0x7f1c20009c40 comp rx=0 tx=0).stop 2024-05-08T05:56:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.677+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/841478835 shutdown_connections 2024-05-08T05:56:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.677+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/841478835 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561c2a67ede0 0x561c2a6811d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.677+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/841478835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c2a617780 0x561c2a67cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.677+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/841478835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c2a5fa060 0x561c2a616ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.677+0000 7f1c45fc5b80 1 --2- 172.21.15.112:0/841478835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561c2a5fe9d0 0x561c2a6170e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.677+0000 7f1c1aa00700 0 client.25378 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T05:56:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.677+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/841478835 >> 172.21.15.112:0/841478835 conn(0x561c2a528b60 msgr2=0x561c2a5eea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.677+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/841478835 shutdown_connections 2024-05-08T05:56:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.677+0000 7f1c45fc5b80 1 -- 172.21.15.112:0/841478835 wait complete. 2024-05-08T05:56:57.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.677+0000 7f1c45fc5b80 20 client.25378 trim_cache size 0 max 16384 2024-05-08T05:56:57.681 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:56:57.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.678+0000 7f1c3b600700 1 -- 172.21.15.112:0/2569451225 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1c1c0b2900 msgr2=0x7f1c1c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:57.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.678+0000 7f1c3b600700 1 --2- 172.21.15.112:0/2569451225 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1c1c0b2900 0x7f1c1c0b4dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00b6c0 tx=0x7f1c2c003bb0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.678+0000 7f1c3b600700 1 -- 172.21.15.112:0/2569451225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c34109ad0 msgr2=0x7f1c341a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:57.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.678+0000 7f1c3b600700 1 --2- 172.21.15.112:0/2569451225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c34109ad0 0x7f1c341a00f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800ec10 tx=0x7f1c2800ef20 comp rx=0 tx=0).stop 2024-05-08T05:56:57.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.678+0000 7f1c3b600700 1 -- 172.21.15.112:0/2569451225 shutdown_connections 2024-05-08T05:56:57.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.678+0000 7f1c3b600700 1 --2- 172.21.15.112:0/2569451225 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1c1c0b2900 0x7f1c1c0b4dc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.678+0000 7f1c3b600700 1 --2- 172.21.15.112:0/2569451225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c3410fb60 0x7f1c341a49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.678+0000 7f1c3b600700 1 --2- 172.21.15.112:0/2569451225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c34109ad0 0x7f1c341a00f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.678+0000 7f1c3b600700 1 --2- 172.21.15.112:0/2569451225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c3410a490 0x7f1c341a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.678+0000 7f1c3b600700 1 -- 172.21.15.112:0/2569451225 >> 172.21.15.112:0/2569451225 conn(0x7f1c34076c70 msgr2=0x7f1c340771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:57.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.678+0000 7f1c3b600700 1 -- 172.21.15.112:0/2569451225 shutdown_connections 2024-05-08T05:56:57.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:56:57.678+0000 7f1c3b600700 1 -- 172.21.15.112:0/2569451225 wait complete. 2024-05-08T05:56:58.620 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-08T05:56:59.059 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:59.294 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:56:59 smithi112 ceph-mon[30003]: pgmap v491: 97 pgs: 97 active+clean; 2.2 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 135 KiB/s wr, 6 op/s 2024-05-08T05:56:59.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:56:59 smithi162 ceph-mon[34800]: pgmap v491: 97 pgs: 97 active+clean; 2.2 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 135 KiB/s wr, 6 op/s 2024-05-08T05:56:59.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:56:59 smithi150 ceph-mon[34315]: pgmap v491: 97 pgs: 97 active+clean; 2.2 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 135 KiB/s wr, 6 op/s 2024-05-08T05:57:01.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:01 smithi112 ceph-mon[30003]: pgmap v492: 97 pgs: 97 active+clean; 15 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-05-08T05:57:01.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:01 smithi162 ceph-mon[34800]: pgmap v492: 97 pgs: 97 active+clean; 15 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-05-08T05:57:01.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:01 smithi150 ceph-mon[34315]: pgmap v492: 97 pgs: 97 active+clean; 15 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 12 op/s 2024-05-08T05:57:01.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.869+0000 7f93e6000700 1 -- 172.21.15.112:0/2301096542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93e010b660 msgr2=0x7f93e010ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:01.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.869+0000 7f93e6000700 1 --2- 172.21.15.112:0/2301096542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93e010b660 0x7f93e010ba40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009990 tx=0x7f93d0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.869+0000 7f93e6000700 1 -- 172.21.15.112:0/2301096542 shutdown_connections 2024-05-08T05:57:01.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.869+0000 7f93e6000700 1 --2- 172.21.15.112:0/2301096542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e0105f90 0x7f93e01124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.869+0000 7f93e6000700 1 --2- 172.21.15.112:0/2301096542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93e010b660 0x7f93e010ba40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.869+0000 7f93e6000700 1 --2- 172.21.15.112:0/2301096542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93e01055d0 0x7f93e0105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.869+0000 7f93e6000700 1 -- 172.21.15.112:0/2301096542 >> 172.21.15.112:0/2301096542 conn(0x7f93e01010c0 msgr2=0x7f93e01034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.869+0000 7f93e6000700 1 -- 172.21.15.112:0/2301096542 shutdown_connections 2024-05-08T05:57:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.869+0000 7f93e6000700 1 -- 172.21.15.112:0/2301096542 wait complete. 2024-05-08T05:57:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.870+0000 7f93e6000700 1 Processor -- start 2024-05-08T05:57:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.871+0000 7f93e6000700 1 -- start start 2024-05-08T05:57:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.871+0000 7f93e6000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93e01055d0 0x7f93e01a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.871+0000 7f93e6000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93e0105f90 0x7f93e01a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.871+0000 7f93e6000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e010b660 0x7f93e019ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.871+0000 7f93e6000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93e01157d0 con 0x7f93e0105f90 2024-05-08T05:57:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.871+0000 7f93e6000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93e0115650 con 0x7f93e010b660 2024-05-08T05:57:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.871+0000 7f93e6000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f93e0115950 con 0x7f93e01055d0 2024-05-08T05:57:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.871+0000 7f93d7e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93e0105f90 0x7f93e01a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.871+0000 7f93d7e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93e0105f90 0x7f93e01a1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55584/0 (socket says 172.21.15.112:55584) 2024-05-08T05:57:01.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.871+0000 7f93d7e00700 1 -- 172.21.15.112:0/3668177518 learned_addr learned my addr 172.21.15.112:0/3668177518 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:01.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.871+0000 7f93e4c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93e01055d0 0x7f93e01a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.871+0000 7f93e5600700 1 --2- 172.21.15.112:0/3668177518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e010b660 0x7f93e019ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.872+0000 7f93d7e00700 1 -- 172.21.15.112:0/3668177518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93e01055d0 msgr2=0x7f93e01a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:01.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.872+0000 7f93d7e00700 1 --2- 172.21.15.112:0/3668177518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93e01055d0 0x7f93e01a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.872+0000 7f93d7e00700 1 -- 172.21.15.112:0/3668177518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e010b660 msgr2=0x7f93e019ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:01.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.872+0000 7f93d7e00700 1 --2- 172.21.15.112:0/3668177518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e010b660 0x7f93e019ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.872+0000 7f93d7e00700 1 -- 172.21.15.112:0/3668177518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e019c1d0 con 0x7f93e0105f90 2024-05-08T05:57:01.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.872+0000 7f93e4c00700 1 --2- 172.21.15.112:0/3668177518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93e01055d0 0x7f93e01a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:57:01.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.872+0000 7f93d7e00700 1 --2- 172.21.15.112:0/3668177518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93e0105f90 0x7f93e01a1da0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00e9c0 tx=0x7f93cc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:01.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.873+0000 7f93de000700 1 -- 172.21.15.112:0/3668177518 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc00cbd0 con 0x7f93e0105f90 2024-05-08T05:57:01.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.873+0000 7f93de000700 1 -- 172.21.15.112:0/3668177518 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93cc00ee60 con 0x7f93e0105f90 2024-05-08T05:57:01.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.873+0000 7f93e6000700 1 -- 172.21.15.112:0/3668177518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e019c4c0 con 0x7f93e0105f90 2024-05-08T05:57:01.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.873+0000 7f93de000700 1 -- 172.21.15.112:0/3668177518 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc0055c0 con 0x7f93e0105f90 2024-05-08T05:57:01.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.873+0000 7f93e6000700 1 -- 172.21.15.112:0/3668177518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e01a8730 con 0x7f93e0105f90 2024-05-08T05:57:01.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.876+0000 7f93de000700 1 -- 172.21.15.112:0/3668177518 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f93cc004750 con 0x7f93e0105f90 2024-05-08T05:57:01.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.877+0000 7f93f1e92b80 1 Processor -- start 2024-05-08T05:57:01.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.878+0000 7f93f1e92b80 1 -- start start 2024-05-08T05:57:01.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.878+0000 7f93de000700 1 --2- 172.21.15.112:0/3668177518 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f93c40b2900 0x7f93c40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.878+0000 7f93f1e92b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e24da0 0x556d69e25180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.878+0000 7f93f1e92b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e256c0 0x556d69e1f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.878+0000 7f93f1e92b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e1fa50 0x556d69e1fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.878+0000 7f93de000700 1 -- 172.21.15.112:0/3668177518 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f93cc0e24a0 con 0x7f93e0105f90 2024-05-08T05:57:01.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.878+0000 7f93f1e92b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556d69d71a20 con 0x556d69e1fa50 2024-05-08T05:57:01.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.878+0000 7f93f1e92b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556d69d0f760 con 0x556d69e256c0 2024-05-08T05:57:01.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.878+0000 7f93f1e92b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x556d69d6d9e0 con 0x556d69e24da0 2024-05-08T05:57:01.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.878+0000 7f93e5600700 1 --2- 172.21.15.112:0/3668177518 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f93c40b2900 0x7f93c40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.878+0000 7f93e4c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e24da0 0x556d69e25180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.879+0000 7f93e4c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e24da0 0x556d69e25180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:34062/0 (socket says 172.21.15.112:34062) 2024-05-08T05:57:01.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.879+0000 7f93e4c00700 1 -- 172.21.15.112:0/3783456849 learned_addr learned my addr 172.21.15.112:0/3783456849 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:01.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.879+0000 7f93d7e00700 1 --2- 172.21.15.112:0/3783456849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e256c0 0x556d69e1f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.879+0000 7f93e5600700 1 --2- 172.21.15.112:0/3783456849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e1fa50 0x556d69e1fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.879+0000 7f93d7e00700 1 -- 172.21.15.112:0/3783456849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e24da0 msgr2=0x556d69e25180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:01.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.879+0000 7f93d7e00700 1 --2- 172.21.15.112:0/3783456849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e24da0 0x556d69e25180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.879+0000 7f93d7e00700 1 -- 172.21.15.112:0/3783456849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e1fa50 msgr2=0x556d69e1fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:01.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.879+0000 7f93d7e00700 1 --2- 172.21.15.112:0/3783456849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e1fa50 0x556d69e1fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.879+0000 7f93d7e00700 1 -- 172.21.15.112:0/3783456849 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d69cbe8d0 con 0x556d69e256c0 2024-05-08T05:57:01.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.879+0000 7f93e5600700 1 --2- 172.21.15.112:0/3783456849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e1fa50 0x556d69e1fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:57:01.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.879+0000 7f93d7e00700 1 --2- 172.21.15.112:0/3783456849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e256c0 0x556d69e1f430 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009860 tx=0x7f93cc0a91a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c44850dbbe70be2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:01.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.879+0000 7f93e5600700 1 --2- 172.21.15.112:0/3668177518 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f93c40b2900 0x7f93c40b4dc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x556d69e20410 tx=0x7f93d800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:01.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.879+0000 7f93d6000700 1 -- 172.21.15.112:0/3783456849 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc0c3070 con 0x556d69e256c0 2024-05-08T05:57:01.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.880+0000 7f93d6000700 1 -- 172.21.15.112:0/3783456849 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93cc0b21e0 con 0x556d69e256c0 2024-05-08T05:57:01.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.880+0000 7f93d6000700 1 -- 172.21.15.112:0/3783456849 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc0a9eb0 con 0x556d69e256c0 2024-05-08T05:57:01.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.880+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3783456849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e256c0 msgr2=0x556d69e1f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:01.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.880+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/3783456849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e256c0 0x556d69e1f430 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009860 tx=0x7f93cc0a91a0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.881+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3783456849 shutdown_connections 2024-05-08T05:57:01.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.881+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/3783456849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e1fa50 0x556d69e1fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.881+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/3783456849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e24da0 0x556d69e25180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.881+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/3783456849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e256c0 0x556d69e1f430 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.881+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3783456849 >> 172.21.15.112:0/3783456849 conn(0x556d69d976d0 msgr2=0x556d69e19ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:01.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.881+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3783456849 shutdown_connections 2024-05-08T05:57:01.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.881+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3783456849 wait complete. 2024-05-08T05:57:01.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.881+0000 7f93f1e92b80 1 -- start start 2024-05-08T05:57:01.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.881+0000 7f93f1e92b80 1 Processor -- start 2024-05-08T05:57:01.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.882+0000 7f93f1e92b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e1fa50 0x556d69e2b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.882+0000 7f93f1e92b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 0x556d69e28800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.882+0000 7f93f1e92b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e256c0 0x556d69e28d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.882+0000 7f93f1e92b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556d69e2c150 con 0x556d69e256c0 2024-05-08T05:57:01.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.882+0000 7f93f1e92b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556d69cd7b20 con 0x556d69e24da0 2024-05-08T05:57:01.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.882+0000 7f93f1e92b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x556d69e2c2d0 con 0x556d69e1fa50 2024-05-08T05:57:01.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.882+0000 7f93f1e92b80 10 client.0 _ll_register_callbacks cb 0x556d69d18c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:57:01.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.882+0000 7f93d7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 0x556d69e28800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.882+0000 7f93e5600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e256c0 0x556d69e28d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.882+0000 7f93d7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 0x556d69e28800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:34358/0 (socket says 172.21.15.112:34358) 2024-05-08T05:57:01.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.882+0000 7f93d7e00700 1 -- 172.21.15.112:0/1949449605 learned_addr learned my addr 172.21.15.112:0/1949449605 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:01.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.882+0000 7f93e4c00700 1 --2- 172.21.15.112:0/1949449605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e1fa50 0x556d69e2b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.883+0000 7f93d7e00700 1 -- 172.21.15.112:0/1949449605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e1fa50 msgr2=0x556d69e2b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:01.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.883+0000 7f93d7e00700 1 --2- 172.21.15.112:0/1949449605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e1fa50 0x556d69e2b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.883+0000 7f93d7e00700 1 -- 172.21.15.112:0/1949449605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e256c0 msgr2=0x556d69e28d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:01.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.883+0000 7f93d7e00700 1 --2- 172.21.15.112:0/1949449605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e256c0 0x556d69e28d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.883+0000 7f93d7e00700 1 -- 172.21.15.112:0/1949449605 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d69d37420 con 0x556d69e24da0 2024-05-08T05:57:01.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.883+0000 7f93e4c00700 1 --2- 172.21.15.112:0/1949449605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e1fa50 0x556d69e2b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:57:01.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.883+0000 7f93d7e00700 1 --2- 172.21.15.112:0/1949449605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 0x556d69e28800 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f93cc0a9730 tx=0x7f93cc0b2690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1979381664be15c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:01.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.883+0000 7f93caa00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:57:01.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.883+0000 7f93caa00700 1 -- 172.21.15.112:0/1949449605 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc0c3030 con 0x556d69e24da0 2024-05-08T05:57:01.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.883+0000 7f93f1e92b80 1 -- 172.21.15.112:0/1949449605 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93cc0b21e0 con 0x556d69e24da0 2024-05-08T05:57:01.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.883+0000 7f93caa00700 1 -- 172.21.15.112:0/1949449605 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f93cc0a6020 con 0x556d69e24da0 2024-05-08T05:57:01.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.884+0000 7f93caa00700 1 -- 172.21.15.112:0/1949449605 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f93cc0aa130 con 0x556d69e24da0 2024-05-08T05:57:01.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.884+0000 7f93f1e92b80 10 client.25361 fetch_fsmap learned FSMap version 25 2024-05-08T05:57:01.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.884+0000 7f93f1e92b80 1 -- 172.21.15.112:0/1949449605 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556d69e2c150 con 0x556d69e24da0 2024-05-08T05:57:01.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.884+0000 7f93caa00700 1 -- 172.21.15.112:0/1949449605 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93cc0b21e0 con 0x556d69e24da0 2024-05-08T05:57:01.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.884+0000 7f93caa00700 1 -- 172.21.15.112:0/1949449605 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f93cc0cace0 con 0x556d69e24da0 2024-05-08T05:57:01.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.884+0000 7f93f1e92b80 10 client.25361 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:57:01.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.884+0000 7f93f1e92b80 10 client.25361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:57:01.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.884+0000 7f93f1e92b80 20 client.25361 populate_metadata read hostname 'smithi112' 2024-05-08T05:57:01.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.884+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/1949449605 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556d69e38dc0 0x556d69e3b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.884+0000 7f93f1e92b80 4 client.25361 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:57:01.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.884+0000 7f93f1e92b80 1 -- 172.21.15.112:0/1949449605 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d69dec140 con 0x556d69e38dc0 2024-05-08T05:57:01.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.885+0000 7f93e4c00700 1 --2- 172.21.15.112:0/1949449605 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556d69e38dc0 0x556d69e3b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.885+0000 7f93e4c00700 1 --2- 172.21.15.112:0/1949449605 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556d69e38dc0 0x556d69e3b1b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f4d80f11587128 server_cookie=daf900b5375a8611 in_seq=0 out_seq=0 2024-05-08T05:57:01.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.885+0000 7f93caa00700 10 client.25361 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:01.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.886+0000 7f93caa00700 1 -- 172.21.15.112:0/1949449605 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556d69dec140 con 0x556d69e38dc0 2024-05-08T05:57:01.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.886+0000 7f93caa00700 10 client.25361 handle_command_reply: tid=0 2024-05-08T05:57:01.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.886+0000 7f93f1e92b80 1 client.25361 shutdown 2024-05-08T05:57:01.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.886+0000 7f93f1e92b80 1 -- 172.21.15.112:0/1949449605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 msgr2=0x556d69e28800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:01.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.886+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/1949449605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 0x556d69e28800 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f93cc0a9730 tx=0x7f93cc0b2690 comp rx=0 tx=0).stop 2024-05-08T05:57:01.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.886+0000 7f93f1e92b80 1 -- 172.21.15.112:0/1949449605 shutdown_connections 2024-05-08T05:57:01.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.886+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/1949449605 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556d69e38dc0 0x556d69e3b1b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.886+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/1949449605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e256c0 0x556d69e28d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.886+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/1949449605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e1fa50 0x556d69e2b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.886+0000 7f93caa00700 0 client.25361 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:01.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.886+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/1949449605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 0x556d69e28800 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.886+0000 7f93f1e92b80 1 -- 172.21.15.112:0/1949449605 >> 172.21.15.112:0/1949449605 conn(0x556d69d976d0 msgr2=0x556d69e14750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:01.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.887+0000 7f93f1e92b80 1 -- 172.21.15.112:0/1949449605 shutdown_connections 2024-05-08T05:57:01.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.887+0000 7f93f1e92b80 1 -- 172.21.15.112:0/1949449605 wait complete. 2024-05-08T05:57:01.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.887+0000 7f93f1e92b80 20 client.25361 trim_cache size 0 max 16384 2024-05-08T05:57:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.903+0000 7f93f1e92b80 1 Processor -- start 2024-05-08T05:57:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.903+0000 7f93f1e92b80 1 -- start start 2024-05-08T05:57:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.903+0000 7f93f1e92b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e1fa50 0x556d69e1f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.903+0000 7f93f1e92b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 0x556d69e351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.903+0000 7f93f1e92b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e256c0 0x556d69e27980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.903+0000 7f93f1e92b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93e01b2450 con 0x556d69e1fa50 2024-05-08T05:57:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.903+0000 7f93f1e92b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93e01afce0 con 0x556d69e24da0 2024-05-08T05:57:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.903+0000 7f93f1e92b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f93e01adb30 con 0x556d69e256c0 2024-05-08T05:57:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.904+0000 7f93e4c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e1fa50 0x556d69e1f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.904+0000 7f93e4c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e1fa50 0x556d69e1f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55618/0 (socket says 172.21.15.112:55618) 2024-05-08T05:57:01.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.904+0000 7f93e4c00700 1 -- 172.21.15.112:0/3585556595 learned_addr learned my addr 172.21.15.112:0/3585556595 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.904+0000 7f93d7e00700 1 --2- 172.21.15.112:0/3585556595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 0x556d69e351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.904+0000 7f93e5600700 1 --2- 172.21.15.112:0/3585556595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e256c0 0x556d69e27980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.904+0000 7f93e4c00700 1 -- 172.21.15.112:0/3585556595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e256c0 msgr2=0x556d69e27980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.904+0000 7f93e4c00700 1 --2- 172.21.15.112:0/3585556595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e256c0 0x556d69e27980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.904+0000 7f93e4c00700 1 -- 172.21.15.112:0/3585556595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 msgr2=0x556d69e351e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.904+0000 7f93e4c00700 1 --2- 172.21.15.112:0/3585556595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 0x556d69e351e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.904+0000 7f93e4c00700 1 -- 172.21.15.112:0/3585556595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d69e20080 con 0x556d69e1fa50 2024-05-08T05:57:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.904+0000 7f93e5600700 1 --2- 172.21.15.112:0/3585556595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e256c0 0x556d69e27980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:57:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.904+0000 7f93e4c00700 1 --2- 172.21.15.112:0/3585556595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e1fa50 0x556d69e1f430 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f93d0006010 tx=0x7f93d001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259eca22a156bf5c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.905+0000 7f93d5600700 1 -- 172.21.15.112:0/3585556595 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d00056a0 con 0x556d69e1fa50 2024-05-08T05:57:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.905+0000 7f93d5600700 1 -- 172.21.15.112:0/3585556595 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d002f070 con 0x556d69e1fa50 2024-05-08T05:57:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.905+0000 7f93d5600700 1 -- 172.21.15.112:0/3585556595 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0029610 con 0x556d69e1fa50 2024-05-08T05:57:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.905+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3585556595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e1fa50 msgr2=0x556d69e1f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.905+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/3585556595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e1fa50 0x556d69e1f430 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f93d0006010 tx=0x7f93d001a040 comp rx=0 tx=0).stop 2024-05-08T05:57:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.905+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3585556595 shutdown_connections 2024-05-08T05:57:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.905+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/3585556595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e256c0 0x556d69e27980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.905+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/3585556595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e1fa50 0x556d69e1f430 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.905+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/3585556595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 0x556d69e351e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.905+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3585556595 >> 172.21.15.112:0/3585556595 conn(0x556d69d976d0 msgr2=0x556d69e29eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:01.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.905+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3585556595 shutdown_connections 2024-05-08T05:57:01.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.905+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3585556595 wait complete. 2024-05-08T05:57:01.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.906+0000 7f93f1e92b80 1 -- start start 2024-05-08T05:57:01.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.906+0000 7f93f1e92b80 1 Processor -- start 2024-05-08T05:57:01.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.907+0000 7f93f1e92b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e1fa50 0x556d69e318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.907+0000 7f93f1e92b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 0x556d69e2cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.907+0000 7f93f1e92b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e256c0 0x556d69e2d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.907+0000 7f93f1e92b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93e01ae1a0 con 0x556d69e256c0 2024-05-08T05:57:01.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.907+0000 7f93f1e92b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93e01af610 con 0x556d69e24da0 2024-05-08T05:57:01.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.907+0000 7f93f1e92b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f93e01ae870 con 0x556d69e1fa50 2024-05-08T05:57:01.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.907+0000 7f93f1e92b80 10 client.0 _ll_register_callbacks cb 0x556d69d18c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:57:01.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.907+0000 7f93e5600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e256c0 0x556d69e2d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.907+0000 7f93e5600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e256c0 0x556d69e2d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55630/0 (socket says 172.21.15.112:55630) 2024-05-08T05:57:01.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.907+0000 7f93e5600700 1 -- 172.21.15.112:0/3607491932 learned_addr learned my addr 172.21.15.112:0/3607491932 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:01.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.907+0000 7f93e4c00700 1 --2- 172.21.15.112:0/3607491932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e1fa50 0x556d69e318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.907+0000 7f93d7e00700 1 --2- 172.21.15.112:0/3607491932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 0x556d69e2cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.908+0000 7f93e4c00700 1 -- 172.21.15.112:0/3607491932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 msgr2=0x556d69e2cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:01.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.908+0000 7f93e4c00700 1 --2- 172.21.15.112:0/3607491932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 0x556d69e2cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.908+0000 7f93e4c00700 1 -- 172.21.15.112:0/3607491932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e256c0 msgr2=0x556d69e2d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:01.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.908+0000 7f93e4c00700 1 --2- 172.21.15.112:0/3607491932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e256c0 0x556d69e2d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.908+0000 7f93e4c00700 1 -- 172.21.15.112:0/3607491932 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d69e2d7a0 con 0x556d69e1fa50 2024-05-08T05:57:01.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.908+0000 7f93d7e00700 1 --2- 172.21.15.112:0/3607491932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 0x556d69e2cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:57:01.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.908+0000 7f93e5600700 1 --2- 172.21.15.112:0/3607491932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e256c0 0x556d69e2d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:57:01.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.908+0000 7f93e4c00700 1 --2- 172.21.15.112:0/3607491932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e1fa50 0x556d69e318a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f93d000b2e0 tx=0x7f93d00094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4341e6b02a1a0a76 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:01.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.908+0000 7f93caa00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:57:01.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.908+0000 7f93caa00700 1 -- 172.21.15.112:0/3607491932 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0003880 con 0x556d69e1fa50 2024-05-08T05:57:01.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.908+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3607491932 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93d002f070 con 0x556d69e1fa50 2024-05-08T05:57:01.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.908+0000 7f93caa00700 1 -- 172.21.15.112:0/3607491932 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f93d0003a20 con 0x556d69e1fa50 2024-05-08T05:57:01.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.909+0000 7f93caa00700 1 -- 172.21.15.112:0/3607491932 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f93d0003c40 con 0x556d69e1fa50 2024-05-08T05:57:01.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.909+0000 7f93f1e92b80 10 client.25393 fetch_fsmap learned FSMap version 25 2024-05-08T05:57:01.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.909+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3607491932 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556d69ea3100 con 0x556d69e1fa50 2024-05-08T05:57:01.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.909+0000 7f93caa00700 1 -- 172.21.15.112:0/3607491932 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d002f070 con 0x556d69e1fa50 2024-05-08T05:57:01.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.909+0000 7f93caa00700 1 -- 172.21.15.112:0/3607491932 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f93d000f460 con 0x556d69e1fa50 2024-05-08T05:57:01.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.909+0000 7f93f1e92b80 10 client.25393 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T05:57:01.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.909+0000 7f93f1e92b80 10 client.25393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:57:01.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.909+0000 7f93f1e92b80 20 client.25393 populate_metadata read hostname 'smithi112' 2024-05-08T05:57:01.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.909+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/3607491932 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556d69ea3750 0x556d69ea5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:01.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.909+0000 7f93f1e92b80 4 client.25393 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:57:01.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.909+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3607491932 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d69d7f400 con 0x556d69ea3750 2024-05-08T05:57:01.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.910+0000 7f93d7e00700 1 --2- 172.21.15.112:0/3607491932 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556d69ea3750 0x556d69ea5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:01.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.910+0000 7f93d7e00700 1 --2- 172.21.15.112:0/3607491932 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556d69ea3750 0x556d69ea5b40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=811c7c58e5df8b0f server_cookie=c455644df306426d in_seq=0 out_seq=0 2024-05-08T05:57:01.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.910+0000 7f93caa00700 10 client.25393 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:01.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.910+0000 7f93caa00700 1 -- 172.21.15.112:0/3607491932 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d69d7f400 con 0x556d69ea3750 2024-05-08T05:57:01.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.910+0000 7f93caa00700 10 client.25393 handle_command_reply: tid=0 2024-05-08T05:57:01.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.910+0000 7f93f1e92b80 1 client.25393 shutdown 2024-05-08T05:57:01.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.911+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3607491932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e1fa50 msgr2=0x556d69e318a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:01.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.911+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/3607491932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e1fa50 0x556d69e318a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f93d000b2e0 tx=0x7f93d00094b0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.911+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3607491932 shutdown_connections 2024-05-08T05:57:01.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.911+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/3607491932 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556d69ea3750 0x556d69ea5b40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.911+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/3607491932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556d69e256c0 0x556d69e2d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.911+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/3607491932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d69e1fa50 0x556d69e318a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.911+0000 7f93caa00700 0 client.25393 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:01.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.911+0000 7f93f1e92b80 1 --2- 172.21.15.112:0/3607491932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d69e24da0 0x556d69e2cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.911+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3607491932 >> 172.21.15.112:0/3607491932 conn(0x556d69d976d0 msgr2=0x556d69e25aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:01.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.911+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3607491932 shutdown_connections 2024-05-08T05:57:01.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.911+0000 7f93f1e92b80 1 -- 172.21.15.112:0/3607491932 wait complete. 2024-05-08T05:57:01.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.911+0000 7f93f1e92b80 20 client.25393 trim_cache size 0 max 16384 2024-05-08T05:57:01.915 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:57:01.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.912+0000 7f93e6000700 1 -- 172.21.15.112:0/3668177518 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f93c40b2900 msgr2=0x7f93c40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:01.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.912+0000 7f93e6000700 1 --2- 172.21.15.112:0/3668177518 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f93c40b2900 0x7f93c40b4dc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x556d69e20410 tx=0x7f93d800b040 comp rx=0 tx=0).stop 2024-05-08T05:57:01.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.912+0000 7f93e6000700 1 -- 172.21.15.112:0/3668177518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93e0105f90 msgr2=0x7f93e01a1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:01.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.912+0000 7f93e6000700 1 --2- 172.21.15.112:0/3668177518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93e0105f90 0x7f93e01a1da0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00e9c0 tx=0x7f93cc00ecd0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.912+0000 7f93e6000700 1 -- 172.21.15.112:0/3668177518 shutdown_connections 2024-05-08T05:57:01.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.912+0000 7f93e6000700 1 --2- 172.21.15.112:0/3668177518 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f93c40b2900 0x7f93c40b4dc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.912+0000 7f93e6000700 1 --2- 172.21.15.112:0/3668177518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e010b660 0x7f93e019ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.912+0000 7f93e6000700 1 --2- 172.21.15.112:0/3668177518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93e01055d0 0x7f93e01a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.912+0000 7f93e6000700 1 --2- 172.21.15.112:0/3668177518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93e0105f90 0x7f93e01a1da0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:01.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.912+0000 7f93e6000700 1 -- 172.21.15.112:0/3668177518 >> 172.21.15.112:0/3668177518 conn(0x7f93e01010c0 msgr2=0x7f93e0102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:01.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.912+0000 7f93e6000700 1 -- 172.21.15.112:0/3668177518 shutdown_connections 2024-05-08T05:57:01.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:01.912+0000 7f93e6000700 1 -- 172.21.15.112:0/3668177518 wait complete. 2024-05-08T05:57:03.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:03 smithi112 ceph-mon[30003]: pgmap v493: 97 pgs: 97 active+clean; 15 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-05-08T05:57:03.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:03 smithi162 ceph-mon[34800]: pgmap v493: 97 pgs: 97 active+clean; 15 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-05-08T05:57:03.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:03 smithi150 ceph-mon[34315]: pgmap v493: 97 pgs: 97 active+clean; 15 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-05-08T05:57:03.937 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T05:57:03.938 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T05:57:04.465 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:57:05.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:05 smithi112 ceph-mon[30003]: pgmap v494: 97 pgs: 97 active+clean; 15 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 11 op/s 2024-05-08T05:57:05.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:05 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:57:05.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:05 smithi162 ceph-mon[34800]: pgmap v494: 97 pgs: 97 active+clean; 15 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 11 op/s 2024-05-08T05:57:05.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:05 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:57:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:05 smithi150 ceph-mon[34315]: pgmap v494: 97 pgs: 97 active+clean; 15 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 11 op/s 2024-05-08T05:57:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:05 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:57:07.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:07 smithi112 ceph-mon[30003]: pgmap v495: 97 pgs: 97 active+clean; 15 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-05-08T05:57:07.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:07 smithi162 ceph-mon[34800]: pgmap v495: 97 pgs: 97 active+clean; 15 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-05-08T05:57:07.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:07 smithi150 ceph-mon[34315]: pgmap v495: 97 pgs: 97 active+clean; 15 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-05-08T05:57:08.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.011+0000 7f2b6f800700 1 -- 172.21.15.112:0/2554428379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b68075ff0 msgr2=0x7f2b6810f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:08.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.011+0000 7f2b6f800700 1 --2- 172.21.15.112:0/2554428379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b68075ff0 0x7f2b6810f9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009990 tx=0x7f2b54009ca0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.012+0000 7f2b6f800700 1 -- 172.21.15.112:0/2554428379 shutdown_connections 2024-05-08T05:57:08.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.012+0000 7f2b6f800700 1 --2- 172.21.15.112:0/2554428379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b6810ff00 0x7f2b681122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.012+0000 7f2b6f800700 1 --2- 172.21.15.112:0/2554428379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b680756d0 0x7f2b68075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.012+0000 7f2b6f800700 1 --2- 172.21.15.112:0/2554428379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b68075ff0 0x7f2b6810f9c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.012+0000 7f2b6f800700 1 -- 172.21.15.112:0/2554428379 >> 172.21.15.112:0/2554428379 conn(0x7f2b6807a600 msgr2=0x7f2b6807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:08.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.013+0000 7f2b6f800700 1 -- 172.21.15.112:0/2554428379 shutdown_connections 2024-05-08T05:57:08.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.013+0000 7f2b6f800700 1 -- 172.21.15.112:0/2554428379 wait complete. 2024-05-08T05:57:08.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.014+0000 7f2b6f800700 1 Processor -- start 2024-05-08T05:57:08.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.015+0000 7f2b6f800700 1 -- start start 2024-05-08T05:57:08.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.015+0000 7f2b6f800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b680756d0 0x7f2b6819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.015+0000 7f2b6f800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b68075ff0 0x7f2b6819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.015+0000 7f2b6f800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b6810ff00 0x7f2b681a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.015+0000 7f2b6f800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b68115910 con 0x7f2b68075ff0 2024-05-08T05:57:08.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.015+0000 7f2b6f800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2b68115790 con 0x7f2b680756d0 2024-05-08T05:57:08.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.015+0000 7f2b6f800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2b68115a90 con 0x7f2b6810ff00 2024-05-08T05:57:08.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.015+0000 7f2b67e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b68075ff0 0x7f2b6819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.015+0000 7f2b6ce00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b680756d0 0x7f2b6819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.015+0000 7f2b6d800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b6810ff00 0x7f2b681a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.015+0000 7f2b67e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b68075ff0 0x7f2b6819c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59058/0 (socket says 172.21.15.112:59058) 2024-05-08T05:57:08.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.015+0000 7f2b67e00700 1 -- 172.21.15.112:0/1631995105 learned_addr learned my addr 172.21.15.112:0/1631995105 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:08.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.016+0000 7f2b6d800700 1 -- 172.21.15.112:0/1631995105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b680756d0 msgr2=0x7f2b6819bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:08.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.016+0000 7f2b6d800700 1 --2- 172.21.15.112:0/1631995105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b680756d0 0x7f2b6819bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.016+0000 7f2b6d800700 1 -- 172.21.15.112:0/1631995105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b68075ff0 msgr2=0x7f2b6819c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:08.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.016+0000 7f2b6d800700 1 --2- 172.21.15.112:0/1631995105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b68075ff0 0x7f2b6819c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.016+0000 7f2b6d800700 1 -- 172.21.15.112:0/1631995105 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b681a0d90 con 0x7f2b6810ff00 2024-05-08T05:57:08.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.016+0000 7f2b67e00700 1 --2- 172.21.15.112:0/1631995105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b68075ff0 0x7f2b6819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:57:08.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.016+0000 7f2b6d800700 1 --2- 172.21.15.112:0/1631995105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b6810ff00 0x7f2b681a0650 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000ef90 tx=0x7f2b6000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:08.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.017+0000 7f2b65600700 1 -- 172.21.15.112:0/1631995105 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b600092e0 con 0x7f2b6810ff00 2024-05-08T05:57:08.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.017+0000 7f2b6f800700 1 -- 172.21.15.112:0/1631995105 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b681a1020 con 0x7f2b6810ff00 2024-05-08T05:57:08.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.017+0000 7f2b6f800700 1 -- 172.21.15.112:0/1631995105 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b681a8900 con 0x7f2b6810ff00 2024-05-08T05:57:08.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.017+0000 7f2b65600700 1 -- 172.21.15.112:0/1631995105 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b60004bb0 con 0x7f2b6810ff00 2024-05-08T05:57:08.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.017+0000 7f2b65600700 1 -- 172.21.15.112:0/1631995105 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60004020 con 0x7f2b6810ff00 2024-05-08T05:57:08.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.023+0000 7f2b65600700 1 -- 172.21.15.112:0/1631995105 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2b6001b030 con 0x7f2b6810ff00 2024-05-08T05:57:08.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.023+0000 7f2b7a1f5b80 1 Processor -- start 2024-05-08T05:57:08.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.023+0000 7f2b7a1f5b80 1 -- start start 2024-05-08T05:57:08.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.024+0000 7f2b7a1f5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e80c40 0x55e3d4e81020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.024+0000 7f2b7a1f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e81560 0x55e3d4e7b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.024+0000 7f2b7a1f5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e7b970 0x55e3d4e7bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.024+0000 7f2b7a1f5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e3d4dcdaa0 con 0x55e3d4e7b970 2024-05-08T05:57:08.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.024+0000 7f2b7a1f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e3d4d6b7e0 con 0x55e3d4e81560 2024-05-08T05:57:08.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.024+0000 7f2b7a1f5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55e3d4dc9a60 con 0x55e3d4e80c40 2024-05-08T05:57:08.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.025+0000 7f2b67e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e7b970 0x55e3d4e7bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.025+0000 7f2b67e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e7b970 0x55e3d4e7bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59060/0 (socket says 172.21.15.112:59060) 2024-05-08T05:57:08.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.025+0000 7f2b67e00700 1 -- 172.21.15.112:0/3712296960 learned_addr learned my addr 172.21.15.112:0/3712296960 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.025+0000 7f2b6ce00700 1 --2- 172.21.15.112:0/3712296960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e81560 0x55e3d4e7b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.025+0000 7f2b67e00700 1 --2- 172.21.15.112:0/3712296960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e80c40 0x55e3d4e81020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.025+0000 7f2b65600700 1 --2- 172.21.15.112:0/1631995105 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2b500b2980 0x7f2b500b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.026+0000 7f2b6ce00700 1 -- 172.21.15.112:0/3712296960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e80c40 msgr2=0x55e3d4e81020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.026+0000 7f2b6ce00700 1 --2- 172.21.15.112:0/3712296960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e80c40 0x55e3d4e81020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.026+0000 7f2b6ce00700 1 -- 172.21.15.112:0/3712296960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e7b970 msgr2=0x55e3d4e7bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.026+0000 7f2b6ce00700 1 --2- 172.21.15.112:0/3712296960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e7b970 0x55e3d4e7bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.026+0000 7f2b6ce00700 1 -- 172.21.15.112:0/3712296960 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3d4d1a950 con 0x55e3d4e81560 2024-05-08T05:57:08.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.026+0000 7f2b6d800700 1 --2- 172.21.15.112:0/1631995105 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2b500b2980 0x7f2b500b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.026+0000 7f2b65600700 1 -- 172.21.15.112:0/1631995105 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f2b600e1bb0 con 0x7f2b6810ff00 2024-05-08T05:57:08.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.026+0000 7f2b6ce00700 1 --2- 172.21.15.112:0/3712296960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e81560 0x55e3d4e7b350 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009cb0 tx=0x7f2b5c011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5586b650a877795 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:08.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.027+0000 7f2b6d800700 1 --2- 172.21.15.112:0/1631995105 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2b500b2980 0x7f2b500b4e40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2b60017000 tx=0x7f2b600573b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:08.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.027+0000 7f2b5a000700 1 -- 172.21.15.112:0/3712296960 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c011d20 con 0x55e3d4e81560 2024-05-08T05:57:08.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.027+0000 7f2b5a000700 1 -- 172.21.15.112:0/3712296960 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5c004500 con 0x55e3d4e81560 2024-05-08T05:57:08.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.027+0000 7f2b5a000700 1 -- 172.21.15.112:0/3712296960 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c005280 con 0x55e3d4e81560 2024-05-08T05:57:08.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.028+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/3712296960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e81560 msgr2=0x55e3d4e7b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:08.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.028+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/3712296960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e81560 0x55e3d4e7b350 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009cb0 tx=0x7f2b5c011440 comp rx=0 tx=0).stop 2024-05-08T05:57:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.028+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/3712296960 shutdown_connections 2024-05-08T05:57:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.028+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/3712296960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e7b970 0x55e3d4e7bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.028+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/3712296960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e80c40 0x55e3d4e81020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.028+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/3712296960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e81560 0x55e3d4e7b350 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.028+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/3712296960 >> 172.21.15.112:0/3712296960 conn(0x55e3d4df3750 msgr2=0x55e3d4e75b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.028+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/3712296960 shutdown_connections 2024-05-08T05:57:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.028+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/3712296960 wait complete. 2024-05-08T05:57:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.029+0000 7f2b7a1f5b80 1 -- start start 2024-05-08T05:57:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.029+0000 7f2b7a1f5b80 1 Processor -- start 2024-05-08T05:57:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.030+0000 7f2b7a1f5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e7b970 0x55e3d4e87130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.030+0000 7f2b7a1f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e80c40 0x55e3d4e844d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.030+0000 7f2b7a1f5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e81560 0x55e3d4e84a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.030+0000 7f2b7a1f5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e3d4e87ff0 con 0x55e3d4e81560 2024-05-08T05:57:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.030+0000 7f2b7a1f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e3d4d33ba0 con 0x55e3d4e80c40 2024-05-08T05:57:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.030+0000 7f2b7a1f5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55e3d4e88170 con 0x55e3d4e7b970 2024-05-08T05:57:08.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.030+0000 7f2b7a1f5b80 10 client.0 _ll_register_callbacks cb 0x55e3d4d74cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:57:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.031+0000 7f2b67e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e7b970 0x55e3d4e87130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.031+0000 7f2b67e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e7b970 0x55e3d4e87130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:35910/0 (socket says 172.21.15.112:35910) 2024-05-08T05:57:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.031+0000 7f2b67e00700 1 -- 172.21.15.112:0/23215263 learned_addr learned my addr 172.21.15.112:0/23215263 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.031+0000 7f2b67e00700 1 --2- 172.21.15.112:0/23215263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e81560 0x55e3d4e84a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.031+0000 7f2b6ce00700 1 --2- 172.21.15.112:0/23215263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e80c40 0x55e3d4e844d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.031+0000 7f2b67e00700 1 -- 172.21.15.112:0/23215263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e80c40 msgr2=0x55e3d4e844d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.031+0000 7f2b67e00700 1 --2- 172.21.15.112:0/23215263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e80c40 0x55e3d4e844d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.032+0000 7f2b67e00700 1 -- 172.21.15.112:0/23215263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e81560 msgr2=0x55e3d4e84a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.032+0000 7f2b67e00700 1 --2- 172.21.15.112:0/23215263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e81560 0x55e3d4e84a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.032+0000 7f2b67e00700 1 -- 172.21.15.112:0/23215263 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3d4d934a0 con 0x55e3d4e7b970 2024-05-08T05:57:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.032+0000 7f2b67e00700 1 --2- 172.21.15.112:0/23215263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e7b970 0x55e3d4e87130 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2b54028f10 tx=0x7f2b5401a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57ed54885cdda9bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.032+0000 7f2b4ea00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:57:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.033+0000 7f2b4ea00700 1 -- 172.21.15.112:0/23215263 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54003680 con 0x55e3d4e7b970 2024-05-08T05:57:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.033+0000 7f2b4ea00700 1 -- 172.21.15.112:0/23215263 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f2b54003820 con 0x55e3d4e7b970 2024-05-08T05:57:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.033+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/23215263 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b5c004500 con 0x55e3d4e7b970 2024-05-08T05:57:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.033+0000 7f2b4ea00700 1 -- 172.21.15.112:0/23215263 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2b54003a20 con 0x55e3d4e7b970 2024-05-08T05:57:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.033+0000 7f2b7a1f5b80 10 client.25405 fetch_fsmap learned FSMap version 26 2024-05-08T05:57:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.033+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/23215263 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e3d4e92810 con 0x55e3d4e7b970 2024-05-08T05:57:08.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.034+0000 7f2b4ea00700 1 -- 172.21.15.112:0/23215263 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5c004500 con 0x55e3d4e7b970 2024-05-08T05:57:08.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.035+0000 7f2b4ea00700 1 -- 172.21.15.112:0/23215263 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2b5403e020 con 0x55e3d4e7b970 2024-05-08T05:57:08.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.035+0000 7f2b7a1f5b80 10 client.25405 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T05:57:08.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.035+0000 7f2b7a1f5b80 10 client.25405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:57:08.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.035+0000 7f2b7a1f5b80 20 client.25405 populate_metadata read hostname 'smithi112' 2024-05-08T05:57:08.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.035+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/23215263 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55e3d4e94ea0 0x55e3d4e972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.035+0000 7f2b7a1f5b80 4 client.25405 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:57:08.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.035+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/23215263 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3d4e481c0 con 0x55e3d4e94ea0 2024-05-08T05:57:08.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.035+0000 7f2b6ce00700 1 --2- 172.21.15.112:0/23215263 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55e3d4e94ea0 0x55e3d4e972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.036+0000 7f2b6ce00700 1 --2- 172.21.15.112:0/23215263 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55e3d4e94ea0 0x55e3d4e972f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=368b85f329e10e23 server_cookie=777894cbbd0129c3 in_seq=0 out_seq=0 2024-05-08T05:57:08.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.036+0000 7f2b4ea00700 10 client.25405 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.038+0000 7f2b4ea00700 1 -- 172.21.15.112:0/23215263 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e3d4e481c0 con 0x55e3d4e94ea0 2024-05-08T05:57:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.038+0000 7f2b4ea00700 10 client.25405 handle_command_reply: tid=0 2024-05-08T05:57:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.038+0000 7f2b7a1f5b80 1 client.25405 shutdown 2024-05-08T05:57:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.039+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/23215263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e7b970 msgr2=0x55e3d4e87130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.039+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/23215263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e7b970 0x55e3d4e87130 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2b54028f10 tx=0x7f2b5401a040 comp rx=0 tx=0).stop 2024-05-08T05:57:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.039+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/23215263 shutdown_connections 2024-05-08T05:57:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.039+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/23215263 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55e3d4e94ea0 0x55e3d4e972f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.039+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/23215263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e81560 0x55e3d4e84a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.039+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/23215263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e7b970 0x55e3d4e87130 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.039+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/23215263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e80c40 0x55e3d4e844d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.039+0000 7f2b4ea00700 0 client.25405 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.039+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/23215263 >> 172.21.15.112:0/23215263 conn(0x55e3d4df3750 msgr2=0x55e3d4e75210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.039+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/23215263 shutdown_connections 2024-05-08T05:57:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.039+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/23215263 wait complete. 2024-05-08T05:57:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.040+0000 7f2b7a1f5b80 20 client.25405 trim_cache size 0 max 16384 2024-05-08T05:57:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.067+0000 7f2b7a1f5b80 1 Processor -- start 2024-05-08T05:57:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.067+0000 7f2b7a1f5b80 1 -- start start 2024-05-08T05:57:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.068+0000 7f2b7a1f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e7b970 0x55e3d4e7b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.068+0000 7f2b7a1f5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e80c40 0x55e3d4e75990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.068+0000 7f2b7a1f5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e81560 0x55e3d4e84490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.068+0000 7f2b7a1f5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4800fce0 con 0x55e3d4e81560 2024-05-08T05:57:08.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.068+0000 7f2b7a1f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4800d570 con 0x55e3d4e7b970 2024-05-08T05:57:08.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.068+0000 7f2b7a1f5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4800b3c0 con 0x55e3d4e80c40 2024-05-08T05:57:08.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.068+0000 7f2b67e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e7b970 0x55e3d4e7b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.068+0000 7f2b67e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e7b970 0x55e3d4e7b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:46606/0 (socket says 172.21.15.112:46606) 2024-05-08T05:57:08.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.068+0000 7f2b67e00700 1 -- 172.21.15.112:0/979442063 learned_addr learned my addr 172.21.15.112:0/979442063 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:08.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.068+0000 7f2b67e00700 1 --2- 172.21.15.112:0/979442063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e81560 0x55e3d4e84490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.068+0000 7f2b6ce00700 1 --2- 172.21.15.112:0/979442063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e80c40 0x55e3d4e75990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.069+0000 7f2b67e00700 1 -- 172.21.15.112:0/979442063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e80c40 msgr2=0x55e3d4e75990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:08.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.069+0000 7f2b67e00700 1 --2- 172.21.15.112:0/979442063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e80c40 0x55e3d4e75990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.069+0000 7f2b67e00700 1 -- 172.21.15.112:0/979442063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e81560 msgr2=0x55e3d4e84490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:08.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.069+0000 7f2b67e00700 1 --2- 172.21.15.112:0/979442063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e81560 0x55e3d4e84490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.069+0000 7f2b67e00700 1 -- 172.21.15.112:0/979442063 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3d4e70b30 con 0x55e3d4e7b970 2024-05-08T05:57:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.069+0000 7f2b6ce00700 1 --2- 172.21.15.112:0/979442063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e80c40 0x55e3d4e75990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:57:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.069+0000 7f2b67e00700 1 --2- 172.21.15.112:0/979442063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e7b970 0x55e3d4e7b350 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400fbf0 tx=0x7f2b54009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1cf1e7842ad7bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.069+0000 7f2b59600700 1 -- 172.21.15.112:0/979442063 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54003680 con 0x55e3d4e7b970 2024-05-08T05:57:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.069+0000 7f2b59600700 1 -- 172.21.15.112:0/979442063 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b54003820 con 0x55e3d4e7b970 2024-05-08T05:57:08.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.069+0000 7f2b59600700 1 -- 172.21.15.112:0/979442063 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54036c20 con 0x55e3d4e7b970 2024-05-08T05:57:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.070+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/979442063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e7b970 msgr2=0x55e3d4e7b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.070+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/979442063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e7b970 0x55e3d4e7b350 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400fbf0 tx=0x7f2b54009ca0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.070+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/979442063 shutdown_connections 2024-05-08T05:57:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.070+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/979442063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e81560 0x55e3d4e84490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.070+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/979442063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e7b970 0x55e3d4e7b350 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.070+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/979442063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e80c40 0x55e3d4e75990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.070+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/979442063 >> 172.21.15.112:0/979442063 conn(0x55e3d4df3750 msgr2=0x55e3d4e83d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.071+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/979442063 shutdown_connections 2024-05-08T05:57:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.071+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/979442063 wait complete. 2024-05-08T05:57:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.071+0000 7f2b7a1f5b80 1 -- start start 2024-05-08T05:57:08.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.072+0000 7f2b7a1f5b80 1 Processor -- start 2024-05-08T05:57:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.072+0000 7f2b7a1f5b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e7b970 0x55e3d4e8a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.072+0000 7f2b7a1f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e80c40 0x55e3d4e8a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.072+0000 7f2b7a1f5b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e81560 0x55e3d4e8aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.072+0000 7f2b7a1f5b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4800ba30 con 0x55e3d4e7b970 2024-05-08T05:57:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.072+0000 7f2b7a1f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4800cea0 con 0x55e3d4e80c40 2024-05-08T05:57:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.072+0000 7f2b7a1f5b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4800c100 con 0x55e3d4e81560 2024-05-08T05:57:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.072+0000 7f2b7a1f5b80 10 client.0 _ll_register_callbacks cb 0x55e3d4d74cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:57:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.073+0000 7f2b6ce00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e80c40 0x55e3d4e8a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.073+0000 7f2b6ce00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e80c40 0x55e3d4e8a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:46614/0 (socket says 172.21.15.112:46614) 2024-05-08T05:57:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.073+0000 7f2b6ce00700 1 -- 172.21.15.112:0/2685220437 learned_addr learned my addr 172.21.15.112:0/2685220437 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.073+0000 7f2b67e00700 1 --2- 172.21.15.112:0/2685220437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e7b970 0x55e3d4e8a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.073+0000 7f2b67e00700 1 --2- 172.21.15.112:0/2685220437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e81560 0x55e3d4e8aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.073+0000 7f2b6ce00700 1 -- 172.21.15.112:0/2685220437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e81560 msgr2=0x55e3d4e8aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.073+0000 7f2b6ce00700 1 --2- 172.21.15.112:0/2685220437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e81560 0x55e3d4e8aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.073+0000 7f2b6ce00700 1 -- 172.21.15.112:0/2685220437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e7b970 msgr2=0x55e3d4e8a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:08.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.073+0000 7f2b6ce00700 1 --2- 172.21.15.112:0/2685220437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e7b970 0x55e3d4e8a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.073+0000 7f2b6ce00700 1 -- 172.21.15.112:0/2685220437 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3d4eff800 con 0x55e3d4e80c40 2024-05-08T05:57:08.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.073+0000 7f2b67e00700 1 --2- 172.21.15.112:0/2685220437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e7b970 0x55e3d4e8a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:57:08.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.074+0000 7f2b6ce00700 1 --2- 172.21.15.112:0/2685220437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e80c40 0x55e3d4e8a9a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c027fd0 tx=0x7f2b5c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ecebf7433bc566e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:08.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.074+0000 7f2b4ea00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:57:08.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.074+0000 7f2b4ea00700 1 -- 172.21.15.112:0/2685220437 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c004a70 con 0x55e3d4e80c40 2024-05-08T05:57:08.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.074+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/2685220437 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b54003820 con 0x55e3d4e80c40 2024-05-08T05:57:08.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.074+0000 7f2b4ea00700 1 -- 172.21.15.112:0/2685220437 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f2b5c004c10 con 0x55e3d4e80c40 2024-05-08T05:57:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.074+0000 7f2b4ea00700 1 -- 172.21.15.112:0/2685220437 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2b5c011b20 con 0x55e3d4e80c40 2024-05-08T05:57:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.074+0000 7f2b7a1f5b80 10 client.25382 fetch_fsmap learned FSMap version 26 2024-05-08T05:57:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.075+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/2685220437 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e3d4f00b00 con 0x55e3d4e80c40 2024-05-08T05:57:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.075+0000 7f2b4ea00700 1 -- 172.21.15.112:0/2685220437 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b54003820 con 0x55e3d4e80c40 2024-05-08T05:57:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.075+0000 7f2b4ea00700 1 -- 172.21.15.112:0/2685220437 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2b5c018070 con 0x55e3d4e80c40 2024-05-08T05:57:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.075+0000 7f2b7a1f5b80 10 client.25382 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T05:57:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.075+0000 7f2b7a1f5b80 10 client.25382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:57:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.076+0000 7f2b7a1f5b80 20 client.25382 populate_metadata read hostname 'smithi112' 2024-05-08T05:57:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.076+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/2685220437 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55e3d4f01060 0x55e3d4f03480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.076+0000 7f2b7a1f5b80 4 client.25382 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T05:57:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.076+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/2685220437 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e3d4ddb480 con 0x55e3d4f01060 2024-05-08T05:57:08.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.076+0000 7f2b67e00700 1 --2- 172.21.15.112:0/2685220437 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55e3d4f01060 0x55e3d4f03480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:08.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.076+0000 7f2b67e00700 1 --2- 172.21.15.112:0/2685220437 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55e3d4f01060 0x55e3d4f03480 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7e63c227983fd server_cookie=f314405820cb6ce9 in_seq=0 out_seq=0 2024-05-08T05:57:08.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.076+0000 7f2b4ea00700 10 client.25382 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.078+0000 7f2b4ea00700 1 -- 172.21.15.112:0/2685220437 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e3d4ddb480 con 0x55e3d4f01060 2024-05-08T05:57:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.078+0000 7f2b4ea00700 10 client.25382 handle_command_reply: tid=0 2024-05-08T05:57:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.078+0000 7f2b7a1f5b80 1 client.25382 shutdown 2024-05-08T05:57:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.078+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/2685220437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e80c40 msgr2=0x55e3d4e8a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.078+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/2685220437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e80c40 0x55e3d4e8a9a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c027fd0 tx=0x7f2b5c009780 comp rx=0 tx=0).stop 2024-05-08T05:57:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.078+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/2685220437 shutdown_connections 2024-05-08T05:57:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.078+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/2685220437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3d4e81560 0x55e3d4e8aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.078+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/2685220437 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55e3d4f01060 0x55e3d4f03480 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.078+0000 7f2b4ea00700 0 client.25382 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.078+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/2685220437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e3d4e7b970 0x55e3d4e8a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.078+0000 7f2b7a1f5b80 1 --2- 172.21.15.112:0/2685220437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d4e80c40 0x55e3d4e8a9a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.078+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/2685220437 >> 172.21.15.112:0/2685220437 conn(0x55e3d4df3750 msgr2=0x55e3d4e84610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:08.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.079+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/2685220437 shutdown_connections 2024-05-08T05:57:08.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.079+0000 7f2b7a1f5b80 1 -- 172.21.15.112:0/2685220437 wait complete. 2024-05-08T05:57:08.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.079+0000 7f2b7a1f5b80 20 client.25382 trim_cache size 0 max 16384 2024-05-08T05:57:08.083 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:57:08.083 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-08T05:57:08.083 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "5994005e-74db-4055-aad9-e5736dd5066b", 2024-05-08T05:57:08.083 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-08T05:57:08.083 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:57:08.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.080+0000 7f2b6f800700 1 -- 172.21.15.112:0/1631995105 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2b500b2980 msgr2=0x7f2b500b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:08.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.080+0000 7f2b6f800700 1 --2- 172.21.15.112:0/1631995105 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2b500b2980 0x7f2b500b4e40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2b60017000 tx=0x7f2b600573b0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.080+0000 7f2b6f800700 1 -- 172.21.15.112:0/1631995105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b6810ff00 msgr2=0x7f2b681a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:08.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.080+0000 7f2b6f800700 1 --2- 172.21.15.112:0/1631995105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b6810ff00 0x7f2b681a0650 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000ef90 tx=0x7f2b6000c3a0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.080+0000 7f2b6f800700 1 -- 172.21.15.112:0/1631995105 shutdown_connections 2024-05-08T05:57:08.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.080+0000 7f2b6f800700 1 --2- 172.21.15.112:0/1631995105 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2b500b2980 0x7f2b500b4e40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.080+0000 7f2b6f800700 1 --2- 172.21.15.112:0/1631995105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b6810ff00 0x7f2b681a0650 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.080+0000 7f2b6f800700 1 --2- 172.21.15.112:0/1631995105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b680756d0 0x7f2b6819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.080+0000 7f2b6f800700 1 --2- 172.21.15.112:0/1631995105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b68075ff0 0x7f2b6819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:08.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.080+0000 7f2b6f800700 1 -- 172.21.15.112:0/1631995105 >> 172.21.15.112:0/1631995105 conn(0x7f2b6807a600 msgr2=0x7f2b68101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:08.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.081+0000 7f2b6f800700 1 -- 172.21.15.112:0/1631995105 shutdown_connections 2024-05-08T05:57:08.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:08.081+0000 7f2b6f800700 1 -- 172.21.15.112:0/1631995105 wait complete. 2024-05-08T05:57:09.237 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T05:57:09.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:09 smithi162 ceph-mon[34800]: pgmap v496: 97 pgs: 97 active+clean; 15 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-05-08T05:57:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:09 smithi162 ceph-mon[34800]: scrub queued for path: / 2024-05-08T05:57:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:09 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:57:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:09 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T05:57:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:09 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T05:57:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:09 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T05:57:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:09 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:57:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:09 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:57:09.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:09 smithi112 ceph-mon[30003]: pgmap v496: 97 pgs: 97 active+clean; 15 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-05-08T05:57:09.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:09 smithi112 ceph-mon[30003]: scrub queued for path: / 2024-05-08T05:57:09.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:09 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:57:09.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:09 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T05:57:09.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:09 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T05:57:09.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:09 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T05:57:09.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:09 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:57:09.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:09 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:57:09.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:09 smithi150 ceph-mon[34315]: pgmap v496: 97 pgs: 97 active+clean; 15 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-05-08T05:57:09.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:09 smithi150 ceph-mon[34315]: scrub queued for path: / 2024-05-08T05:57:09.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:09 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:57:09.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:09 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T05:57:09.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:09 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T05:57:09.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:09 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T05:57:09.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:09 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:57:09.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:09 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:57:09.821 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:57:11.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:11 smithi112 ceph-mon[30003]: pgmap v497: 97 pgs: 97 active+clean; 20 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 51 op/s 2024-05-08T05:57:11.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:11 smithi162 ceph-mon[34800]: pgmap v497: 97 pgs: 97 active+clean; 20 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 51 op/s 2024-05-08T05:57:11.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:11 smithi150 ceph-mon[34315]: pgmap v497: 97 pgs: 97 active+clean; 20 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 51 op/s 2024-05-08T05:57:12.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.373+0000 7f7f54400700 1 -- 172.21.15.112:0/1106572792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f4c108850 msgr2=0x7f7f4c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:12.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.373+0000 7f7f54400700 1 --2- 172.21.15.112:0/1106572792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f4c108850 0x7f7f4c1103d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400b0d0 tx=0x7f7f4400b3e0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.374+0000 7f7f54400700 1 -- 172.21.15.112:0/1106572792 shutdown_connections 2024-05-08T05:57:12.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.374+0000 7f7f54400700 1 --2- 172.21.15.112:0/1106572792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f4c108850 0x7f7f4c1103d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.374+0000 7f7f54400700 1 --2- 172.21.15.112:0/1106572792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f4c103750 0x7f7f4c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.374+0000 7f7f54400700 1 --2- 172.21.15.112:0/1106572792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f4c104100 0x7f7f4c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.374+0000 7f7f54400700 1 -- 172.21.15.112:0/1106572792 >> 172.21.15.112:0/1106572792 conn(0x7f7f4c07a600 msgr2=0x7f7f4c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:12.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.374+0000 7f7f54400700 1 -- 172.21.15.112:0/1106572792 shutdown_connections 2024-05-08T05:57:12.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.374+0000 7f7f54400700 1 -- 172.21.15.112:0/1106572792 wait complete. 2024-05-08T05:57:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.375+0000 7f7f54400700 1 Processor -- start 2024-05-08T05:57:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.375+0000 7f7f54400700 1 -- start start 2024-05-08T05:57:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.376+0000 7f7f54400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f4c103750 0x7f7f4c1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.376+0000 7f7f54400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f4c104100 0x7f7f4c1a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.376+0000 7f7f54400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f4c108850 0x7f7f4c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.376+0000 7f7f54400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c113670 con 0x7f7f4c108850 2024-05-08T05:57:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.376+0000 7f7f54400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1134f0 con 0x7f7f4c103750 2024-05-08T05:57:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.376+0000 7f7f54400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1137f0 con 0x7f7f4c104100 2024-05-08T05:57:12.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.376+0000 7f7f52400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f4c108850 0x7f7f4c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.376+0000 7f7f52400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f4c108850 0x7f7f4c19bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59104/0 (socket says 172.21.15.112:59104) 2024-05-08T05:57:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.377+0000 7f7f52400700 1 -- 172.21.15.112:0/1506357509 learned_addr learned my addr 172.21.15.112:0/1506357509 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.377+0000 7f7f51000700 1 --2- 172.21.15.112:0/1506357509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f4c104100 0x7f7f4c1a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.377+0000 7f7f51a00700 1 --2- 172.21.15.112:0/1506357509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f4c103750 0x7f7f4c1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.377+0000 7f7f52400700 1 -- 172.21.15.112:0/1506357509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f4c104100 msgr2=0x7f7f4c1a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.377+0000 7f7f52400700 1 --2- 172.21.15.112:0/1506357509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f4c104100 0x7f7f4c1a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.377+0000 7f7f52400700 1 -- 172.21.15.112:0/1506357509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f4c103750 msgr2=0x7f7f4c1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.377+0000 7f7f52400700 1 --2- 172.21.15.112:0/1506357509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f4c103750 0x7f7f4c1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.377+0000 7f7f52400700 1 -- 172.21.15.112:0/1506357509 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c19c270 con 0x7f7f4c108850 2024-05-08T05:57:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.378+0000 7f7f52400700 1 --2- 172.21.15.112:0/1506357509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f4c108850 0x7f7f4c19bb10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7f440079a0 tx=0x7f7f440079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.378+0000 7f7f4a000700 1 -- 172.21.15.112:0/1506357509 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4401f070 con 0x7f7f4c108850 2024-05-08T05:57:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.378+0000 7f7f54400700 1 -- 172.21.15.112:0/1506357509 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c19c500 con 0x7f7f4c108850 2024-05-08T05:57:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.378+0000 7f7f4a000700 1 -- 172.21.15.112:0/1506357509 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f44003c00 con 0x7f7f4c108850 2024-05-08T05:57:12.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.378+0000 7f7f4a000700 1 -- 172.21.15.112:0/1506357509 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4401b430 con 0x7f7f4c108850 2024-05-08T05:57:12.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.379+0000 7f7f54400700 1 -- 172.21.15.112:0/1506357509 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c1a87d0 con 0x7f7f4c108850 2024-05-08T05:57:12.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.382+0000 7f7f4a000700 1 -- 172.21.15.112:0/1506357509 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7f44022070 con 0x7f7f4c108850 2024-05-08T05:57:12.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.384+0000 7f7f4a000700 1 --2- 172.21.15.112:0/1506357509 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7f340b2880 0x7f7f340b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.384+0000 7f7f4a000700 1 -- 172.21.15.112:0/1506357509 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f7f4400f020 con 0x7f7f4c108850 2024-05-08T05:57:12.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.384+0000 7f7f51a00700 1 --2- 172.21.15.112:0/1506357509 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7f340b2880 0x7f7f340b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.385+0000 7f7f51a00700 1 --2- 172.21.15.112:0/1506357509 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7f340b2880 0x7f7f340b4d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c07b080 tx=0x7f7f40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:12.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.386+0000 7f7f5ee04b80 1 Processor -- start 2024-05-08T05:57:12.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.386+0000 7f7f5ee04b80 1 -- start start 2024-05-08T05:57:12.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.386+0000 7f7f5ee04b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af64cbc0 0x5590af64cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.386+0000 7f7f5ee04b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64d4e0 0x5590af6472d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.386+0000 7f7f5ee04b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af6478f0 0x5590af647d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.386+0000 7f7f5ee04b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5590af599a20 con 0x5590af6478f0 2024-05-08T05:57:12.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.386+0000 7f7f5ee04b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5590af537760 con 0x5590af64d4e0 2024-05-08T05:57:12.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.386+0000 7f7f5ee04b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5590af5959e0 con 0x5590af64cbc0 2024-05-08T05:57:12.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.387+0000 7f7f51a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af64cbc0 0x5590af64cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.387+0000 7f7f51a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af64cbc0 0x5590af64cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:35954/0 (socket says 172.21.15.112:35954) 2024-05-08T05:57:12.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.387+0000 7f7f51a00700 1 -- 172.21.15.112:0/2570818207 learned_addr learned my addr 172.21.15.112:0/2570818207 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:12.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.387+0000 7f7f51000700 1 --2- 172.21.15.112:0/2570818207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64d4e0 0x5590af6472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.387+0000 7f7f52400700 1 --2- 172.21.15.112:0/2570818207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af6478f0 0x5590af647d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.387+0000 7f7f51000700 1 -- 172.21.15.112:0/2570818207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af64cbc0 msgr2=0x5590af64cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:12.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.387+0000 7f7f51000700 1 --2- 172.21.15.112:0/2570818207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af64cbc0 0x5590af64cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.387+0000 7f7f51000700 1 -- 172.21.15.112:0/2570818207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af6478f0 msgr2=0x5590af647d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:12.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.388+0000 7f7f51000700 1 --2- 172.21.15.112:0/2570818207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af6478f0 0x5590af647d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.388+0000 7f7f51000700 1 -- 172.21.15.112:0/2570818207 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590af4e68d0 con 0x5590af64d4e0 2024-05-08T05:57:12.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.388+0000 7f7f51a00700 1 --2- 172.21.15.112:0/2570818207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af64cbc0 0x5590af64cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:57:12.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.388+0000 7f7f52400700 1 --2- 172.21.15.112:0/2570818207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af6478f0 0x5590af647d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:57:12.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.388+0000 7f7f51000700 1 --2- 172.21.15.112:0/2570818207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64d4e0 0x5590af6472d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009c80 tx=0x7f7f38009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1693c8bca6711672 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:12.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.389+0000 7f7f3ea00700 1 -- 172.21.15.112:0/2570818207 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3801e070 con 0x5590af64d4e0 2024-05-08T05:57:12.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.389+0000 7f7f3ea00700 1 -- 172.21.15.112:0/2570818207 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f38011cb0 con 0x5590af64d4e0 2024-05-08T05:57:12.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.389+0000 7f7f3ea00700 1 -- 172.21.15.112:0/2570818207 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800e730 con 0x5590af64d4e0 2024-05-08T05:57:12.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.390+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2570818207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64d4e0 msgr2=0x5590af6472d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:12.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.390+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2570818207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64d4e0 0x5590af6472d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009c80 tx=0x7f7f38009f90 comp rx=0 tx=0).stop 2024-05-08T05:57:12.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.390+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2570818207 shutdown_connections 2024-05-08T05:57:12.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.390+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2570818207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af6478f0 0x5590af647d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.390+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2570818207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af64cbc0 0x5590af64cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.390+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2570818207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64d4e0 0x5590af6472d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.390+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2570818207 >> 172.21.15.112:0/2570818207 conn(0x5590af5bf6d0 msgr2=0x5590af641b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:12.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.390+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2570818207 shutdown_connections 2024-05-08T05:57:12.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.390+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2570818207 wait complete. 2024-05-08T05:57:12.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.391+0000 7f7f5ee04b80 1 -- start start 2024-05-08T05:57:12.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.391+0000 7f7f5ee04b80 1 Processor -- start 2024-05-08T05:57:12.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.392+0000 7f7f5ee04b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af6478f0 0x5590af6531d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.392+0000 7f7f5ee04b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 0x5590af650570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.392+0000 7f7f5ee04b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af64d4e0 0x5590af650ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.392+0000 7f7f5ee04b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5590af653f20 con 0x5590af64d4e0 2024-05-08T05:57:12.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.392+0000 7f7f5ee04b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5590af4ffb20 con 0x5590af64cbc0 2024-05-08T05:57:12.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.392+0000 7f7f5ee04b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5590af6540a0 con 0x5590af6478f0 2024-05-08T05:57:12.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.392+0000 7f7f5ee04b80 10 client.0 _ll_register_callbacks cb 0x5590af540c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:57:12.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.393+0000 7f7f51a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af6478f0 0x5590af6531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.393+0000 7f7f51a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af6478f0 0x5590af6531d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:35968/0 (socket says 172.21.15.112:35968) 2024-05-08T05:57:12.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.393+0000 7f7f51a00700 1 -- 172.21.15.112:0/2252794975 learned_addr learned my addr 172.21.15.112:0/2252794975 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:12.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.393+0000 7f7f51000700 1 --2- 172.21.15.112:0/2252794975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 0x5590af650570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.393+0000 7f7f52400700 1 --2- 172.21.15.112:0/2252794975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af64d4e0 0x5590af650ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.393+0000 7f7f51a00700 1 -- 172.21.15.112:0/2252794975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 msgr2=0x5590af650570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:12.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.393+0000 7f7f51a00700 1 --2- 172.21.15.112:0/2252794975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 0x5590af650570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.393+0000 7f7f51a00700 1 -- 172.21.15.112:0/2252794975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af64d4e0 msgr2=0x5590af650ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:12.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.393+0000 7f7f51a00700 1 --2- 172.21.15.112:0/2252794975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af64d4e0 0x5590af650ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.393+0000 7f7f51a00700 1 -- 172.21.15.112:0/2252794975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590af55f420 con 0x5590af6478f0 2024-05-08T05:57:12.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.394+0000 7f7f51a00700 1 --2- 172.21.15.112:0/2252794975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af6478f0 0x5590af6531d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009870 tx=0x7f7f400047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c818cdac88789ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:12.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.394+0000 7f7f33400700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:57:12.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.394+0000 7f7f33400700 1 -- 172.21.15.112:0/2252794975 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40027420 con 0x5590af6478f0 2024-05-08T05:57:12.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.394+0000 7f7f33400700 1 -- 172.21.15.112:0/2252794975 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f7f4001d410 con 0x5590af6478f0 2024-05-08T05:57:12.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.394+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2252794975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f38011cb0 con 0x5590af6478f0 2024-05-08T05:57:12.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.394+0000 7f7f33400700 1 -- 172.21.15.112:0/2252794975 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f4001d610 con 0x5590af6478f0 2024-05-08T05:57:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.395+0000 7f7f5ee04b80 10 client.25423 fetch_fsmap learned FSMap version 26 2024-05-08T05:57:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.395+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2252794975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5590af65e730 con 0x5590af6478f0 2024-05-08T05:57:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.395+0000 7f7f33400700 1 -- 172.21.15.112:0/2252794975 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f38011cb0 con 0x5590af6478f0 2024-05-08T05:57:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.395+0000 7f7f33400700 1 -- 172.21.15.112:0/2252794975 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7f4001c940 con 0x5590af6478f0 2024-05-08T05:57:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.396+0000 7f7f5ee04b80 10 client.25423 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T05:57:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.396+0000 7f7f5ee04b80 10 client.25423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:57:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.396+0000 7f7f5ee04b80 20 client.25423 populate_metadata read hostname 'smithi112' 2024-05-08T05:57:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.396+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2252794975 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5590af6611f0 0x5590af663630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.396+0000 7f7f5ee04b80 4 client.25423 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:57:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.396+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2252794975 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590af614140 con 0x5590af6611f0 2024-05-08T05:57:12.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.396+0000 7f7f51000700 1 --2- 172.21.15.112:0/2252794975 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5590af6611f0 0x5590af663630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.397+0000 7f7f51000700 1 --2- 172.21.15.112:0/2252794975 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5590af6611f0 0x5590af663630 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efc8bc728878d8b1 server_cookie=f1ebe0b5222f5d9b in_seq=0 out_seq=0 2024-05-08T05:57:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.397+0000 7f7f33400700 10 client.25423 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.398+0000 7f7f33400700 1 -- 172.21.15.112:0/2252794975 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5590af614140 con 0x5590af6611f0 2024-05-08T05:57:12.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.398+0000 7f7f33400700 10 client.25423 handle_command_reply: tid=0 2024-05-08T05:57:12.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.398+0000 7f7f5ee04b80 1 client.25423 shutdown 2024-05-08T05:57:12.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.399+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2252794975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af6478f0 msgr2=0x5590af6531d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:12.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.399+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2252794975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af6478f0 0x5590af6531d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009870 tx=0x7f7f400047c0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.399+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2252794975 shutdown_connections 2024-05-08T05:57:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.399+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2252794975 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5590af6611f0 0x5590af663630 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.399+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2252794975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af64d4e0 0x5590af650ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.399+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2252794975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af6478f0 0x5590af6531d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.399+0000 7f7f33400700 0 client.25423 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.399+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2252794975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 0x5590af650570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.399+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2252794975 >> 172.21.15.112:0/2252794975 conn(0x5590af5bf6d0 msgr2=0x5590af641190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.399+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2252794975 shutdown_connections 2024-05-08T05:57:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.399+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2252794975 wait complete. 2024-05-08T05:57:12.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.399+0000 7f7f5ee04b80 20 client.25423 trim_cache size 0 max 16384 2024-05-08T05:57:12.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.436+0000 7f7f5ee04b80 1 Processor -- start 2024-05-08T05:57:12.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.436+0000 7f7f5ee04b80 1 -- start start 2024-05-08T05:57:12.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.436+0000 7f7f5ee04b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af6478f0 0x5590af65cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.436+0000 7f7f5ee04b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 0x5590af650ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.436+0000 7f7f5ee04b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af64d4e0 0x5590af641910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.436+0000 7f7f5ee04b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1b2540 con 0x5590af6478f0 2024-05-08T05:57:12.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.436+0000 7f7f5ee04b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1afdd0 con 0x5590af64cbc0 2024-05-08T05:57:12.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.436+0000 7f7f5ee04b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1adc20 con 0x5590af64d4e0 2024-05-08T05:57:12.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.437+0000 7f7f52400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af64d4e0 0x5590af641910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.437+0000 7f7f51a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af6478f0 0x5590af65cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.437+0000 7f7f52400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af64d4e0 0x5590af641910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:35982/0 (socket says 172.21.15.112:35982) 2024-05-08T05:57:12.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.437+0000 7f7f52400700 1 -- 172.21.15.112:0/3462964018 learned_addr learned my addr 172.21.15.112:0/3462964018 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:12.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.437+0000 7f7f51000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 0x5590af650ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.437+0000 7f7f52400700 1 -- 172.21.15.112:0/3462964018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 msgr2=0x5590af650ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:12.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.437+0000 7f7f52400700 1 --2- 172.21.15.112:0/3462964018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 0x5590af650ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.437+0000 7f7f52400700 1 -- 172.21.15.112:0/3462964018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af6478f0 msgr2=0x5590af65cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:12.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.437+0000 7f7f52400700 1 --2- 172.21.15.112:0/3462964018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af6478f0 0x5590af65cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.437+0000 7f7f52400700 1 -- 172.21.15.112:0/3462964018 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590af653540 con 0x5590af64d4e0 2024-05-08T05:57:12.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.437+0000 7f7f51000700 1 --2- 172.21.15.112:0/3462964018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 0x5590af650ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:57:12.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.437+0000 7f7f51a00700 1 --2- 172.21.15.112:0/3462964018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af6478f0 0x5590af65cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:57:12.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.437+0000 7f7f52400700 1 --2- 172.21.15.112:0/3462964018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af64d4e0 0x5590af641910 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7f440e06d0 tx=0x7f7f44005620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a762772fc5767b1d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:12.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.438+0000 7f7f3e000700 1 -- 172.21.15.112:0/3462964018 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f440aa280 con 0x5590af64d4e0 2024-05-08T05:57:12.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.438+0000 7f7f3e000700 1 -- 172.21.15.112:0/3462964018 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f440ae630 con 0x5590af64d4e0 2024-05-08T05:57:12.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.438+0000 7f7f3e000700 1 -- 172.21.15.112:0/3462964018 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f440b2bc0 con 0x5590af64d4e0 2024-05-08T05:57:12.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.439+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/3462964018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af64d4e0 msgr2=0x5590af641910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:12.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.439+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/3462964018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af64d4e0 0x5590af641910 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7f440e06d0 tx=0x7f7f44005620 comp rx=0 tx=0).stop 2024-05-08T05:57:12.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.439+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/3462964018 shutdown_connections 2024-05-08T05:57:12.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.439+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/3462964018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af64d4e0 0x5590af641910 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.439+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/3462964018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af6478f0 0x5590af65cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.439+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/3462964018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 0x5590af650ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.439+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/3462964018 >> 172.21.15.112:0/3462964018 conn(0x5590af5bf6d0 msgr2=0x5590af64b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:12.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.439+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/3462964018 shutdown_connections 2024-05-08T05:57:12.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.439+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/3462964018 wait complete. 2024-05-08T05:57:12.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.440+0000 7f7f5ee04b80 1 -- start start 2024-05-08T05:57:12.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.440+0000 7f7f5ee04b80 1 Processor -- start 2024-05-08T05:57:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.441+0000 7f7f5ee04b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af6478f0 0x5590af668870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.441+0000 7f7f5ee04b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 0x5590af668db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.441+0000 7f7f5ee04b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af64d4e0 0x5590af6692f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.441+0000 7f7f5ee04b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ae290 con 0x5590af64d4e0 2024-05-08T05:57:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.441+0000 7f7f5ee04b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1af700 con 0x5590af64cbc0 2024-05-08T05:57:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.441+0000 7f7f5ee04b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ae960 con 0x5590af6478f0 2024-05-08T05:57:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.441+0000 7f7f5ee04b80 10 client.0 _ll_register_callbacks cb 0x5590af540c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:57:12.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.441+0000 7f7f51a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af6478f0 0x5590af668870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.441+0000 7f7f51a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af6478f0 0x5590af668870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:35984/0 (socket says 172.21.15.112:35984) 2024-05-08T05:57:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.441+0000 7f7f51a00700 1 -- 172.21.15.112:0/2751913514 learned_addr learned my addr 172.21.15.112:0/2751913514 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.441+0000 7f7f51000700 1 --2- 172.21.15.112:0/2751913514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 0x5590af668db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.441+0000 7f7f52400700 1 --2- 172.21.15.112:0/2751913514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af64d4e0 0x5590af6692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.442+0000 7f7f51000700 1 -- 172.21.15.112:0/2751913514 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af6478f0 msgr2=0x5590af668870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.442+0000 7f7f51000700 1 --2- 172.21.15.112:0/2751913514 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af6478f0 0x5590af668870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.442+0000 7f7f51000700 1 -- 172.21.15.112:0/2751913514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af64d4e0 msgr2=0x5590af6692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.442+0000 7f7f51000700 1 --2- 172.21.15.112:0/2751913514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af64d4e0 0x5590af6692f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.442+0000 7f7f51000700 1 -- 172.21.15.112:0/2751913514 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590af6cac90 con 0x5590af64cbc0 2024-05-08T05:57:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.442+0000 7f7f51000700 1 --2- 172.21.15.112:0/2751913514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 0x5590af668db0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7f38019eb0 tx=0x7f7f38005d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=607ef07fd9c46e94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.442+0000 7f7f33400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:57:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.443+0000 7f7f33400700 1 -- 172.21.15.112:0/2751913514 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3801e040 con 0x5590af64cbc0 2024-05-08T05:57:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.443+0000 7f7f33400700 1 -- 172.21.15.112:0/2751913514 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f7f38005240 con 0x5590af64cbc0 2024-05-08T05:57:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.443+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2751913514 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f440ae630 con 0x5590af64cbc0 2024-05-08T05:57:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.443+0000 7f7f33400700 1 -- 172.21.15.112:0/2751913514 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f38035050 con 0x5590af64cbc0 2024-05-08T05:57:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.443+0000 7f7f5ee04b80 10 client.25400 fetch_fsmap learned FSMap version 26 2024-05-08T05:57:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.443+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2751913514 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5590af6d4730 con 0x5590af64cbc0 2024-05-08T05:57:12.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.444+0000 7f7f33400700 1 -- 172.21.15.112:0/2751913514 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f3802a070 con 0x5590af64cbc0 2024-05-08T05:57:12.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.444+0000 7f7f33400700 1 -- 172.21.15.112:0/2751913514 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7f38003680 con 0x5590af64cbc0 2024-05-08T05:57:12.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.444+0000 7f7f5ee04b80 10 client.25400 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T05:57:12.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.444+0000 7f7f5ee04b80 10 client.25400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:57:12.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.444+0000 7f7f5ee04b80 20 client.25400 populate_metadata read hostname 'smithi112' 2024-05-08T05:57:12.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.444+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2751913514 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5590af6d4cb0 0x5590af6d70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:12.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.444+0000 7f7f5ee04b80 4 client.25400 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:57:12.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.444+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2751913514 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f7f440ae630 con 0x5590af6d4cb0 2024-05-08T05:57:12.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.445+0000 7f7f51a00700 1 --2- 172.21.15.112:0/2751913514 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5590af6d4cb0 0x5590af6d70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:12.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.446+0000 7f7f51a00700 1 --2- 172.21.15.112:0/2751913514 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5590af6d4cb0 0x5590af6d70d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73703309c88e9d8c server_cookie=580c5a8b45e8efbd in_seq=0 out_seq=0 2024-05-08T05:57:12.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.446+0000 7f7f33400700 10 client.25400 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:12.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.449+0000 7f7f33400700 1 -- 172.21.15.112:0/2751913514 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f7f440ae630 con 0x5590af6d4cb0 2024-05-08T05:57:12.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.449+0000 7f7f33400700 10 client.25400 handle_command_reply: tid=0 2024-05-08T05:57:12.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.449+0000 7f7f5ee04b80 1 client.25400 shutdown 2024-05-08T05:57:12.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.449+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2751913514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 msgr2=0x5590af668db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:12.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.449+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2751913514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 0x5590af668db0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7f38019eb0 tx=0x7f7f38005d70 comp rx=0 tx=0).stop 2024-05-08T05:57:12.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.449+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2751913514 shutdown_connections 2024-05-08T05:57:12.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.449+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2751913514 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5590af6d4cb0 0x5590af6d70d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.449+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2751913514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590af64d4e0 0x5590af6692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.449+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2751913514 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590af6478f0 0x5590af668870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.449+0000 7f7f5ee04b80 1 --2- 172.21.15.112:0/2751913514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5590af64cbc0 0x5590af668db0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.449+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2751913514 >> 172.21.15.112:0/2751913514 conn(0x5590af5bf6d0 msgr2=0x5590af64bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:12.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.449+0000 7f7f33400700 0 client.25400 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:12.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.450+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2751913514 shutdown_connections 2024-05-08T05:57:12.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.450+0000 7f7f5ee04b80 1 -- 172.21.15.112:0/2751913514 wait complete. 2024-05-08T05:57:12.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.450+0000 7f7f5ee04b80 20 client.25400 trim_cache size 0 max 16384 2024-05-08T05:57:12.450 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:57:12.450 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T05:57:12.451 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-05-08T05:57:12.451 INFO:teuthology.orchestra.run.smithi112.stdout: "5994005e-74db-4055-aad9-e5736dd5066b": { 2024-05-08T05:57:12.451 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-05-08T05:57:12.451 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "5994005e-74db-4055-aad9-e5736dd5066b", 2024-05-08T05:57:12.451 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-05-08T05:57:12.451 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T05:57:12.451 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T05:57:12.451 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:57:12.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.451+0000 7f7f54400700 1 -- 172.21.15.112:0/1506357509 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7f340b2880 msgr2=0x7f7f340b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:12.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.451+0000 7f7f54400700 1 --2- 172.21.15.112:0/1506357509 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7f340b2880 0x7f7f340b4d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c07b080 tx=0x7f7f40009260 comp rx=0 tx=0).stop 2024-05-08T05:57:12.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.451+0000 7f7f54400700 1 -- 172.21.15.112:0/1506357509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f4c108850 msgr2=0x7f7f4c19bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:12.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.451+0000 7f7f54400700 1 --2- 172.21.15.112:0/1506357509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f4c108850 0x7f7f4c19bb10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7f440079a0 tx=0x7f7f440079d0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.451+0000 7f7f54400700 1 -- 172.21.15.112:0/1506357509 shutdown_connections 2024-05-08T05:57:12.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.451+0000 7f7f54400700 1 --2- 172.21.15.112:0/1506357509 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f7f340b2880 0x7f7f340b4d40 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c07b080 tx=0x7f7f40009260 comp rx=0 tx=0).stop 2024-05-08T05:57:12.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.451+0000 7f7f54400700 1 --2- 172.21.15.112:0/1506357509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f4c108850 0x7f7f4c19bb10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.451+0000 7f7f54400700 1 --2- 172.21.15.112:0/1506357509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f4c103750 0x7f7f4c1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.451+0000 7f7f54400700 1 --2- 172.21.15.112:0/1506357509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f4c104100 0x7f7f4c1a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:12.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.451+0000 7f7f54400700 1 -- 172.21.15.112:0/1506357509 >> 172.21.15.112:0/1506357509 conn(0x7f7f4c07a600 msgr2=0x7f7f4c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:12.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.451+0000 7f7f54400700 1 -- 172.21.15.112:0/1506357509 shutdown_connections 2024-05-08T05:57:12.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:12.452+0000 7f7f54400700 1 -- 172.21.15.112:0/1506357509 wait complete. 2024-05-08T05:57:13.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:13 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:57:13.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:13 smithi112 ceph-mon[30003]: pgmap v498: 97 pgs: 97 active+clean; 24 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 998 KiB/s wr, 120 op/s 2024-05-08T05:57:13.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:13 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:57:13.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:13 smithi162 ceph-mon[34800]: pgmap v498: 97 pgs: 97 active+clean; 24 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 998 KiB/s wr, 120 op/s 2024-05-08T05:57:13.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:13 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:57:13.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:13 smithi150 ceph-mon[34315]: pgmap v498: 97 pgs: 97 active+clean; 24 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 998 KiB/s wr, 120 op/s 2024-05-08T05:57:13.487 INFO:tasks.cephfs.filesystem:scrub status for tag:5994005e-74db-4055-aad9-e5736dd5066b - {'path': '/', 'tag': '5994005e-74db-4055-aad9-e5736dd5066b', 'options': 'recursive,force'} 2024-05-08T05:57:14.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:14 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T05:57:14.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:14 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T05:57:14.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:14 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T05:57:14.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:14 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T05:57:14.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:14 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T05:57:14.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:14 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T05:57:14.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:14 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T05:57:14.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:14 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T05:57:14.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:14 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T05:57:15.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:15 smithi112 ceph-mon[30003]: pgmap v499: 97 pgs: 97 active+clean; 27 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-05-08T05:57:15.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:15 smithi162 ceph-mon[34800]: pgmap v499: 97 pgs: 97 active+clean; 27 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-05-08T05:57:15.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:15 smithi150 ceph-mon[34315]: pgmap v499: 97 pgs: 97 active+clean; 27 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-05-08T05:57:16.046 INFO:tasks.workunit.client.0.smithi112.stderr:Updating files: 7% (321/4287) Updating files: 8% (343/4287) Updating files: 8% (361/4287) Updating files: 9% (386/4287) Updating files: 10% (429/4287) Updating files: 11% (472/4287) Updating files: 11% (490/4287) Updating files: 12% (515/4287) Updating files: 13% (558/4287) Updating files: 13% (559/4287) Updating files: 14% (601/4287) Updating files: 15% (644/4287) Updating files: 16% (686/4287) Updating files: 17% (729/4287) Updating files: 18% (772/4287) Updating files: 19% (815/4287) Updating files: 20% (858/4287) Updating files: 21% (901/4287) Updating files: 22% (944/4287) Updating files: 23% (987/4287) Updating files: 24% (1029/4287) Updating files: 25% (1072/4287) Updating files: 25% (1083/4287) Updating files: 26% (1115/4287) Updating files: 27% (1158/4287) Updating files: 28% (1201/4287) Updating files: 29% (1244/4287) Updating files: 30% (1287/4287) Updating files: 31% (1329/4287) Updating files: 32% (1372/4287) Updating files: 33% (1415/4287) Updating files: 34% (1458/4287) Updating files: 35% (1501/4287) Updating files: 36% (1544/4287) Updating files: 36% (1549/4287) Updating files: 37% (1587/4287) Updating files: 38% (1630/4287) Updating files: 39% (1672/4287) Updating files: 40% (1715/4287) Updating files: 41% (1758/4287) Updating files: 42% (1801/4287) Updating files: 43% (1844/4287) Updating files: 44% (1887/4287) Updating files: 45% (1930/4287) Updating files: 46% (1973/4287) Updating files: 47% (2015/4287) Updating files: 48% (2058/4287) Updating files: 48% (2063/4287) Updating files: 49% (2101/4287) Updating files: 50% (2144/4287) Updating files: 51% (2187/4287) Updating files: 52% (2230/4287) Updating files: 53% (2273/4287) Updating files: 54% (2315/4287) Updating files: 55% (2358/4287) Updating files: 55% (2390/4287) Updating files: 56% (2401/4287) Updating files: 57% (2444/4287) Updating files: 58% (2487/4287) Updating files: 59% (2530/4287) Updating files: 60% (2573/4287) Updating files: 61% (2616/4287) Updating files: 62% (2658/4287) Updating files: 63% (2701/4287) Updating files: 64% (2744/4287) Updating files: 65% (2787/4287) Updating files: 65% (2808/4287) Updating files: 66% (2830/4287) Updating files: 67% (2873/4287) Updating files: 68% (2916/4287) Updating files: 69% (2959/4287) Updating files: 70% (3001/4287) Updating files: 70% (3022/4287) Updating files: 71% (3044/4287) Updating files: 72% (3087/4287) Updating files: 73% (3130/4287) Updating files: 73% (3155/4287) Updating files: 74% (3173/4287) Updating files: 75% (3216/4287) Updating files: 76% (3259/4287) Updating files: 77% (3301/4287) Updating files: 78% (3344/4287) Updating files: 79% (3387/4287) Updating files: 80% (3430/4287) Updating files: 81% (3473/4287) Updating files: 82% (3516/4287) Updating files: 83% (3559/4287) Updating files: 84% (3602/4287) Updating files: 85% (3644/4287) Updating files: 86% (3687/4287) Updating files: 87% (3730/4287) Updating files: 87% (3771/4287) Updating files: 88% (3773/4287) Updating files: 89% (3816/4287) Updating files: 90% (3859/4287) Updating files: 91% (3902/4287) Updating files: 91% (3927/4287) Updating files: 92% (3945/4287) Updating files: 93% (3987/4287) Updating files: 93% (3995/4287) Updating files: 94% (4030/4287) Updating files: 95% (4073/4287) Updating files: 96% (4116/4287) Updating files: 97% (4159/4287) Updating files: 97% (4195/4287) Updating files: 98% (4202/4287) Updating files: 99% (4245/4287) Updating files: 100% (4287/4287) Updating files: 100% (4287/4287), done. 2024-05-08T05:57:16.072 INFO:tasks.workunit.client.0.smithi112.stderr:+ cd xfstests-dev 2024-05-08T05:57:16.072 INFO:tasks.workunit.client.0.smithi112.stderr:+ make -j4 2024-05-08T05:57:16.104 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize -cfi 2024-05-08T05:57:17.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:17 smithi112 ceph-mon[30003]: pgmap v500: 97 pgs: 97 active+clean; 32 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-05-08T05:57:17.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:17 smithi162 ceph-mon[34800]: pgmap v500: 97 pgs: 97 active+clean; 32 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-05-08T05:57:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:17 smithi150 ceph-mon[34315]: pgmap v500: 97 pgs: 97 active+clean; 32 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-05-08T05:57:18.512 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: putting auxiliary files in '.'. 2024-05-08T05:57:18.512 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file './config.guess' 2024-05-08T05:57:18.527 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file './config.sub' 2024-05-08T05:57:18.542 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file './install-sh' 2024-05-08T05:57:18.558 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file './ltmain.sh' 2024-05-08T05:57:18.644 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-05-08T05:57:18.644 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-05-08T05:57:18.663 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-05-08T05:57:18.681 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-05-08T05:57:18.701 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-05-08T05:57:18.720 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-05-08T05:57:18.744 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-05-08T05:57:18.748 INFO:tasks.workunit.client.0.smithi112.stdout:cp include/install-sh . 2024-05-08T05:57:18.765 INFO:tasks.workunit.client.0.smithi112.stdout:aclocal -I m4 2024-05-08T05:57:19.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:19 smithi112 ceph-mon[30003]: pgmap v501: 97 pgs: 97 active+clean; 34 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.9 MiB/s wr, 275 op/s 2024-05-08T05:57:19.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:19 smithi162 ceph-mon[34800]: pgmap v501: 97 pgs: 97 active+clean; 34 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.9 MiB/s wr, 275 op/s 2024-05-08T05:57:19.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:19 smithi150 ceph-mon[34315]: pgmap v501: 97 pgs: 97 active+clean; 34 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.9 MiB/s wr, 275 op/s 2024-05-08T05:57:20.427 INFO:tasks.workunit.client.0.smithi112.stdout:autoheader 2024-05-08T05:57:20.881 INFO:tasks.workunit.client.0.smithi112.stdout:autoconf 2024-05-08T05:57:21.155 INFO:tasks.workunit.client.0.smithi112.stdout:./configure \ 2024-05-08T05:57:21.155 INFO:tasks.workunit.client.0.smithi112.stdout: --libexecdir=/usr/lib \ 2024-05-08T05:57:21.155 INFO:tasks.workunit.client.0.smithi112.stdout: --exec_prefix=/var/lib 2024-05-08T05:57:21.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:21 smithi112 ceph-mon[30003]: pgmap v502: 97 pgs: 97 active+clean; 37 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 311 op/s 2024-05-08T05:57:21.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:21 smithi162 ceph-mon[34800]: pgmap v502: 97 pgs: 97 active+clean; 37 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 311 op/s 2024-05-08T05:57:21.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:21 smithi150 ceph-mon[34315]: pgmap v502: 97 pgs: 97 active+clean; 37 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 311 op/s 2024-05-08T05:57:21.571 INFO:tasks.workunit.client.0.smithi112.stdout:checking build system type... x86_64-pc-linux-gnu 2024-05-08T05:57:21.571 INFO:tasks.workunit.client.0.smithi112.stdout:checking host system type... x86_64-pc-linux-gnu 2024-05-08T05:57:21.574 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to print strings... printf 2024-05-08T05:57:21.575 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc... gcc 2024-05-08T05:57:21.799 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether the C compiler works... yes 2024-05-08T05:57:21.799 INFO:tasks.workunit.client.0.smithi112.stdout:checking for C compiler default output file name... a.out 2024-05-08T05:57:21.878 INFO:tasks.workunit.client.0.smithi112.stdout:checking for suffix of executables... 2024-05-08T05:57:21.966 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether we are cross compiling... no 2024-05-08T05:57:22.056 INFO:tasks.workunit.client.0.smithi112.stdout:checking for suffix of object files... o 2024-05-08T05:57:22.111 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether we are using the GNU C compiler... yes 2024-05-08T05:57:22.167 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether gcc accepts -g... yes 2024-05-08T05:57:22.310 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-08T05:57:22.327 INFO:tasks.workunit.client.0.smithi112.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-05-08T05:57:22.342 INFO:tasks.workunit.client.0.smithi112.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-08T05:57:22.346 INFO:tasks.workunit.client.0.smithi112.stdout:checking for egrep... /usr/bin/grep -E 2024-05-08T05:57:22.351 INFO:tasks.workunit.client.0.smithi112.stdout:checking for fgrep... /usr/bin/grep -F 2024-05-08T05:57:22.363 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ld used by gcc... /usr/bin/ld 2024-05-08T05:57:22.369 INFO:tasks.workunit.client.0.smithi112.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-05-08T05:57:22.394 INFO:tasks.workunit.client.0.smithi112.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-05-08T05:57:23.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:23 smithi112 ceph-mon[30003]: pgmap v503: 97 pgs: 97 active+clean; 37 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.7 MiB/s wr, 276 op/s 2024-05-08T05:57:23.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:23 smithi162 ceph-mon[34800]: pgmap v503: 97 pgs: 97 active+clean; 37 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.7 MiB/s wr, 276 op/s 2024-05-08T05:57:23.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:23 smithi150 ceph-mon[34315]: pgmap v503: 97 pgs: 97 active+clean; 37 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.7 MiB/s wr, 276 op/s 2024-05-08T05:57:23.571 INFO:tasks.workunit.client.0.smithi112.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-05-08T05:57:23.571 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether ln -s works... yes 2024-05-08T05:57:23.585 INFO:tasks.workunit.client.0.smithi112.stdout:checking the maximum length of command line arguments... 1572864 2024-05-08T05:57:23.593 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-05-08T05:57:23.593 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-05-08T05:57:23.593 INFO:tasks.workunit.client.0.smithi112.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-05-08T05:57:23.594 INFO:tasks.workunit.client.0.smithi112.stdout:checking for objdump... objdump 2024-05-08T05:57:23.594 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to recognize dependent libraries... pass_all 2024-05-08T05:57:23.595 INFO:tasks.workunit.client.0.smithi112.stdout:checking for dlltool... no 2024-05-08T05:57:23.595 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-05-08T05:57:23.595 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ar... ar 2024-05-08T05:57:23.771 INFO:tasks.workunit.client.0.smithi112.stdout:checking for archiver @FILE support... @ 2024-05-08T05:57:23.772 INFO:tasks.workunit.client.0.smithi112.stdout:checking for strip... strip 2024-05-08T05:57:23.773 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ranlib... ranlib 2024-05-08T05:57:23.774 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gawk... gawk 2024-05-08T05:57:24.021 INFO:tasks.workunit.client.0.smithi112.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-05-08T05:57:24.026 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sysroot... no 2024-05-08T05:57:24.045 INFO:tasks.workunit.client.0.smithi112.stdout:checking for a working dd... /usr/bin/dd 2024-05-08T05:57:24.072 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-05-08T05:57:24.144 INFO:tasks.workunit.client.0.smithi112.stdout:checking for mt... no 2024-05-08T05:57:24.179 INFO:tasks.workunit.client.0.smithi112.stdout:checking if : is a manifest tool... no 2024-05-08T05:57:24.345 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to run the C preprocessor... gcc -E 2024-05-08T05:57:24.791 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ANSI C header files... yes 2024-05-08T05:57:24.866 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/types.h... yes 2024-05-08T05:57:24.959 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/stat.h... yes 2024-05-08T05:57:25.059 INFO:tasks.workunit.client.0.smithi112.stdout:checking for stdlib.h... yes 2024-05-08T05:57:25.152 INFO:tasks.workunit.client.0.smithi112.stdout:checking for string.h... yes 2024-05-08T05:57:25.254 INFO:tasks.workunit.client.0.smithi112.stdout:checking for memory.h... yes 2024-05-08T05:57:25.340 INFO:tasks.workunit.client.0.smithi112.stdout:checking for strings.h... yes 2024-05-08T05:57:25.425 INFO:tasks.workunit.client.0.smithi112.stdout:checking for inttypes.h... yes 2024-05-08T05:57:25.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:25 smithi150 ceph-mon[34315]: pgmap v504: 97 pgs: 97 active+clean; 37 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-05-08T05:57:25.524 INFO:tasks.workunit.client.0.smithi112.stdout:checking for stdint.h... yes 2024-05-08T05:57:25.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:25 smithi112 ceph-mon[30003]: pgmap v504: 97 pgs: 97 active+clean; 37 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-05-08T05:57:25.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:25 smithi162 ceph-mon[34800]: pgmap v504: 97 pgs: 97 active+clean; 37 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-05-08T05:57:25.626 INFO:tasks.workunit.client.0.smithi112.stdout:checking for unistd.h... yes 2024-05-08T05:57:25.728 INFO:tasks.workunit.client.0.smithi112.stdout:checking for dlfcn.h... yes 2024-05-08T05:57:25.743 INFO:tasks.workunit.client.0.smithi112.stdout:checking for objdir... .libs 2024-05-08T05:57:25.985 INFO:tasks.workunit.client.0.smithi112.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-05-08T05:57:25.986 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-05-08T05:57:26.085 INFO:tasks.workunit.client.0.smithi112.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-05-08T05:57:26.199 INFO:tasks.workunit.client.0.smithi112.stdout:checking if gcc static flag -static works... no 2024-05-08T05:57:26.378 INFO:tasks.workunit.client.0.smithi112.stdout:checking if gcc supports -c -o file.o... yes 2024-05-08T05:57:26.378 INFO:tasks.workunit.client.0.smithi112.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-05-08T05:57:26.398 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-05-08T05:57:26.512 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether -lc should be explicitly linked in... no 2024-05-08T05:57:26.678 INFO:tasks.workunit.client.0.smithi112.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-05-08T05:57:26.679 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to hardcode library paths into programs... immediate 2024-05-08T05:57:26.680 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether stripping libraries is possible... yes 2024-05-08T05:57:26.681 INFO:tasks.workunit.client.0.smithi112.stdout:checking if libtool supports shared libraries... yes 2024-05-08T05:57:26.681 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether to build shared libraries... yes 2024-05-08T05:57:26.681 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether to build static libraries... yes 2024-05-08T05:57:26.703 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc... (cached) gcc 2024-05-08T05:57:26.810 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-08T05:57:26.810 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-08T05:57:26.813 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-08T05:57:26.814 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gmake... /usr/bin/gmake 2024-05-08T05:57:26.815 INFO:tasks.workunit.client.0.smithi112.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-05-08T05:57:26.815 INFO:tasks.workunit.client.0.smithi112.stdout:checking for tar... /bin/tar 2024-05-08T05:57:26.816 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gzip... /bin/gzip 2024-05-08T05:57:26.883 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether gcc -MM is supported... yes 2024-05-08T05:57:26.884 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sort... /bin/sort 2024-05-08T05:57:26.884 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether ln -s works... yes 2024-05-08T05:57:26.885 INFO:tasks.workunit.client.0.smithi112.stdout:checking for rpm... /bin/rpm 2024-05-08T05:57:26.897 INFO:tasks.workunit.client.0.smithi112.stdout:checking for rpmbuild... no 2024-05-08T05:57:27.009 INFO:tasks.workunit.client.0.smithi112.stdout:checking assert.h usability... yes 2024-05-08T05:57:27.049 INFO:tasks.workunit.client.0.smithi112.stdout:checking assert.h presence... yes 2024-05-08T05:57:27.049 INFO:tasks.workunit.client.0.smithi112.stdout:checking for assert.h... yes 2024-05-08T05:57:27.171 INFO:tasks.workunit.client.0.smithi112.stdout:checking bstring.h usability... no 2024-05-08T05:57:27.245 INFO:tasks.workunit.client.0.smithi112.stdout:checking bstring.h presence... no 2024-05-08T05:57:27.245 INFO:tasks.workunit.client.0.smithi112.stdout:checking for bstring.h... no 2024-05-08T05:57:27.334 INFO:tasks.workunit.client.0.smithi112.stdout:checking libgen.h usability... yes 2024-05-08T05:57:27.368 INFO:tasks.workunit.client.0.smithi112.stdout:checking libgen.h presence... yes 2024-05-08T05:57:27.369 INFO:tasks.workunit.client.0.smithi112.stdout:checking for libgen.h... yes 2024-05-08T05:57:27.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:27 smithi150 ceph-mon[34315]: pgmap v505: 97 pgs: 97 active+clean; 38 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-05-08T05:57:27.467 INFO:tasks.workunit.client.0.smithi112.stdout:checking dirent.h usability... yes 2024-05-08T05:57:27.509 INFO:tasks.workunit.client.0.smithi112.stdout:checking dirent.h presence... yes 2024-05-08T05:57:27.510 INFO:tasks.workunit.client.0.smithi112.stdout:checking for dirent.h... yes 2024-05-08T05:57:27.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:27 smithi112 ceph-mon[30003]: pgmap v505: 97 pgs: 97 active+clean; 38 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-05-08T05:57:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:27 smithi162 ceph-mon[34800]: pgmap v505: 97 pgs: 97 active+clean; 38 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-05-08T05:57:27.614 INFO:tasks.workunit.client.0.smithi112.stdout:checking errno.h usability... yes 2024-05-08T05:57:27.662 INFO:tasks.workunit.client.0.smithi112.stdout:checking errno.h presence... yes 2024-05-08T05:57:27.663 INFO:tasks.workunit.client.0.smithi112.stdout:checking for errno.h... yes 2024-05-08T05:57:27.779 INFO:tasks.workunit.client.0.smithi112.stdout:checking malloc.h usability... yes 2024-05-08T05:57:28.153 INFO:tasks.workunit.client.0.smithi112.stdout:checking malloc.h presence... yes 2024-05-08T05:57:28.153 INFO:tasks.workunit.client.0.smithi112.stdout:checking for malloc.h... yes 2024-05-08T05:57:28.153 INFO:tasks.workunit.client.0.smithi112.stdout:checking uuid.h usability... no 2024-05-08T05:57:28.153 INFO:tasks.workunit.client.0.smithi112.stdout:checking uuid.h presence... no 2024-05-08T05:57:28.154 INFO:tasks.workunit.client.0.smithi112.stdout:checking for uuid.h... no 2024-05-08T05:57:28.154 INFO:tasks.workunit.client.0.smithi112.stdout:checking uuid/uuid.h usability... yes 2024-05-08T05:57:28.173 INFO:tasks.workunit.client.0.smithi112.stdout:checking uuid/uuid.h presence... yes 2024-05-08T05:57:28.173 INFO:tasks.workunit.client.0.smithi112.stdout:checking for uuid/uuid.h... yes 2024-05-08T05:57:28.286 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/uuid.h usability... no 2024-05-08T05:57:28.353 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/uuid.h presence... no 2024-05-08T05:57:28.353 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/uuid.h... no 2024-05-08T05:57:28.443 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/file.h usability... yes 2024-05-08T05:57:28.477 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/file.h presence... yes 2024-05-08T05:57:28.477 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/file.h... yes 2024-05-08T05:57:28.568 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fcntl.h usability... yes 2024-05-08T05:57:28.600 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fcntl.h presence... yes 2024-05-08T05:57:28.601 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/fcntl.h... yes 2024-05-08T05:57:28.699 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/syssgi.h usability... no 2024-05-08T05:57:28.777 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/syssgi.h presence... no 2024-05-08T05:57:28.777 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/syssgi.h... no 2024-05-08T05:57:28.889 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/param.h usability... yes 2024-05-08T05:57:28.932 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/param.h presence... yes 2024-05-08T05:57:28.932 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/param.h... yes 2024-05-08T05:57:28.964 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/stat.h... (cached) yes 2024-05-08T05:57:29.061 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/statvfs.h usability... yes 2024-05-08T05:57:29.099 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/statvfs.h presence... yes 2024-05-08T05:57:29.100 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/statvfs.h... yes 2024-05-08T05:57:29.187 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/time.h usability... yes 2024-05-08T05:57:29.226 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/time.h presence... yes 2024-05-08T05:57:29.226 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/time.h... yes 2024-05-08T05:57:29.317 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/ioctl.h usability... yes 2024-05-08T05:57:29.365 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/ioctl.h presence... yes 2024-05-08T05:57:29.365 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/ioctl.h... yes 2024-05-08T05:57:29.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:29 smithi150 ceph-mon[34315]: pgmap v506: 97 pgs: 97 active+clean; 38 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 629 KiB/s wr, 84 op/s 2024-05-08T05:57:29.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:29 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:57:29.485 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/wait.h usability... yes 2024-05-08T05:57:29.524 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/wait.h presence... yes 2024-05-08T05:57:29.524 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/wait.h... yes 2024-05-08T05:57:29.558 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/types.h... (cached) yes 2024-05-08T05:57:29.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:29 smithi112 ceph-mon[30003]: pgmap v506: 97 pgs: 97 active+clean; 38 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 629 KiB/s wr, 84 op/s 2024-05-08T05:57:29.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:29 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:57:29.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:29 smithi162 ceph-mon[34800]: pgmap v506: 97 pgs: 97 active+clean; 38 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 629 KiB/s wr, 84 op/s 2024-05-08T05:57:29.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:29 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:57:29.574 INFO:tasks.workunit.client.0.smithi112.stdout:checking for strings.h... (cached) yes 2024-05-08T05:57:29.654 INFO:tasks.workunit.client.0.smithi112.stdout:checking err.h usability... yes 2024-05-08T05:57:29.684 INFO:tasks.workunit.client.0.smithi112.stdout:checking err.h presence... yes 2024-05-08T05:57:29.685 INFO:tasks.workunit.client.0.smithi112.stdout:checking for err.h... yes 2024-05-08T05:57:29.773 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/falloc.h usability... yes 2024-05-08T05:57:29.815 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/falloc.h presence... yes 2024-05-08T05:57:29.816 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/falloc.h... yes 2024-05-08T05:57:29.930 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-05-08T05:57:29.995 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-05-08T05:57:29.995 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/fs/xfs_fsops.h... no 2024-05-08T05:57:30.100 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fs/xfs_itable.h usability... no 2024-05-08T05:57:30.178 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fs/xfs_itable.h presence... no 2024-05-08T05:57:30.178 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/fs/xfs_itable.h... no 2024-05-08T05:57:30.285 INFO:tasks.workunit.client.0.smithi112.stdout:checking xfs/platform_defs.h usability... no 2024-05-08T05:57:30.359 INFO:tasks.workunit.client.0.smithi112.stdout:checking xfs/platform_defs.h presence... no 2024-05-08T05:57:30.359 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/platform_defs.h... no 2024-05-08T05:57:30.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:30 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:57:30.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:30 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:57:30.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:30 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:57:30.476 INFO:tasks.workunit.client.0.smithi112.stdout:checking btrfs/ioctl.h usability... no 2024-05-08T05:57:30.547 INFO:tasks.workunit.client.0.smithi112.stdout:checking btrfs/ioctl.h presence... no 2024-05-08T05:57:30.547 INFO:tasks.workunit.client.0.smithi112.stdout:checking for btrfs/ioctl.h... no 2024-05-08T05:57:30.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:57:30.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:57:30.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:57:30.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:30 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:57:30.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:30 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:57:30.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:30 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:57:30.679 INFO:tasks.workunit.client.0.smithi112.stdout:checking cifs/ioctl.h usability... no 2024-05-08T05:57:30.772 INFO:tasks.workunit.client.0.smithi112.stdout:checking cifs/ioctl.h presence... no 2024-05-08T05:57:30.772 INFO:tasks.workunit.client.0.smithi112.stdout:checking for cifs/ioctl.h... no 2024-05-08T05:57:30.888 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/mman.h usability... yes 2024-05-08T05:57:30.928 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/mman.h presence... yes 2024-05-08T05:57:30.928 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/mman.h... yes 2024-05-08T05:57:31.036 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/ext4.h usability... no 2024-05-08T05:57:31.116 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/ext4.h presence... no 2024-05-08T05:57:31.116 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/ext4.h... no 2024-05-08T05:57:31.181 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/xfs_log_format.h... no 2024-05-08T05:57:31.300 INFO:tasks.workunit.client.0.smithi112.stdout:checking for uuid_compare... no 2024-05-08T05:57:31.392 INFO:tasks.workunit.client.0.smithi112.stdout:checking for uuid_compare in -luuid... yes 2024-05-08T05:57:31.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:31 smithi150 ceph-mon[34315]: pgmap v507: 97 pgs: 97 active+clean; 40 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 516 KiB/s wr, 62 op/s 2024-05-08T05:57:31.512 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/xfs.h... yes 2024-05-08T05:57:31.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:31 smithi112 ceph-mon[30003]: pgmap v507: 97 pgs: 97 active+clean; 40 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 516 KiB/s wr, 62 op/s 2024-05-08T05:57:31.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:31 smithi162 ceph-mon[34800]: pgmap v507: 97 pgs: 97 active+clean; 40 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 516 KiB/s wr, 62 op/s 2024-05-08T05:57:31.599 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/libxfs.h... no 2024-05-08T05:57:31.685 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether xlog_assign_lsn is declared... no 2024-05-08T05:57:31.783 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/xqm.h... yes 2024-05-08T05:57:31.899 INFO:tasks.workunit.client.0.smithi112.stdout:checking xfsctl from xfs/xfs.h... ok 2024-05-08T05:57:31.974 INFO:tasks.workunit.client.0.smithi112.stdout:checking xfs/handle.h usability... yes 2024-05-08T05:57:32.022 INFO:tasks.workunit.client.0.smithi112.stdout:checking xfs/handle.h presence... yes 2024-05-08T05:57:32.022 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/handle.h... yes 2024-05-08T05:57:32.138 INFO:tasks.workunit.client.0.smithi112.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-05-08T05:57:32.227 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/acl.h usability... yes 2024-05-08T05:57:32.277 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/acl.h presence... yes 2024-05-08T05:57:32.277 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/acl.h... yes 2024-05-08T05:57:32.363 INFO:tasks.workunit.client.0.smithi112.stdout:checking acl/libacl.h usability... yes 2024-05-08T05:57:32.418 INFO:tasks.workunit.client.0.smithi112.stdout:checking acl/libacl.h presence... yes 2024-05-08T05:57:32.419 INFO:tasks.workunit.client.0.smithi112.stdout:checking for acl/libacl.h... yes 2024-05-08T05:57:32.528 INFO:tasks.workunit.client.0.smithi112.stdout:checking for acl_init in -lacl... yes 2024-05-08T05:57:32.664 INFO:tasks.workunit.client.0.smithi112.stdout:checking gdbm-ndbm.h usability... no 2024-05-08T05:57:32.747 INFO:tasks.workunit.client.0.smithi112.stdout:checking gdbm-ndbm.h presence... no 2024-05-08T05:57:32.747 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gdbm-ndbm.h... no 2024-05-08T05:57:32.844 INFO:tasks.workunit.client.0.smithi112.stdout:checking gdbm/ndbm.h usability... no 2024-05-08T05:57:32.919 INFO:tasks.workunit.client.0.smithi112.stdout:checking gdbm/ndbm.h presence... no 2024-05-08T05:57:32.919 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gdbm/ndbm.h... no 2024-05-08T05:57:33.002 INFO:tasks.workunit.client.0.smithi112.stdout:checking ndbm.h usability... no 2024-05-08T05:57:33.090 INFO:tasks.workunit.client.0.smithi112.stdout:checking ndbm.h presence... no 2024-05-08T05:57:33.090 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ndbm.h... no 2024-05-08T05:57:33.170 INFO:tasks.workunit.client.0.smithi112.stdout:checking libaio.h usability... yes 2024-05-08T05:57:33.219 INFO:tasks.workunit.client.0.smithi112.stdout:checking libaio.h presence... yes 2024-05-08T05:57:33.220 INFO:tasks.workunit.client.0.smithi112.stdout:checking for libaio.h... yes 2024-05-08T05:57:33.377 INFO:tasks.workunit.client.0.smithi112.stdout:checking liburing.h usability... no 2024-05-08T05:57:33.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:33 smithi150 ceph-mon[34315]: pgmap v508: 97 pgs: 97 active+clean; 40 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 236 KiB/s wr, 25 op/s 2024-05-08T05:57:33.488 INFO:tasks.workunit.client.0.smithi112.stdout:checking liburing.h presence... no 2024-05-08T05:57:33.488 INFO:tasks.workunit.client.0.smithi112.stdout:checking for liburing.h... no 2024-05-08T05:57:33.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:33 smithi112 ceph-mon[30003]: pgmap v508: 97 pgs: 97 active+clean; 40 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 236 KiB/s wr, 25 op/s 2024-05-08T05:57:33.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:33 smithi162 ceph-mon[34800]: pgmap v508: 97 pgs: 97 active+clean; 40 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 236 KiB/s wr, 25 op/s 2024-05-08T05:57:33.619 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/capability.h usability... no 2024-05-08T05:57:33.711 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/capability.h presence... no 2024-05-08T05:57:33.711 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/capability.h... no 2024-05-08T05:57:33.791 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/fiemap.h usability... yes 2024-05-08T05:57:33.829 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/fiemap.h presence... yes 2024-05-08T05:57:33.829 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/fiemap.h... yes 2024-05-08T05:57:33.923 INFO:tasks.workunit.client.0.smithi112.stdout:checking for fallocate... yes 2024-05-08T05:57:34.032 INFO:tasks.workunit.client.0.smithi112.stdout:checking for open_by_handle_at... yes 2024-05-08T05:57:34.120 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/fs.h usability... yes 2024-05-08T05:57:34.171 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/fs.h presence... yes 2024-05-08T05:57:34.171 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/fs.h... yes 2024-05-08T05:57:34.306 INFO:tasks.workunit.client.0.smithi112.stdout:checking btrfsutil.h usability... no 2024-05-08T05:57:34.380 INFO:tasks.workunit.client.0.smithi112.stdout:checking btrfsutil.h presence... no 2024-05-08T05:57:34.381 INFO:tasks.workunit.client.0.smithi112.stdout:checking for btrfsutil.h... no 2024-05-08T05:57:34.484 INFO:tasks.workunit.client.0.smithi112.stdout:checking for copy_file_range... yes 2024-05-08T05:57:34.592 INFO:tasks.workunit.client.0.smithi112.stdout:checking for SEEK_DATA... yes 2024-05-08T05:57:34.907 INFO:tasks.workunit.client.0.smithi112.stdout:checking for BMV_OF_SHARED... yes 2024-05-08T05:57:35.020 INFO:tasks.workunit.client.0.smithi112.stdout:checking for nftw... yes 2024-05-08T05:57:35.115 INFO:tasks.workunit.client.0.smithi112.stdout:checking for RLIMIT_NOFILE... yes 2024-05-08T05:57:35.218 INFO:tasks.workunit.client.0.smithi112.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-05-08T05:57:35.412 INFO:tasks.workunit.client.0.smithi112.stdout:checking for FICLONE... yes 2024-05-08T05:57:35.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:35 smithi150 ceph-mon[34315]: pgmap v509: 97 pgs: 97 active+clean; 40 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 236 KiB/s wr, 25 op/s 2024-05-08T05:57:35.509 INFO:tasks.workunit.client.0.smithi112.stdout:checking for renameat2... yes 2024-05-08T05:57:35.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:35 smithi112 ceph-mon[30003]: pgmap v509: 97 pgs: 97 active+clean; 40 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 236 KiB/s wr, 25 op/s 2024-05-08T05:57:35.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:35 smithi162 ceph-mon[34800]: pgmap v509: 97 pgs: 97 active+clean; 40 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 236 KiB/s wr, 25 op/s 2024-05-08T05:57:35.604 INFO:tasks.workunit.client.0.smithi112.stdout:checking for reallocarray... yes 2024-05-08T05:57:35.765 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct mount_attr... yes 2024-05-08T05:57:35.960 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_qgroup_limit... yes 2024-05-08T05:57:36.069 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-05-08T05:57:36.198 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-05-08T05:57:36.346 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-05-08T05:57:36.487 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-05-08T05:57:36.616 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-05-08T05:57:36.756 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-05-08T05:57:36.847 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/btrfs.h usability... yes 2024-05-08T05:57:36.910 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/btrfs.h presence... yes 2024-05-08T05:57:36.910 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/btrfs.h... yes 2024-05-08T05:57:37.009 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/btrfs_tree.h usability... yes 2024-05-08T05:57:37.053 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/btrfs_tree.h presence... yes 2024-05-08T05:57:37.053 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/btrfs_tree.h... yes 2024-05-08T05:57:37.146 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-05-08T05:57:37.222 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-05-08T05:57:37.260 INFO:tasks.workunit.client.0.smithi112.stdout:configure: creating ./config.status 2024-05-08T05:57:37.367 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:37 smithi112 ceph-mon[30003]: pgmap v510: 97 pgs: 97 active+clean; 44 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 599 KiB/s wr, 59 op/s 2024-05-08T05:57:37.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:37 smithi150 ceph-mon[34315]: pgmap v510: 97 pgs: 97 active+clean; 44 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 599 KiB/s wr, 59 op/s 2024-05-08T05:57:37.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:37 smithi162 ceph-mon[34800]: pgmap v510: 97 pgs: 97 active+clean; 44 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 599 KiB/s wr, 59 op/s 2024-05-08T05:57:38.449 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: creating include/builddefs 2024-05-08T05:57:39.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:39 smithi112 ceph-mon[30003]: pgmap v511: 97 pgs: 97 active+clean; 45 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 709 KiB/s wr, 100 op/s 2024-05-08T05:57:39.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:39 smithi150 ceph-mon[34315]: pgmap v511: 97 pgs: 97 active+clean; 45 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 709 KiB/s wr, 100 op/s 2024-05-08T05:57:39.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:39 smithi162 ceph-mon[34800]: pgmap v511: 97 pgs: 97 active+clean; 45 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 709 KiB/s wr, 100 op/s 2024-05-08T05:57:41.094 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: creating include/config.h 2024-05-08T05:57:41.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:41 smithi150 ceph-mon[34315]: pgmap v512: 97 pgs: 97 active+clean; 51 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-05-08T05:57:41.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:41 smithi112 ceph-mon[30003]: pgmap v512: 97 pgs: 97 active+clean; 51 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-05-08T05:57:41.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:41 smithi162 ceph-mon[34800]: pgmap v512: 97 pgs: 97 active+clean; 51 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-05-08T05:57:43.436 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:43 smithi150 ceph-mon[34315]: pgmap v513: 97 pgs: 97 active+clean; 50 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.0 MiB/s wr, 162 op/s 2024-05-08T05:57:43.488 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T05:57:43.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:43 smithi112 ceph-mon[30003]: pgmap v513: 97 pgs: 97 active+clean; 50 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.0 MiB/s wr, 162 op/s 2024-05-08T05:57:43.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:43 smithi162 ceph-mon[34800]: pgmap v513: 97 pgs: 97 active+clean; 50 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.0 MiB/s wr, 162 op/s 2024-05-08T05:57:43.952 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:57:45.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:45 smithi112 ceph-mon[30003]: pgmap v514: 97 pgs: 97 active+clean; 50 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.1 MiB/s wr, 167 op/s 2024-05-08T05:57:45.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:45 smithi150 ceph-mon[34315]: pgmap v514: 97 pgs: 97 active+clean; 50 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.1 MiB/s wr, 167 op/s 2024-05-08T05:57:45.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:45 smithi162 ceph-mon[34800]: pgmap v514: 97 pgs: 97 active+clean; 50 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.1 MiB/s wr, 167 op/s 2024-05-08T05:57:46.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.251+0000 7fd043000700 1 -- 172.21.15.112:0/2750602230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd03c1060a0 msgr2=0x7fd03c1125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:46.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.251+0000 7fd043000700 1 --2- 172.21.15.112:0/2750602230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd03c1060a0 0x7fd03c1125b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd03400b0a0 tx=0x7fd03400b3b0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.252+0000 7fd043000700 1 -- 172.21.15.112:0/2750602230 shutdown_connections 2024-05-08T05:57:46.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.252+0000 7fd043000700 1 --2- 172.21.15.112:0/2750602230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd03c1060a0 0x7fd03c1125b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.252+0000 7fd043000700 1 --2- 172.21.15.112:0/2750602230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd03c10b6e0 0x7fd03c10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.252+0000 7fd043000700 1 --2- 172.21.15.112:0/2750602230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd03c1056e0 0x7fd03c105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.252+0000 7fd043000700 1 -- 172.21.15.112:0/2750602230 >> 172.21.15.112:0/2750602230 conn(0x7fd03c1011d0 msgr2=0x7fd03c1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:46.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.252+0000 7fd043000700 1 -- 172.21.15.112:0/2750602230 shutdown_connections 2024-05-08T05:57:46.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.252+0000 7fd043000700 1 -- 172.21.15.112:0/2750602230 wait complete. 2024-05-08T05:57:46.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.253+0000 7fd043000700 1 Processor -- start 2024-05-08T05:57:46.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.253+0000 7fd043000700 1 -- start start 2024-05-08T05:57:46.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.253+0000 7fd043000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd03c1056e0 0x7fd03c19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.253+0000 7fd043000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd03c1060a0 0x7fd03c19c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.253+0000 7fd043000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd03c10b6e0 0x7fd03c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.253+0000 7fd043000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c115970 con 0x7fd03c1060a0 2024-05-08T05:57:46.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.253+0000 7fd043000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1157f0 con 0x7fd03c10b6e0 2024-05-08T05:57:46.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.253+0000 7fd043000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c115af0 con 0x7fd03c1056e0 2024-05-08T05:57:46.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.254+0000 7fd03b400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd03c1060a0 0x7fd03c19c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.254+0000 7fd03be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd03c1056e0 0x7fd03c19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.254+0000 7fd03b400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd03c1060a0 0x7fd03c19c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58346/0 (socket says 172.21.15.112:58346) 2024-05-08T05:57:46.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.254+0000 7fd03b400700 1 -- 172.21.15.112:0/3545608000 learned_addr learned my addr 172.21.15.112:0/3545608000 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:46.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.254+0000 7fd041000700 1 --2- 172.21.15.112:0/3545608000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd03c10b6e0 0x7fd03c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.254+0000 7fd03be00700 1 -- 172.21.15.112:0/3545608000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd03c10b6e0 msgr2=0x7fd03c1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:46.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.254+0000 7fd03be00700 1 --2- 172.21.15.112:0/3545608000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd03c10b6e0 0x7fd03c1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.254+0000 7fd03be00700 1 -- 172.21.15.112:0/3545608000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd03c1060a0 msgr2=0x7fd03c19c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:46.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.254+0000 7fd03be00700 1 --2- 172.21.15.112:0/3545608000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd03c1060a0 0x7fd03c19c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.254+0000 7fd03be00700 1 -- 172.21.15.112:0/3545608000 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c1a0df0 con 0x7fd03c1056e0 2024-05-08T05:57:46.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.254+0000 7fd03b400700 1 --2- 172.21.15.112:0/3545608000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd03c1060a0 0x7fd03c19c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:57:46.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.255+0000 7fd03be00700 1 --2- 172.21.15.112:0/3545608000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd03c1056e0 0x7fd03c19bde0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00e9f0 tx=0x7fd02c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:46.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.255+0000 7fd038c00700 1 -- 172.21.15.112:0/3545608000 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c004220 con 0x7fd03c1056e0 2024-05-08T05:57:46.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.255+0000 7fd038c00700 1 -- 172.21.15.112:0/3545608000 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd02c0043c0 con 0x7fd03c1056e0 2024-05-08T05:57:46.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.255+0000 7fd043000700 1 -- 172.21.15.112:0/3545608000 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03c1a10e0 con 0x7fd03c1056e0 2024-05-08T05:57:46.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.255+0000 7fd038c00700 1 -- 172.21.15.112:0/3545608000 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c005020 con 0x7fd03c1056e0 2024-05-08T05:57:46.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.255+0000 7fd043000700 1 -- 172.21.15.112:0/3545608000 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03c1a8960 con 0x7fd03c1056e0 2024-05-08T05:57:46.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.259+0000 7fd038c00700 1 -- 172.21.15.112:0/3545608000 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd02c004750 con 0x7fd03c1056e0 2024-05-08T05:57:46.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.260+0000 7fd04da16b80 1 Processor -- start 2024-05-08T05:57:46.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.260+0000 7fd04da16b80 1 -- start start 2024-05-08T05:57:46.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.260+0000 7fd038c00700 1 --2- 172.21.15.112:0/3545608000 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd0200b2880 0x7fd0200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.260+0000 7fd04da16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e33bc0 0x55ef58e33fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.260+0000 7fd04da16b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e344e0 0x55ef58e2e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.260+0000 7fd04da16b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e2e8f0 0x55ef58e2ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.260+0000 7fd038c00700 1 -- 172.21.15.112:0/3545608000 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd02c0e21a0 con 0x7fd03c1056e0 2024-05-08T05:57:46.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.260+0000 7fd04da16b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ef58d80a20 con 0x55ef58e344e0 2024-05-08T05:57:46.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.260+0000 7fd04da16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ef58d1e760 con 0x55ef58e33bc0 2024-05-08T05:57:46.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.260+0000 7fd04da16b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55ef58d7c9e0 con 0x55ef58e2e8f0 2024-05-08T05:57:46.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.260+0000 7fd041000700 1 --2- 172.21.15.112:0/3545608000 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd0200b2880 0x7fd0200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.261+0000 7fd03be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e33bc0 0x55ef58e33fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.261+0000 7fd041000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e2e8f0 0x55ef58e2ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.261+0000 7fd041000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e2e8f0 0x55ef58e2ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:45336/0 (socket says 172.21.15.112:45336) 2024-05-08T05:57:46.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.261+0000 7fd041000700 1 -- 172.21.15.112:0/928485416 learned_addr learned my addr 172.21.15.112:0/928485416 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:46.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.261+0000 7fd03b400700 1 --2- 172.21.15.112:0/928485416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e344e0 0x55ef58e2e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.261+0000 7fd041000700 1 -- 172.21.15.112:0/928485416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e33bc0 msgr2=0x55ef58e33fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:46.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.261+0000 7fd041000700 1 --2- 172.21.15.112:0/928485416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e33bc0 0x55ef58e33fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.261+0000 7fd041000700 1 -- 172.21.15.112:0/928485416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e344e0 msgr2=0x55ef58e2e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:46.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.261+0000 7fd041000700 1 --2- 172.21.15.112:0/928485416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e344e0 0x55ef58e2e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.261+0000 7fd041000700 1 -- 172.21.15.112:0/928485416 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef58ccd8d0 con 0x55ef58e2e8f0 2024-05-08T05:57:46.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.261+0000 7fd03b400700 1 --2- 172.21.15.112:0/928485416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e344e0 0x55ef58e2e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:57:46.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.262+0000 7fd041000700 1 --2- 172.21.15.112:0/3545608000 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd0200b2880 0x7fd0200b4d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd034009610 tx=0x7fd034009340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:46.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.262+0000 7fd041000700 1 --2- 172.21.15.112:0/928485416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e2e8f0 0x55ef58e2ed70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd03400b0a0 tx=0x7fd034026000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e6aee4b82daebc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:46.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.262+0000 7fd031600700 1 -- 172.21.15.112:0/928485416 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0340074e0 con 0x55ef58e2e8f0 2024-05-08T05:57:46.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.262+0000 7fd031600700 1 -- 172.21.15.112:0/928485416 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd034007b30 con 0x55ef58e2e8f0 2024-05-08T05:57:46.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.262+0000 7fd031600700 1 -- 172.21.15.112:0/928485416 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03401bb00 con 0x55ef58e2e8f0 2024-05-08T05:57:46.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.263+0000 7fd04da16b80 1 -- 172.21.15.112:0/928485416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e2e8f0 msgr2=0x55ef58e2ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:46.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.263+0000 7fd04da16b80 1 --2- 172.21.15.112:0/928485416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e2e8f0 0x55ef58e2ed70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd03400b0a0 tx=0x7fd034026000 comp rx=0 tx=0).stop 2024-05-08T05:57:46.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.263+0000 7fd04da16b80 1 -- 172.21.15.112:0/928485416 shutdown_connections 2024-05-08T05:57:46.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.263+0000 7fd04da16b80 1 --2- 172.21.15.112:0/928485416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e2e8f0 0x55ef58e2ed70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.263+0000 7fd04da16b80 1 --2- 172.21.15.112:0/928485416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e33bc0 0x55ef58e33fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.263+0000 7fd04da16b80 1 --2- 172.21.15.112:0/928485416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e344e0 0x55ef58e2e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.263+0000 7fd04da16b80 1 -- 172.21.15.112:0/928485416 >> 172.21.15.112:0/928485416 conn(0x55ef58da66d0 msgr2=0x55ef58e28b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:46.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.263+0000 7fd04da16b80 1 -- 172.21.15.112:0/928485416 shutdown_connections 2024-05-08T05:57:46.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.263+0000 7fd04da16b80 1 -- 172.21.15.112:0/928485416 wait complete. 2024-05-08T05:57:46.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.263+0000 7fd04da16b80 1 -- start start 2024-05-08T05:57:46.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.264+0000 7fd04da16b80 1 Processor -- start 2024-05-08T05:57:46.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.264+0000 7fd04da16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e3a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.264+0000 7fd04da16b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e33bc0 0x55ef58e375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.264+0000 7fd04da16b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e344e0 0x55ef58e37b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.264+0000 7fd04da16b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ef58e3aec0 con 0x55ef58e33bc0 2024-05-08T05:57:46.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.264+0000 7fd04da16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ef58ce6b20 con 0x55ef58e2e8f0 2024-05-08T05:57:46.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.264+0000 7fd04da16b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55ef58e3b040 con 0x55ef58e344e0 2024-05-08T05:57:46.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.264+0000 7fd04da16b80 10 client.0 _ll_register_callbacks cb 0x55ef58d27c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:57:46.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.265+0000 7fd03b400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e33bc0 0x55ef58e375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.265+0000 7fd03be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e3a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.265+0000 7fd03be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e3a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:43056/0 (socket says 172.21.15.112:43056) 2024-05-08T05:57:46.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.265+0000 7fd03be00700 1 -- 172.21.15.112:0/2479192866 learned_addr learned my addr 172.21.15.112:0/2479192866 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:46.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.265+0000 7fd041000700 1 --2- 172.21.15.112:0/2479192866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e344e0 0x55ef58e37b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.265+0000 7fd03be00700 1 -- 172.21.15.112:0/2479192866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e344e0 msgr2=0x55ef58e37b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:46.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.265+0000 7fd03be00700 1 --2- 172.21.15.112:0/2479192866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e344e0 0x55ef58e37b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.265+0000 7fd03be00700 1 -- 172.21.15.112:0/2479192866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e33bc0 msgr2=0x55ef58e375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:46.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.265+0000 7fd03be00700 1 --2- 172.21.15.112:0/2479192866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e33bc0 0x55ef58e375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.265+0000 7fd03be00700 1 -- 172.21.15.112:0/2479192866 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef58d46420 con 0x55ef58e2e8f0 2024-05-08T05:57:46.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.265+0000 7fd041000700 1 --2- 172.21.15.112:0/2479192866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e344e0 0x55ef58e37b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:57:46.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.265+0000 7fd03be00700 1 --2- 172.21.15.112:0/2479192866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e3a200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd02c004620 tx=0x7fd02c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e680890e6770a88 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:46.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.265+0000 7fd026000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:57:46.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.266+0000 7fd026000700 1 -- 172.21.15.112:0/2479192866 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c01c040 con 0x55ef58e2e8f0 2024-05-08T05:57:46.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.266+0000 7fd026000700 1 -- 172.21.15.112:0/2479192866 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd02c0b2100 con 0x55ef58e2e8f0 2024-05-08T05:57:46.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.266+0000 7fd04da16b80 1 -- 172.21.15.112:0/2479192866 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd034007b30 con 0x55ef58e2e8f0 2024-05-08T05:57:46.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.266+0000 7fd026000700 1 -- 172.21.15.112:0/2479192866 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd02c0c8050 con 0x55ef58e2e8f0 2024-05-08T05:57:46.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.266+0000 7fd04da16b80 10 client.25412 fetch_fsmap learned FSMap version 26 2024-05-08T05:57:46.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.266+0000 7fd04da16b80 1 -- 172.21.15.112:0/2479192866 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef58e457d0 con 0x55ef58e2e8f0 2024-05-08T05:57:46.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.266+0000 7fd026000700 1 -- 172.21.15.112:0/2479192866 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd034007b30 con 0x55ef58e2e8f0 2024-05-08T05:57:46.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.267+0000 7fd026000700 1 -- 172.21.15.112:0/2479192866 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd02c06e020 con 0x55ef58e2e8f0 2024-05-08T05:57:46.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.267+0000 7fd04da16b80 10 client.25412 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T05:57:46.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.267+0000 7fd04da16b80 10 client.25412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:57:46.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.267+0000 7fd04da16b80 20 client.25412 populate_metadata read hostname 'smithi112' 2024-05-08T05:57:46.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.267+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2479192866 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55ef58e47e60 0x55ef58e4a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.267+0000 7fd04da16b80 4 client.25412 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:57:46.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.267+0000 7fd04da16b80 1 -- 172.21.15.112:0/2479192866 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef58dfb140 con 0x55ef58e47e60 2024-05-08T05:57:46.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.267+0000 7fd03b400700 1 --2- 172.21.15.112:0/2479192866 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55ef58e47e60 0x55ef58e4a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.268+0000 7fd03b400700 1 --2- 172.21.15.112:0/2479192866 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55ef58e47e60 0x55ef58e4a250 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2a77fd89adb6ef server_cookie=3cfdf069936e7e81 in_seq=0 out_seq=0 2024-05-08T05:57:46.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.268+0000 7fd026000700 10 client.25412 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:46.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.269+0000 7fd026000700 1 -- 172.21.15.112:0/2479192866 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef58dfb140 con 0x55ef58e47e60 2024-05-08T05:57:46.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.269+0000 7fd026000700 10 client.25412 handle_command_reply: tid=0 2024-05-08T05:57:46.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.269+0000 7fd04da16b80 1 client.25412 shutdown 2024-05-08T05:57:46.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.269+0000 7fd04da16b80 1 -- 172.21.15.112:0/2479192866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 msgr2=0x55ef58e3a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:46.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.269+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2479192866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e3a200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd02c004620 tx=0x7fd02c01f000 comp rx=0 tx=0).stop 2024-05-08T05:57:46.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.269+0000 7fd04da16b80 1 -- 172.21.15.112:0/2479192866 shutdown_connections 2024-05-08T05:57:46.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.269+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2479192866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e344e0 0x55ef58e37b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.269+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2479192866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e3a200 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.269+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2479192866 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55ef58e47e60 0x55ef58e4a250 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.269+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2479192866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e33bc0 0x55ef58e375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.269+0000 7fd026000700 0 client.25412 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:46.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.269+0000 7fd04da16b80 1 -- 172.21.15.112:0/2479192866 >> 172.21.15.112:0/2479192866 conn(0x55ef58da66d0 msgr2=0x55ef58e28190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:46.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.269+0000 7fd04da16b80 1 -- 172.21.15.112:0/2479192866 shutdown_connections 2024-05-08T05:57:46.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.269+0000 7fd04da16b80 1 -- 172.21.15.112:0/2479192866 wait complete. 2024-05-08T05:57:46.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.270+0000 7fd04da16b80 20 client.25412 trim_cache size 0 max 16384 2024-05-08T05:57:46.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.291+0000 7fd04da16b80 1 Processor -- start 2024-05-08T05:57:46.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.291+0000 7fd04da16b80 1 -- start start 2024-05-08T05:57:46.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.292+0000 7fd04da16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e2e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.292+0000 7fd04da16b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e33bc0 0x55ef58e43f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.292+0000 7fd04da16b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e344e0 0x55ef58e394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.292+0000 7fd04da16b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1b2620 con 0x55ef58e344e0 2024-05-08T05:57:46.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.292+0000 7fd04da16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1afeb0 con 0x55ef58e2e8f0 2024-05-08T05:57:46.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.292+0000 7fd04da16b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1add00 con 0x55ef58e33bc0 2024-05-08T05:57:46.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.292+0000 7fd041000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e344e0 0x55ef58e394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.292+0000 7fd03be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e2e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.292+0000 7fd041000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e344e0 0x55ef58e394e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58362/0 (socket says 172.21.15.112:58362) 2024-05-08T05:57:46.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.292+0000 7fd041000700 1 -- 172.21.15.112:0/2117202620 learned_addr learned my addr 172.21.15.112:0/2117202620 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:46.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.292+0000 7fd03b400700 1 --2- 172.21.15.112:0/2117202620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e33bc0 0x55ef58e43f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.293+0000 7fd041000700 1 -- 172.21.15.112:0/2117202620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e33bc0 msgr2=0x55ef58e43f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:46.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.293+0000 7fd041000700 1 --2- 172.21.15.112:0/2117202620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e33bc0 0x55ef58e43f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.293+0000 7fd041000700 1 -- 172.21.15.112:0/2117202620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 msgr2=0x55ef58e2e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:46.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.293+0000 7fd041000700 1 --2- 172.21.15.112:0/2117202620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e2e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.293+0000 7fd041000700 1 -- 172.21.15.112:0/2117202620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef58e4a8b0 con 0x55ef58e344e0 2024-05-08T05:57:46.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.293+0000 7fd03be00700 1 --2- 172.21.15.112:0/2117202620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e2e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:57:46.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.293+0000 7fd041000700 1 --2- 172.21.15.112:0/2117202620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e344e0 0x55ef58e394e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd03400bfa0 tx=0x7fd034009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f010b331f0b168 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:46.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.293+0000 7fd030c00700 1 -- 172.21.15.112:0/2117202620 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034032070 con 0x55ef58e344e0 2024-05-08T05:57:46.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.293+0000 7fd030c00700 1 -- 172.21.15.112:0/2117202620 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd034007950 con 0x55ef58e344e0 2024-05-08T05:57:46.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.293+0000 7fd030c00700 1 -- 172.21.15.112:0/2117202620 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03401ba00 con 0x55ef58e344e0 2024-05-08T05:57:46.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.294+0000 7fd04da16b80 1 -- 172.21.15.112:0/2117202620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e344e0 msgr2=0x55ef58e394e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:46.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.294+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2117202620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e344e0 0x55ef58e394e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd03400bfa0 tx=0x7fd034009d70 comp rx=0 tx=0).stop 2024-05-08T05:57:46.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.294+0000 7fd04da16b80 1 -- 172.21.15.112:0/2117202620 shutdown_connections 2024-05-08T05:57:46.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.294+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2117202620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e344e0 0x55ef58e394e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.294+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2117202620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e2e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.294+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2117202620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e33bc0 0x55ef58e43f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.294+0000 7fd04da16b80 1 -- 172.21.15.112:0/2117202620 >> 172.21.15.112:0/2117202620 conn(0x55ef58da66d0 msgr2=0x55ef58e34f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:46.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.294+0000 7fd04da16b80 1 -- 172.21.15.112:0/2117202620 shutdown_connections 2024-05-08T05:57:46.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.294+0000 7fd04da16b80 1 -- 172.21.15.112:0/2117202620 wait complete. 2024-05-08T05:57:46.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.295+0000 7fd04da16b80 1 -- start start 2024-05-08T05:57:46.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.295+0000 7fd04da16b80 1 Processor -- start 2024-05-08T05:57:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd04da16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd04da16b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e33bc0 0x55ef58e3d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd04da16b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e344e0 0x55ef58e3d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd04da16b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1ae370 con 0x55ef58e33bc0 2024-05-08T05:57:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd04da16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1af7e0 con 0x55ef58e2e8f0 2024-05-08T05:57:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd04da16b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1aea40 con 0x55ef58e344e0 2024-05-08T05:57:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd04da16b80 10 client.0 _ll_register_callbacks cb 0x55ef58d27c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:57:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd03b400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e33bc0 0x55ef58e3d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd041000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e344e0 0x55ef58e3d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd03b400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e33bc0 0x55ef58e3d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58378/0 (socket says 172.21.15.112:58378) 2024-05-08T05:57:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd03b400700 1 -- 172.21.15.112:0/2751388868 learned_addr learned my addr 172.21.15.112:0/2751388868 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd03be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd03be00700 1 -- 172.21.15.112:0/2751388868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e344e0 msgr2=0x55ef58e3d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd03be00700 1 --2- 172.21.15.112:0/2751388868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e344e0 0x55ef58e3d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd03be00700 1 -- 172.21.15.112:0/2751388868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e33bc0 msgr2=0x55ef58e3d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd03be00700 1 --2- 172.21.15.112:0/2751388868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e33bc0 0x55ef58e3d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd03be00700 1 -- 172.21.15.112:0/2751388868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef58e3ddd0 con 0x55ef58e2e8f0 2024-05-08T05:57:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd041000700 1 --2- 172.21.15.112:0/2751388868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e344e0 0x55ef58e3d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:57:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.296+0000 7fd03b400700 1 --2- 172.21.15.112:0/2751388868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e33bc0 0x55ef58e3d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:57:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.297+0000 7fd03be00700 1 --2- 172.21.15.112:0/2751388868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e405b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00c800 tx=0x7fd02c0dfcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=877f7632645658a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.297+0000 7fd026000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:57:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.297+0000 7fd026000700 1 -- 172.21.15.112:0/2751388868 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c01c070 con 0x55ef58e2e8f0 2024-05-08T05:57:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.297+0000 7fd026000700 1 -- 172.21.15.112:0/2751388868 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd02c0b2100 con 0x55ef58e2e8f0 2024-05-08T05:57:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.297+0000 7fd04da16b80 1 -- 172.21.15.112:0/2751388868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd034007950 con 0x55ef58e2e8f0 2024-05-08T05:57:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.297+0000 7fd026000700 1 -- 172.21.15.112:0/2751388868 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd02c0a9e30 con 0x55ef58e2e8f0 2024-05-08T05:57:46.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.297+0000 7fd04da16b80 10 client.25418 fetch_fsmap learned FSMap version 26 2024-05-08T05:57:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.297+0000 7fd04da16b80 1 -- 172.21.15.112:0/2751388868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef58eb3510 con 0x55ef58e2e8f0 2024-05-08T05:57:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.297+0000 7fd026000700 1 -- 172.21.15.112:0/2751388868 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd034007950 con 0x55ef58e2e8f0 2024-05-08T05:57:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.298+0000 7fd026000700 1 -- 172.21.15.112:0/2751388868 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd02c0a50a0 con 0x55ef58e2e8f0 2024-05-08T05:57:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.298+0000 7fd04da16b80 10 client.25418 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T05:57:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.298+0000 7fd04da16b80 10 client.25418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:57:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.298+0000 7fd04da16b80 20 client.25418 populate_metadata read hostname 'smithi112' 2024-05-08T05:57:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.298+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2751388868 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55ef58eb3b30 0x55ef58eb5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.298+0000 7fd04da16b80 4 client.25418 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:57:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.298+0000 7fd04da16b80 1 -- 172.21.15.112:0/2751388868 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef58d8e400 con 0x55ef58eb3b30 2024-05-08T05:57:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.298+0000 7fd03b400700 1 --2- 172.21.15.112:0/2751388868 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55ef58eb3b30 0x55ef58eb5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd03b400700 1 --2- 172.21.15.112:0/2751388868 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55ef58eb3b30 0x55ef58eb5f50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b166f6f4f82a22b server_cookie=bb5ba8688faef514 in_seq=0 out_seq=0 2024-05-08T05:57:46.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd026000700 10 client.25418 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:46.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd026000700 1 -- 172.21.15.112:0/2751388868 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ef58d8e400 con 0x55ef58eb3b30 2024-05-08T05:57:46.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd026000700 10 client.25418 handle_command_reply: tid=0 2024-05-08T05:57:46.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd04da16b80 1 client.25418 shutdown 2024-05-08T05:57:46.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd04da16b80 1 -- 172.21.15.112:0/2751388868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 msgr2=0x55ef58e405b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:46.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2751388868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e405b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00c800 tx=0x7fd02c0dfcb0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd04da16b80 1 -- 172.21.15.112:0/2751388868 shutdown_connections 2024-05-08T05:57:46.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2751388868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef58e344e0 0x55ef58e3d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2751388868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef58e2e8f0 0x55ef58e405b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2751388868 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55ef58eb3b30 0x55ef58eb5f50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd04da16b80 1 --2- 172.21.15.112:0/2751388868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ef58e33bc0 0x55ef58e3d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd04da16b80 1 -- 172.21.15.112:0/2751388868 >> 172.21.15.112:0/2751388868 conn(0x55ef58da66d0 msgr2=0x55ef58e32f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:46.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd026000700 0 client.25418 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:57:46.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd04da16b80 1 -- 172.21.15.112:0/2751388868 shutdown_connections 2024-05-08T05:57:46.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.299+0000 7fd04da16b80 1 -- 172.21.15.112:0/2751388868 wait complete. 2024-05-08T05:57:46.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.300+0000 7fd04da16b80 20 client.25418 trim_cache size 0 max 16384 2024-05-08T05:57:46.304 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:57:46.304 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-08T05:57:46.304 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-08T05:57:46.304 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:57:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.300+0000 7fd043000700 1 -- 172.21.15.112:0/3545608000 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd0200b2880 msgr2=0x7fd0200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.300+0000 7fd043000700 1 --2- 172.21.15.112:0/3545608000 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd0200b2880 0x7fd0200b4d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd034009610 tx=0x7fd034009340 comp rx=0 tx=0).stop 2024-05-08T05:57:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.300+0000 7fd043000700 1 -- 172.21.15.112:0/3545608000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd03c1056e0 msgr2=0x7fd03c19bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.300+0000 7fd043000700 1 --2- 172.21.15.112:0/3545608000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd03c1056e0 0x7fd03c19bde0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00e9f0 tx=0x7fd02c00ed00 comp rx=0 tx=0).stop 2024-05-08T05:57:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.300+0000 7fd043000700 1 -- 172.21.15.112:0/3545608000 shutdown_connections 2024-05-08T05:57:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.300+0000 7fd043000700 1 --2- 172.21.15.112:0/3545608000 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd0200b2880 0x7fd0200b4d40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.300+0000 7fd043000700 1 --2- 172.21.15.112:0/3545608000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd03c10b6e0 0x7fd03c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.300+0000 7fd043000700 1 --2- 172.21.15.112:0/3545608000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd03c1056e0 0x7fd03c19bde0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.300+0000 7fd043000700 1 --2- 172.21.15.112:0/3545608000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd03c1060a0 0x7fd03c19c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.300+0000 7fd043000700 1 -- 172.21.15.112:0/3545608000 >> 172.21.15.112:0/3545608000 conn(0x7fd03c1011d0 msgr2=0x7fd03c1016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.301+0000 7fd043000700 1 -- 172.21.15.112:0/3545608000 shutdown_connections 2024-05-08T05:57:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:46.301+0000 7fd043000700 1 -- 172.21.15.112:0/3545608000 wait complete. 2024-05-08T05:57:47.312 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:57:47.312 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:57:47.386 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:47 smithi112 ceph-mon[30003]: pgmap v515: 97 pgs: 97 active+clean; 51 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-05-08T05:57:47.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:47 smithi150 ceph-mon[34315]: pgmap v515: 97 pgs: 97 active+clean; 51 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-05-08T05:57:47.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:47 smithi162 ceph-mon[34800]: pgmap v515: 97 pgs: 97 active+clean; 51 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-05-08T05:57:47.756 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:57:48.325 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: executing libtool commands 2024-05-08T05:57:48.482 INFO:tasks.workunit.client.0.smithi112.stdout:Building include 2024-05-08T05:57:48.482 INFO:tasks.workunit.client.0.smithi112.stdout:Building lib 2024-05-08T05:57:48.482 INFO:tasks.workunit.client.0.smithi112.stdout:Building tests 2024-05-08T05:57:48.498 INFO:tasks.workunit.client.0.smithi112.stdout: [LTDEP] 2024-05-08T05:57:48.513 INFO:tasks.workunit.client.0.smithi112.stdout:Building btrfs 2024-05-08T05:57:48.513 INFO:tasks.workunit.client.0.smithi112.stdout:Building ceph 2024-05-08T05:57:48.513 INFO:tasks.workunit.client.0.smithi112.stdout:Building cifs 2024-05-08T05:57:48.528 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-05-08T05:57:48.529 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-05-08T05:57:48.560 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-05-08T05:57:48.577 INFO:tasks.workunit.client.0.smithi112.stdout:Building ext4 2024-05-08T05:57:48.584 INFO:tasks.workunit.client.0.smithi112.stdout:Building f2fs 2024-05-08T05:57:48.597 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-05-08T05:57:48.610 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-05-08T05:57:48.664 INFO:tasks.workunit.client.0.smithi112.stdout:Building generic 2024-05-08T05:57:48.744 INFO:tasks.workunit.client.0.smithi112.stdout:Building nfs 2024-05-08T05:57:48.758 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-05-08T05:57:48.796 INFO:tasks.workunit.client.0.smithi112.stdout:Building ocfs2 2024-05-08T05:57:48.807 INFO:tasks.workunit.client.0.smithi112.stdout:Building overlay 2024-05-08T05:57:48.809 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-05-08T05:57:48.816 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-05-08T05:57:48.840 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-05-08T05:57:48.854 INFO:tasks.workunit.client.0.smithi112.stdout:Building perf 2024-05-08T05:57:48.870 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-05-08T05:57:48.908 INFO:tasks.workunit.client.0.smithi112.stdout:Building selftest 2024-05-08T05:57:48.921 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-05-08T05:57:48.961 INFO:tasks.workunit.client.0.smithi112.stdout:Building shared 2024-05-08T05:57:48.975 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-05-08T05:57:48.990 INFO:tasks.workunit.client.0.smithi112.stdout:Building tmpfs 2024-05-08T05:57:49.003 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-05-08T05:57:49.032 INFO:tasks.workunit.client.0.smithi112.stdout:Building udf 2024-05-08T05:57:49.045 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-05-08T05:57:49.046 INFO:tasks.workunit.client.0.smithi112.stdout:Building xfs 2024-05-08T05:57:49.269 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-05-08T05:57:49.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:49 smithi150 ceph-mon[34315]: pgmap v516: 97 pgs: 97 active+clean; 51 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 752 KiB/s wr, 153 op/s 2024-05-08T05:57:49.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:49 smithi162 ceph-mon[34800]: pgmap v516: 97 pgs: 97 active+clean; 51 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 752 KiB/s wr, 153 op/s 2024-05-08T05:57:49.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:49 smithi112 ceph-mon[30003]: pgmap v516: 97 pgs: 97 active+clean; 51 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 752 KiB/s wr, 153 op/s 2024-05-08T05:57:49.793 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dataascii.lo 2024-05-08T05:57:49.794 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] databin.lo 2024-05-08T05:57:49.795 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] datapid.lo 2024-05-08T05:57:50.081 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] forker.lo 2024-05-08T05:57:50.081 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] file_lock.lo 2024-05-08T05:57:50.144 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] pattern.lo 2024-05-08T05:57:50.412 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] open_flags.lo 2024-05-08T05:57:50.422 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] random_range.lo 2024-05-08T05:57:50.422 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] string_to_tokens.lo 2024-05-08T05:57:50.564 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] str_to_bytes.lo 2024-05-08T05:57:50.677 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] tlibio.lo 2024-05-08T05:57:50.728 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] write_log.lo 2024-05-08T05:57:50.844 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] random.lo 2024-05-08T05:57:51.164 INFO:tasks.workunit.client.0.smithi112.stdout: [LD] 2024-05-08T05:57:51.373 INFO:tasks.workunit.client.0.smithi112.stdout:Building ltp 2024-05-08T05:57:51.373 INFO:tasks.workunit.client.0.smithi112.stdout:Building src 2024-05-08T05:57:51.373 INFO:tasks.workunit.client.0.smithi112.stdout:Building m4 2024-05-08T05:57:51.374 INFO:tasks.workunit.client.0.smithi112.stdout:Building common 2024-05-08T05:57:51.394 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-05-08T05:57:51.411 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:51 smithi112 ceph-mon[30003]: pgmap v517: 97 pgs: 97 active+clean; 51 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 603 KiB/s wr, 111 op/s 2024-05-08T05:57:51.411 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:51 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:57:51.415 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-05-08T05:57:51.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:51 smithi150 ceph-mon[34315]: pgmap v517: 97 pgs: 97 active+clean; 51 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 603 KiB/s wr, 111 op/s 2024-05-08T05:57:51.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:51 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:57:51.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:51 smithi162 ceph-mon[34800]: pgmap v517: 97 pgs: 97 active+clean; 51 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 603 KiB/s wr, 111 op/s 2024-05-08T05:57:51.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:51 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:57:51.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.577+0000 7fcf6bc00700 1 -- 172.21.15.112:0/4293621320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf64109600 msgr2=0x7fcf641099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:51.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.577+0000 7fcf6bc00700 1 --2- 172.21.15.112:0/4293621320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf64109600 0x7fcf641099e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009990 tx=0x7fcf58009ca0 comp rx=0 tx=0).stop 2024-05-08T05:57:51.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.577+0000 7fcf6bc00700 1 -- 172.21.15.112:0/4293621320 shutdown_connections 2024-05-08T05:57:51.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.577+0000 7fcf6bc00700 1 --2- 172.21.15.112:0/4293621320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf64103eb0 0x7fcf64110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:51.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.577+0000 7fcf6bc00700 1 --2- 172.21.15.112:0/4293621320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf64109600 0x7fcf641099e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:51.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.577+0000 7fcf6bc00700 1 --2- 172.21.15.112:0/4293621320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf641034f0 0x7fcf64103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:51.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.577+0000 7fcf6bc00700 1 -- 172.21.15.112:0/4293621320 >> 172.21.15.112:0/4293621320 conn(0x7fcf6407a650 msgr2=0x7fcf6407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:51.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.577+0000 7fcf6bc00700 1 -- 172.21.15.112:0/4293621320 shutdown_connections 2024-05-08T05:57:51.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.577+0000 7fcf6bc00700 1 -- 172.21.15.112:0/4293621320 wait complete. 2024-05-08T05:57:51.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.579+0000 7fcf6bc00700 1 Processor -- start 2024-05-08T05:57:51.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.579+0000 7fcf6bc00700 1 -- start start 2024-05-08T05:57:51.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.579+0000 7fcf6bc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf641034f0 0x7fcf640757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:51.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.579+0000 7fcf6bc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf64103eb0 0x7fcf64075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:51.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.579+0000 7fcf6bc00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf64109600 0x7fcf64076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:51.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.579+0000 7fcf6bc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcf641136a0 con 0x7fcf641034f0 2024-05-08T05:57:51.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.579+0000 7fcf6bc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcf64113520 con 0x7fcf64103eb0 2024-05-08T05:57:51.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.579+0000 7fcf6bc00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fcf64113820 con 0x7fcf64109600 2024-05-08T05:57:51.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.579+0000 7fcf63e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf64103eb0 0x7fcf64075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:51.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.580+0000 7fcf63e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf64103eb0 0x7fcf64075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:43100/0 (socket says 172.21.15.112:43100) 2024-05-08T05:57:51.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.580+0000 7fcf63e00700 1 -- 172.21.15.112:0/1846068383 learned_addr learned my addr 172.21.15.112:0/1846068383 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:51.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.580+0000 7fcf69200700 1 --2- 172.21.15.112:0/1846068383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf641034f0 0x7fcf640757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:51.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.580+0000 7fcf63e00700 1 -- 172.21.15.112:0/1846068383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf64109600 msgr2=0x7fcf64076220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:57:51.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.580+0000 7fcf63e00700 1 --2- 172.21.15.112:0/1846068383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf64109600 0x7fcf64076220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:51.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.580+0000 7fcf63e00700 1 -- 172.21.15.112:0/1846068383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf641034f0 msgr2=0x7fcf640757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:51.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.580+0000 7fcf63e00700 1 --2- 172.21.15.112:0/1846068383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf641034f0 0x7fcf640757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:51.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.580+0000 7fcf63e00700 1 -- 172.21.15.112:0/1846068383 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf641a8040 con 0x7fcf64103eb0 2024-05-08T05:57:51.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.580+0000 7fcf63e00700 1 --2- 172.21.15.112:0/1846068383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf64103eb0 0x7fcf64075ce0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000e9a0 tx=0x7fcf5000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:51.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.581+0000 7fcf61600700 1 -- 172.21.15.112:0/1846068383 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000cbd0 con 0x7fcf64103eb0 2024-05-08T05:57:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.581+0000 7fcf61600700 1 -- 172.21.15.112:0/1846068383 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf5000cd70 con 0x7fcf64103eb0 2024-05-08T05:57:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.581+0000 7fcf61600700 1 -- 172.21.15.112:0/1846068383 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50018680 con 0x7fcf64103eb0 2024-05-08T05:57:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.581+0000 7fcf6bc00700 1 -- 172.21.15.112:0/1846068383 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf641a8330 con 0x7fcf64103eb0 2024-05-08T05:57:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.581+0000 7fcf6bc00700 1 -- 172.21.15.112:0/1846068383 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf641a8820 con 0x7fcf64103eb0 2024-05-08T05:57:51.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.582+0000 7fcf6bc00700 1 -- 172.21.15.112:0/1846068383 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf6410ba50 con 0x7fcf64103eb0 2024-05-08T05:57:51.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.584+0000 7fcf61600700 1 -- 172.21.15.112:0/1846068383 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fcf50010430 con 0x7fcf64103eb0 2024-05-08T05:57:51.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.586+0000 7fcf61600700 1 --2- 172.21.15.112:0/1846068383 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcf4c0b2900 0x7fcf4c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:51.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.586+0000 7fcf61600700 1 -- 172.21.15.112:0/1846068383 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fcf500e2400 con 0x7fcf64103eb0 2024-05-08T05:57:51.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.586+0000 7fcf69200700 1 --2- 172.21.15.112:0/1846068383 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcf4c0b2900 0x7fcf4c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:51.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.587+0000 7fcf69200700 1 --2- 172.21.15.112:0/1846068383 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcf4c0b2900 0x7fcf4c0b4dc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcf58005b00 tx=0x7fcf58005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:51.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.590+0000 7fcf61600700 1 -- 172.21.15.112:0/1846068383 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf500aa090 con 0x7fcf64103eb0 2024-05-08T05:57:51.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.878+0000 7fcf6bc00700 1 -- 172.21.15.112:0/1846068383 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf6410b430 con 0x7fcf64103eb0 2024-05-08T05:57:51.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.879+0000 7fcf61600700 1 -- 172.21.15.112:0/1846068383 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11895 (secure 0 0 0) 0x7fcf500b2110 con 0x7fcf64103eb0 2024-05-08T05:57:51.878 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:57:51.878 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T05:57:49.405742+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24518,24511]},"id":1}]} 2024-05-08T05:57:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.882+0000 7fcf6bc00700 1 -- 172.21.15.112:0/1846068383 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcf4c0b2900 msgr2=0x7fcf4c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.882+0000 7fcf6bc00700 1 --2- 172.21.15.112:0/1846068383 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcf4c0b2900 0x7fcf4c0b4dc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcf58005b00 tx=0x7fcf58005a90 comp rx=0 tx=0).stop 2024-05-08T05:57:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.882+0000 7fcf6bc00700 1 -- 172.21.15.112:0/1846068383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf64103eb0 msgr2=0x7fcf64075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.882+0000 7fcf6bc00700 1 --2- 172.21.15.112:0/1846068383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf64103eb0 0x7fcf64075ce0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000e9a0 tx=0x7fcf5000ecb0 comp rx=0 tx=0).stop 2024-05-08T05:57:51.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.882+0000 7fcf6bc00700 1 -- 172.21.15.112:0/1846068383 shutdown_connections 2024-05-08T05:57:51.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.882+0000 7fcf6bc00700 1 --2- 172.21.15.112:0/1846068383 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcf4c0b2900 0x7fcf4c0b4dc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:51.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.882+0000 7fcf6bc00700 1 --2- 172.21.15.112:0/1846068383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf64109600 0x7fcf64076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:51.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.882+0000 7fcf6bc00700 1 --2- 172.21.15.112:0/1846068383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf641034f0 0x7fcf640757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:51.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.882+0000 7fcf6bc00700 1 --2- 172.21.15.112:0/1846068383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf64103eb0 0x7fcf64075ce0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:51.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.882+0000 7fcf6bc00700 1 -- 172.21.15.112:0/1846068383 >> 172.21.15.112:0/1846068383 conn(0x7fcf6407a650 msgr2=0x7fcf641020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:51.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.883+0000 7fcf6bc00700 1 -- 172.21.15.112:0/1846068383 shutdown_connections 2024-05-08T05:57:51.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:51.883+0000 7fcf6bc00700 1 -- 172.21.15.112:0/1846068383 wait complete. 2024-05-08T05:57:51.883 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 27 2024-05-08T05:57:51.895 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] doio 2024-05-08T05:57:51.896 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fsstress 2024-05-08T05:57:51.896 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fsx 2024-05-08T05:57:52.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:52 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1846068383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:57:52.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:52 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1846068383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:57:52.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:52 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1846068383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:57:52.810 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] growfiles 2024-05-08T05:57:52.833 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-08T05:57:52.926 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] iogen 2024-05-08T05:57:53.325 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:57:53.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:53 smithi150 ceph-mon[34315]: pgmap v518: 97 pgs: 97 active+clean; 51 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 111 KiB/s wr, 69 op/s 2024-05-08T05:57:53.467 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:53 smithi162 ceph-mon[34800]: pgmap v518: 97 pgs: 97 active+clean; 51 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 111 KiB/s wr, 69 op/s 2024-05-08T05:57:53.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:53 smithi112 ceph-mon[30003]: pgmap v518: 97 pgs: 97 active+clean; 51 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 111 KiB/s wr, 69 op/s 2024-05-08T05:57:53.584 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-stress 2024-05-08T05:57:55.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:55 smithi112 ceph-mon[30003]: pgmap v519: 97 pgs: 97 active+clean; 51 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 86 KiB/s wr, 72 op/s 2024-05-08T05:57:55.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:55 smithi150 ceph-mon[34315]: pgmap v519: 97 pgs: 97 active+clean; 51 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 86 KiB/s wr, 72 op/s 2024-05-08T05:57:55.563 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-05-08T05:57:55.563 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-05-08T05:57:55.564 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-05-08T05:57:55.564 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-05-08T05:57:55.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:55 smithi162 ceph-mon[34800]: pgmap v519: 97 pgs: 97 active+clean; 51 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 86 KiB/s wr, 72 op/s 2024-05-08T05:57:55.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.839+0000 7f8446c00700 1 -- 172.21.15.112:0/829865479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8440075ff0 msgr2=0x7f844010f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:55.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.839+0000 7f8446c00700 1 --2- 172.21.15.112:0/829865479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8440075ff0 0x7f844010f960 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f842c009990 tx=0x7f842c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.839+0000 7f8446c00700 1 -- 172.21.15.112:0/829865479 shutdown_connections 2024-05-08T05:57:55.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.839+0000 7f8446c00700 1 --2- 172.21.15.112:0/829865479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844010fea0 0x7f8440112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.839+0000 7f8446c00700 1 --2- 172.21.15.112:0/829865479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84400756d0 0x7f8440075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.839+0000 7f8446c00700 1 --2- 172.21.15.112:0/829865479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8440075ff0 0x7f844010f960 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.839+0000 7f8446c00700 1 -- 172.21.15.112:0/829865479 >> 172.21.15.112:0/829865479 conn(0x7f844007a600 msgr2=0x7f844007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:55.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.839+0000 7f8446c00700 1 -- 172.21.15.112:0/829865479 shutdown_connections 2024-05-08T05:57:55.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.839+0000 7f8446c00700 1 -- 172.21.15.112:0/829865479 wait complete. 2024-05-08T05:57:55.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.841+0000 7f8446c00700 1 Processor -- start 2024-05-08T05:57:55.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.841+0000 7f8446c00700 1 -- start start 2024-05-08T05:57:55.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.841+0000 7f8446c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84400756d0 0x7f844019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.841+0000 7f8446c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8440075ff0 0x7f844019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.841+0000 7f8446c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f844010fea0 0x7f84401a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.841+0000 7f8446c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84401158b0 con 0x7f844010fea0 2024-05-08T05:57:55.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.841+0000 7f8446c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8440115730 con 0x7f8440075ff0 2024-05-08T05:57:55.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.841+0000 7f8446c00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8440115a30 con 0x7f84400756d0 2024-05-08T05:57:55.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.842+0000 7f843f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8440075ff0 0x7f844019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.842+0000 7f843fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84400756d0 0x7f844019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.842+0000 7f8444c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f844010fea0 0x7f84401a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.842+0000 7f843f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8440075ff0 0x7f844019c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:51062/0 (socket says 172.21.15.112:51062) 2024-05-08T05:57:55.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.842+0000 7f843f400700 1 -- 172.21.15.112:0/4043121720 learned_addr learned my addr 172.21.15.112:0/4043121720 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:55.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.842+0000 7f843f400700 1 -- 172.21.15.112:0/4043121720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84400756d0 msgr2=0x7f844019bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:55.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.842+0000 7f843f400700 1 --2- 172.21.15.112:0/4043121720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84400756d0 0x7f844019bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.842+0000 7f843f400700 1 -- 172.21.15.112:0/4043121720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f844010fea0 msgr2=0x7f84401a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:55.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.842+0000 7f843f400700 1 --2- 172.21.15.112:0/4043121720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f844010fea0 0x7f84401a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.842+0000 7f843f400700 1 -- 172.21.15.112:0/4043121720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84401a0d70 con 0x7f8440075ff0 2024-05-08T05:57:55.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.842+0000 7f843fe00700 1 --2- 172.21.15.112:0/4043121720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84400756d0 0x7f844019bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:57:55.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.843+0000 7f843f400700 1 --2- 172.21.15.112:0/4043121720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8440075ff0 0x7f844019c2a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f842c000c00 tx=0x7f842c0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:55.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.843+0000 7f843cc00700 1 -- 172.21.15.112:0/4043121720 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c027070 con 0x7f8440075ff0 2024-05-08T05:57:55.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.843+0000 7f843cc00700 1 -- 172.21.15.112:0/4043121720 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842c022470 con 0x7f8440075ff0 2024-05-08T05:57:55.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.843+0000 7f8446c00700 1 -- 172.21.15.112:0/4043121720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84401a1000 con 0x7f8440075ff0 2024-05-08T05:57:55.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.843+0000 7f843cc00700 1 -- 172.21.15.112:0/4043121720 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c017650 con 0x7f8440075ff0 2024-05-08T05:57:55.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.843+0000 7f8446c00700 1 -- 172.21.15.112:0/4043121720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84401a88e0 con 0x7f8440075ff0 2024-05-08T05:57:55.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.848+0000 7f843cc00700 1 -- 172.21.15.112:0/4043121720 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f842c02a070 con 0x7f8440075ff0 2024-05-08T05:57:55.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.848+0000 7f84514b9b80 1 Processor -- start 2024-05-08T05:57:55.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.848+0000 7f84514b9b80 1 -- start start 2024-05-08T05:57:55.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.849+0000 7f84514b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec02fbc0 0x563eec02ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.849+0000 7f84514b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec0304e0 0x563eec02a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.849+0000 7f84514b9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 0x563eec02ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.849+0000 7f84514b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563eebf7ca20 con 0x563eec0304e0 2024-05-08T05:57:55.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.849+0000 7f84514b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563eebf1a760 con 0x563eec02fbc0 2024-05-08T05:57:55.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.849+0000 7f84514b9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563eebf789e0 con 0x563eec02a8f0 2024-05-08T05:57:55.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.849+0000 7f843f400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 0x563eec02ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.849+0000 7f843fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec0304e0 0x563eec02a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.849+0000 7f8444c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec02fbc0 0x563eec02ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.849+0000 7f8444c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec02fbc0 0x563eec02ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:51076/0 (socket says 172.21.15.112:51076) 2024-05-08T05:57:55.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.849+0000 7f8444c00700 1 -- 172.21.15.112:0/1682927610 learned_addr learned my addr 172.21.15.112:0/1682927610 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:55.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.849+0000 7f843fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec0304e0 0x563eec02a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38596/0 (socket says 172.21.15.112:38596) 2024-05-08T05:57:55.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.850+0000 7f843cc00700 1 --2- 172.21.15.112:0/4043121720 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f84240b2880 0x7f84240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.850+0000 7f843cc00700 1 -- 172.21.15.112:0/4043121720 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f842c0e2f70 con 0x7f8440075ff0 2024-05-08T05:57:55.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.850+0000 7f843f400700 1 -- 172.21.15.112:0/1682927610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec02fbc0 msgr2=0x563eec02ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:55.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.850+0000 7f843f400700 1 --2- 172.21.15.112:0/1682927610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec02fbc0 0x563eec02ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.850+0000 7f843f400700 1 -- 172.21.15.112:0/1682927610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec0304e0 msgr2=0x563eec02a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:55.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.850+0000 7f843f400700 1 --2- 172.21.15.112:0/1682927610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec0304e0 0x563eec02a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.850+0000 7f843f400700 1 -- 172.21.15.112:0/1682927610 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eebec98d0 con 0x563eec02a8f0 2024-05-08T05:57:55.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.850+0000 7f8444c00700 1 --2- 172.21.15.112:0/4043121720 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f84240b2880 0x7f84240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.850+0000 7f843f400700 1 --2- 172.21.15.112:0/1682927610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 0x563eec02ad70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f842c0a7800 tx=0x7f842c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc6e6e253a4c79f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:55.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.851+0000 7f8435600700 1 -- 172.21.15.112:0/1682927610 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c0a9ee0 con 0x563eec02a8f0 2024-05-08T05:57:55.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.851+0000 7f8435600700 1 -- 172.21.15.112:0/1682927610 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842c0aa0a0 con 0x563eec02a8f0 2024-05-08T05:57:55.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.851+0000 7f8435600700 1 -- 172.21.15.112:0/1682927610 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c0a9d00 con 0x563eec02a8f0 2024-05-08T05:57:55.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.852+0000 7f84514b9b80 1 -- 172.21.15.112:0/1682927610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 msgr2=0x563eec02ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:55.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.852+0000 7f8444c00700 1 --2- 172.21.15.112:0/4043121720 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f84240b2880 0x7f84240b4d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8438009550 tx=0x7f8438009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:55.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.852+0000 7f84514b9b80 1 --2- 172.21.15.112:0/1682927610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 0x563eec02ad70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f842c0a7800 tx=0x7f842c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.852+0000 7f84514b9b80 1 -- 172.21.15.112:0/1682927610 shutdown_connections 2024-05-08T05:57:55.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.852+0000 7f84514b9b80 1 --2- 172.21.15.112:0/1682927610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 0x563eec02ad70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.852+0000 7f84514b9b80 1 --2- 172.21.15.112:0/1682927610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec02fbc0 0x563eec02ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.852+0000 7f84514b9b80 1 --2- 172.21.15.112:0/1682927610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec0304e0 0x563eec02a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.852+0000 7f84514b9b80 1 -- 172.21.15.112:0/1682927610 >> 172.21.15.112:0/1682927610 conn(0x563eebfa26d0 msgr2=0x563eec024b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:55.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.852+0000 7f84514b9b80 1 -- 172.21.15.112:0/1682927610 shutdown_connections 2024-05-08T05:57:55.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.852+0000 7f84514b9b80 1 -- 172.21.15.112:0/1682927610 wait complete. 2024-05-08T05:57:55.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.853+0000 7f84514b9b80 1 -- start start 2024-05-08T05:57:55.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.853+0000 7f84514b9b80 1 Processor -- start 2024-05-08T05:57:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.853+0000 7f84514b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec02a8f0 0x563eec036200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.853+0000 7f84514b9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02fbc0 0x563eec0335a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.853+0000 7f84514b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec0304e0 0x563eec033ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.853+0000 7f84514b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563eec036ce0 con 0x563eec0304e0 2024-05-08T05:57:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.853+0000 7f84514b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563eebee2b20 con 0x563eec02a8f0 2024-05-08T05:57:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.853+0000 7f84514b9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563eec036e60 con 0x563eec02fbc0 2024-05-08T05:57:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.854+0000 7f84514b9b80 10 client.0 _ll_register_callbacks cb 0x563eebf23c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:57:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.854+0000 7f843fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec02a8f0 0x563eec036200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.854+0000 7f843fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec02a8f0 0x563eec036200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:51082/0 (socket says 172.21.15.112:51082) 2024-05-08T05:57:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.854+0000 7f843fe00700 1 -- 172.21.15.112:0/772487990 learned_addr learned my addr 172.21.15.112:0/772487990 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.854+0000 7f843f400700 1 --2- 172.21.15.112:0/772487990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02fbc0 0x563eec0335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.854+0000 7f8444c00700 1 --2- 172.21.15.112:0/772487990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec0304e0 0x563eec033ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.854+0000 7f843fe00700 1 -- 172.21.15.112:0/772487990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02fbc0 msgr2=0x563eec0335a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.854+0000 7f843fe00700 1 --2- 172.21.15.112:0/772487990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02fbc0 0x563eec0335a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.854+0000 7f843fe00700 1 -- 172.21.15.112:0/772487990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec0304e0 msgr2=0x563eec033ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.854+0000 7f843fe00700 1 --2- 172.21.15.112:0/772487990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec0304e0 0x563eec033ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.854+0000 7f843fe00700 1 -- 172.21.15.112:0/772487990 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eebf42420 con 0x563eec02a8f0 2024-05-08T05:57:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.855+0000 7f843f400700 1 --2- 172.21.15.112:0/772487990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02fbc0 0x563eec0335a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:57:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.855+0000 7f843fe00700 1 --2- 172.21.15.112:0/772487990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec02a8f0 0x563eec036200 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8430012d20 tx=0x7f843000f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=694eb2dbb647ba36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.855+0000 7f842a000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:57:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.855+0000 7f842a000700 1 -- 172.21.15.112:0/772487990 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843000fb80 con 0x563eec02a8f0 2024-05-08T05:57:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.855+0000 7f842a000700 1 -- 172.21.15.112:0/772487990 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f843000fd20 con 0x563eec02a8f0 2024-05-08T05:57:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.855+0000 7f84514b9b80 1 -- 172.21.15.112:0/772487990 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f842c0aa0a0 con 0x563eec02a8f0 2024-05-08T05:57:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.856+0000 7f842a000700 1 -- 172.21.15.112:0/772487990 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8430004050 con 0x563eec02a8f0 2024-05-08T05:57:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.856+0000 7f84514b9b80 10 client.25436 fetch_fsmap learned FSMap version 27 2024-05-08T05:57:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.856+0000 7f84514b9b80 1 -- 172.21.15.112:0/772487990 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563eec0414e0 con 0x563eec02a8f0 2024-05-08T05:57:55.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.856+0000 7f842a000700 1 -- 172.21.15.112:0/772487990 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842c0aa0a0 con 0x563eec02a8f0 2024-05-08T05:57:55.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.856+0000 7f842a000700 1 -- 172.21.15.112:0/772487990 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f8430010430 con 0x563eec02a8f0 2024-05-08T05:57:55.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.856+0000 7f84514b9b80 10 client.25436 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T05:57:55.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.856+0000 7f84514b9b80 10 client.25436 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T05:57:55.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.856+0000 7f84514b9b80 20 client.25436 populate_metadata read hostname 'smithi112' 2024-05-08T05:57:55.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.856+0000 7f84514b9b80 1 --2- 172.21.15.112:0/772487990 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x563eec043a70 0x563eec045ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.856+0000 7f84514b9b80 4 client.25436 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:57:55.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.856+0000 7f84514b9b80 1 -- 172.21.15.112:0/772487990 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563eebff7140 con 0x563eec043a70 2024-05-08T05:57:55.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.857+0000 7f843f400700 1 --2- 172.21.15.112:0/772487990 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x563eec043a70 0x563eec045ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.857+0000 7f843f400700 1 --2- 172.21.15.112:0/772487990 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x563eec043a70 0x563eec045ea0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c019ed70cc02898e server_cookie=4fe9c0e63748cd0d in_seq=0 out_seq=0 2024-05-08T05:57:55.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.857+0000 7f842a000700 10 client.25436 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:57:55.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.858+0000 7f842a000700 1 -- 172.21.15.112:0/772487990 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563eebff7140 con 0x563eec043a70 2024-05-08T05:57:55.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.858+0000 7f842a000700 10 client.25436 handle_command_reply: tid=0 2024-05-08T05:57:55.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.858+0000 7f84514b9b80 1 client.25436 shutdown 2024-05-08T05:57:55.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.858+0000 7f84514b9b80 1 -- 172.21.15.112:0/772487990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec02a8f0 msgr2=0x563eec036200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:55.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.858+0000 7f84514b9b80 1 --2- 172.21.15.112:0/772487990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec02a8f0 0x563eec036200 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8430012d20 tx=0x7f843000f6a0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.859+0000 7f84514b9b80 1 -- 172.21.15.112:0/772487990 shutdown_connections 2024-05-08T05:57:55.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.859+0000 7f84514b9b80 1 --2- 172.21.15.112:0/772487990 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x563eec043a70 0x563eec045ea0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.859+0000 7f84514b9b80 1 --2- 172.21.15.112:0/772487990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec0304e0 0x563eec033ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.859+0000 7f84514b9b80 1 --2- 172.21.15.112:0/772487990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec02a8f0 0x563eec036200 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.859+0000 7f842a000700 0 client.25436 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:57:55.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.859+0000 7f84514b9b80 1 --2- 172.21.15.112:0/772487990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02fbc0 0x563eec0335a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.859+0000 7f84514b9b80 1 -- 172.21.15.112:0/772487990 >> 172.21.15.112:0/772487990 conn(0x563eebfa26d0 msgr2=0x563eec024190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:55.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.859+0000 7f84514b9b80 1 -- 172.21.15.112:0/772487990 shutdown_connections 2024-05-08T05:57:55.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.859+0000 7f84514b9b80 1 -- 172.21.15.112:0/772487990 wait complete. 2024-05-08T05:57:55.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.859+0000 7f84514b9b80 20 client.25436 trim_cache size 0 max 16384 2024-05-08T05:57:55.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.875+0000 7f84514b9b80 1 Processor -- start 2024-05-08T05:57:55.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.875+0000 7f84514b9b80 1 -- start start 2024-05-08T05:57:55.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.876+0000 7f84514b9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 0x563eec03fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.876+0000 7f84514b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec02fbc0 0x563eec024910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.876+0000 7f84514b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec0304e0 0x563eec04b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.876+0000 7f84514b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84401b2610 con 0x563eec02fbc0 2024-05-08T05:57:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.876+0000 7f84514b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f84401afe70 con 0x563eec0304e0 2024-05-08T05:57:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.876+0000 7f84514b9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f84401adcc0 con 0x563eec02a8f0 2024-05-08T05:57:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.876+0000 7f843f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec02fbc0 0x563eec024910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.876+0000 7f843f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec02fbc0 0x563eec024910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38614/0 (socket says 172.21.15.112:38614) 2024-05-08T05:57:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.876+0000 7f843f400700 1 -- 172.21.15.112:0/2237917789 learned_addr learned my addr 172.21.15.112:0/2237917789 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.876+0000 7f843fe00700 1 --2- 172.21.15.112:0/2237917789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 0x563eec03fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.876+0000 7f8444c00700 1 --2- 172.21.15.112:0/2237917789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec0304e0 0x563eec04b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.877+0000 7f843fe00700 1 -- 172.21.15.112:0/2237917789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec0304e0 msgr2=0x563eec04b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:55.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.877+0000 7f843fe00700 1 --2- 172.21.15.112:0/2237917789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec0304e0 0x563eec04b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.877+0000 7f843fe00700 1 -- 172.21.15.112:0/2237917789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec02fbc0 msgr2=0x563eec024910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:55.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.877+0000 7f843fe00700 1 --2- 172.21.15.112:0/2237917789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec02fbc0 0x563eec024910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.877+0000 7f843fe00700 1 -- 172.21.15.112:0/2237917789 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eec025800 con 0x563eec02a8f0 2024-05-08T05:57:55.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.877+0000 7f843f400700 1 --2- 172.21.15.112:0/2237917789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec02fbc0 0x563eec024910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:57:55.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.877+0000 7f843fe00700 1 --2- 172.21.15.112:0/2237917789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 0x563eec03fca0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8430012cf0 tx=0x7f84300123e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da0fa439099213a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:55.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.877+0000 7f8434c00700 1 -- 172.21.15.112:0/2237917789 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430010430 con 0x563eec02a8f0 2024-05-08T05:57:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.877+0000 7f8434c00700 1 -- 172.21.15.112:0/2237917789 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84300105d0 con 0x563eec02a8f0 2024-05-08T05:57:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.878+0000 7f8434c00700 1 -- 172.21.15.112:0/2237917789 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430005740 con 0x563eec02a8f0 2024-05-08T05:57:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.878+0000 7f84514b9b80 1 -- 172.21.15.112:0/2237917789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 msgr2=0x563eec03fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.878+0000 7f84514b9b80 1 --2- 172.21.15.112:0/2237917789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 0x563eec03fca0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8430012cf0 tx=0x7f84300123e0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.878+0000 7f84514b9b80 1 -- 172.21.15.112:0/2237917789 shutdown_connections 2024-05-08T05:57:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.878+0000 7f84514b9b80 1 --2- 172.21.15.112:0/2237917789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec0304e0 0x563eec04b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.878+0000 7f84514b9b80 1 --2- 172.21.15.112:0/2237917789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 0x563eec03fca0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.878+0000 7f84514b9b80 1 --2- 172.21.15.112:0/2237917789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec02fbc0 0x563eec024910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.878+0000 7f84514b9b80 1 -- 172.21.15.112:0/2237917789 >> 172.21.15.112:0/2237917789 conn(0x563eebfa26d0 msgr2=0x563eec02efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.878+0000 7f84514b9b80 1 -- 172.21.15.112:0/2237917789 shutdown_connections 2024-05-08T05:57:55.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.879+0000 7f84514b9b80 1 -- 172.21.15.112:0/2237917789 wait complete. 2024-05-08T05:57:55.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.879+0000 7f84514b9b80 1 -- start start 2024-05-08T05:57:55.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.879+0000 7f84514b9b80 1 Processor -- start 2024-05-08T05:57:55.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f84514b9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 0x563eec03c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f84514b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec02fbc0 0x563eec039900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f84514b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec0304e0 0x563eec039e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f84514b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84401ae330 con 0x563eec02fbc0 2024-05-08T05:57:55.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f84514b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f84401af7a0 con 0x563eec0304e0 2024-05-08T05:57:55.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f84514b9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f84401aea00 con 0x563eec02a8f0 2024-05-08T05:57:55.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f84514b9b80 10 client.0 _ll_register_callbacks cb 0x563eebf23c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:57:55.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f843f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec02fbc0 0x563eec039900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f843f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec02fbc0 0x563eec039900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38620/0 (socket says 172.21.15.112:38620) 2024-05-08T05:57:55.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f843f400700 1 -- 172.21.15.112:0/3059878318 learned_addr learned my addr 172.21.15.112:0/3059878318 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:57:55.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f843fe00700 1 --2- 172.21.15.112:0/3059878318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 0x563eec03c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f8444c00700 1 --2- 172.21.15.112:0/3059878318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec0304e0 0x563eec039e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f843f400700 1 -- 172.21.15.112:0/3059878318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 msgr2=0x563eec03c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:55.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f843f400700 1 --2- 172.21.15.112:0/3059878318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 0x563eec03c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f843f400700 1 -- 172.21.15.112:0/3059878318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec0304e0 msgr2=0x563eec039e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:57:55.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f843f400700 1 --2- 172.21.15.112:0/3059878318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec0304e0 0x563eec039e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f843f400700 1 -- 172.21.15.112:0/3059878318 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eec03a630 con 0x563eec02fbc0 2024-05-08T05:57:55.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.880+0000 7f8444c00700 1 --2- 172.21.15.112:0/3059878318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec0304e0 0x563eec039e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:57:55.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.881+0000 7f843f400700 1 --2- 172.21.15.112:0/3059878318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec02fbc0 0x563eec039900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f842c006010 tx=0x7f842c022cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb689bec140a99d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:55.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.881+0000 7f842a000700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:57:55.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.881+0000 7f842a000700 1 -- 172.21.15.112:0/3059878318 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c0aa0a0 con 0x563eec02fbc0 2024-05-08T05:57:55.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.881+0000 7f842a000700 1 -- 172.21.15.112:0/3059878318 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f842c0aa7f0 con 0x563eec02fbc0 2024-05-08T05:57:55.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.881+0000 7f84514b9b80 1 -- 172.21.15.112:0/3059878318 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84300105d0 con 0x563eec02fbc0 2024-05-08T05:57:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.881+0000 7f842a000700 1 -- 172.21.15.112:0/3059878318 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f842c0aac90 con 0x563eec02fbc0 2024-05-08T05:57:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.881+0000 7f84514b9b80 10 client.15324 fetch_fsmap learned FSMap version 27 2024-05-08T05:57:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.881+0000 7f84514b9b80 1 -- 172.21.15.112:0/3059878318 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563eec0ae3f0 con 0x563eec02fbc0 2024-05-08T05:57:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.882+0000 7f842a000700 1 -- 172.21.15.112:0/3059878318 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84300105d0 con 0x563eec02fbc0 2024-05-08T05:57:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.882+0000 7f842a000700 1 -- 172.21.15.112:0/3059878318 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f842c0cb300 con 0x563eec02fbc0 2024-05-08T05:57:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.882+0000 7f84514b9b80 10 client.15324 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T05:57:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.882+0000 7f84514b9b80 10 client.15324 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T05:57:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.882+0000 7f84514b9b80 20 client.15324 populate_metadata read hostname 'smithi112' 2024-05-08T05:57:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.882+0000 7f84514b9b80 1 --2- 172.21.15.112:0/3059878318 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x563eec0ae7d0 0x563eec0b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.882+0000 7f84514b9b80 4 client.15324 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:57:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.882+0000 7f84514b9b80 1 -- 172.21.15.112:0/3059878318 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563eebf8a400 con 0x563eec0ae7d0 2024-05-08T05:57:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.882+0000 7f843fe00700 1 --2- 172.21.15.112:0/3059878318 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x563eec0ae7d0 0x563eec0b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.883+0000 7f843fe00700 1 --2- 172.21.15.112:0/3059878318 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x563eec0ae7d0 0x563eec0b0bf0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f4835cf3e6980b1 server_cookie=e4950651d55344a9 in_seq=0 out_seq=0 2024-05-08T05:57:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.883+0000 7f842a000700 10 client.15324 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:57:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.883+0000 7f842a000700 1 -- 172.21.15.112:0/3059878318 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563eebf8a400 con 0x563eec0ae7d0 2024-05-08T05:57:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.883+0000 7f842a000700 10 client.15324 handle_command_reply: tid=0 2024-05-08T05:57:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.883+0000 7f84514b9b80 1 client.15324 shutdown 2024-05-08T05:57:55.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.883+0000 7f84514b9b80 1 -- 172.21.15.112:0/3059878318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec02fbc0 msgr2=0x563eec039900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:55.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.883+0000 7f84514b9b80 1 --2- 172.21.15.112:0/3059878318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec02fbc0 0x563eec039900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f842c006010 tx=0x7f842c022cc0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.883+0000 7f84514b9b80 1 -- 172.21.15.112:0/3059878318 shutdown_connections 2024-05-08T05:57:55.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.883+0000 7f84514b9b80 1 --2- 172.21.15.112:0/3059878318 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x563eec0ae7d0 0x563eec0b0bf0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.883+0000 7f84514b9b80 1 --2- 172.21.15.112:0/3059878318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eec0304e0 0x563eec039e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.883+0000 7f84514b9b80 1 --2- 172.21.15.112:0/3059878318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563eec02a8f0 0x563eec03c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.883+0000 7f84514b9b80 1 --2- 172.21.15.112:0/3059878318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563eec02fbc0 0x563eec039900 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.883+0000 7f84514b9b80 1 -- 172.21.15.112:0/3059878318 >> 172.21.15.112:0/3059878318 conn(0x563eebfa26d0 msgr2=0x563eec0292c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:55.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.883+0000 7f842a000700 0 client.15324 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:57:55.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.884+0000 7f84514b9b80 1 -- 172.21.15.112:0/3059878318 shutdown_connections 2024-05-08T05:57:55.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.884+0000 7f84514b9b80 1 -- 172.21.15.112:0/3059878318 wait complete. 2024-05-08T05:57:55.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.884+0000 7f84514b9b80 20 client.15324 trim_cache size 0 max 16384 2024-05-08T05:57:55.888 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:57:55.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.884+0000 7f8446c00700 1 -- 172.21.15.112:0/4043121720 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f84240b2880 msgr2=0x7f84240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:55.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.884+0000 7f8446c00700 1 --2- 172.21.15.112:0/4043121720 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f84240b2880 0x7f84240b4d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8438009550 tx=0x7f8438009040 comp rx=0 tx=0).stop 2024-05-08T05:57:55.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.884+0000 7f8446c00700 1 -- 172.21.15.112:0/4043121720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8440075ff0 msgr2=0x7f844019c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:55.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.884+0000 7f8446c00700 1 --2- 172.21.15.112:0/4043121720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8440075ff0 0x7f844019c2a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f842c000c00 tx=0x7f842c0057c0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.885+0000 7f8446c00700 1 -- 172.21.15.112:0/4043121720 shutdown_connections 2024-05-08T05:57:55.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.885+0000 7f8446c00700 1 --2- 172.21.15.112:0/4043121720 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f84240b2880 0x7f84240b4d40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.885+0000 7f8446c00700 1 --2- 172.21.15.112:0/4043121720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f844010fea0 0x7f84401a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.885+0000 7f8446c00700 1 --2- 172.21.15.112:0/4043121720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84400756d0 0x7f844019bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.885+0000 7f8446c00700 1 --2- 172.21.15.112:0/4043121720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8440075ff0 0x7f844019c2a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:55.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.885+0000 7f8446c00700 1 -- 172.21.15.112:0/4043121720 >> 172.21.15.112:0/4043121720 conn(0x7f844007a600 msgr2=0x7f84401006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:55.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.885+0000 7f8446c00700 1 -- 172.21.15.112:0/4043121720 shutdown_connections 2024-05-08T05:57:55.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:57:55.885+0000 7f8446c00700 1 -- 172.21.15.112:0/4043121720 wait complete. 2024-05-08T05:57:56.088 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dirstress 2024-05-08T05:57:56.088 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fill 2024-05-08T05:57:56.088 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fill2 2024-05-08T05:57:56.089 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] getpagesize 2024-05-08T05:57:56.424 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] holes 2024-05-08T05:57:56.448 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] lstat64 2024-05-08T05:57:56.558 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] nametest 2024-05-08T05:57:56.569 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] permname 2024-05-08T05:57:56.797 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] randholes 2024-05-08T05:57:56.806 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] runas 2024-05-08T05:57:56.923 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] truncfile 2024-05-08T05:57:57.030 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] usemem 2024-05-08T05:57:57.035 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-08T05:57:57.154 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] mmapcat 2024-05-08T05:57:57.284 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] append_reader 2024-05-08T05:57:57.295 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] append_writer 2024-05-08T05:57:57.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:57 smithi112 ceph-mon[30003]: pgmap v520: 97 pgs: 97 active+clean; 51 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 107 KiB/s wr, 98 op/s 2024-05-08T05:57:57.381 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dirperf 2024-05-08T05:57:57.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:57 smithi150 ceph-mon[34315]: pgmap v520: 97 pgs: 97 active+clean; 51 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 107 KiB/s wr, 98 op/s 2024-05-08T05:57:57.511 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] metaperf 2024-05-08T05:57:57.548 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:57:57.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:57 smithi162 ceph-mon[34800]: pgmap v520: 97 pgs: 97 active+clean; 51 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 107 KiB/s wr, 98 op/s 2024-05-08T05:57:57.614 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] devzero 2024-05-08T05:57:57.625 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] feature 2024-05-08T05:57:57.824 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] alloc 2024-05-08T05:57:57.966 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fault 2024-05-08T05:57:58.016 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fstest 2024-05-08T05:57:58.028 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_access_root 2024-05-08T05:57:58.276 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] godown 2024-05-08T05:57:58.277 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] resvtest 2024-05-08T05:57:58.345 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] writemod 2024-05-08T05:57:58.450 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] writev_on_pagefault 2024-05-08T05:57:58.616 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] makeextents 2024-05-08T05:57:58.618 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] itrash 2024-05-08T05:57:58.622 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] rename 2024-05-08T05:57:58.778 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] multi_open_unlink 2024-05-08T05:57:58.904 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] unwritten_sync 2024-05-08T05:57:58.910 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] genhashnames 2024-05-08T05:57:58.946 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_holes 2024-05-08T05:57:59.114 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_writev 2024-05-08T05:57:59.232 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_truncate_cmtime 2024-05-08T05:57:59.277 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dirhash_collide 2024-05-08T05:57:59.287 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_rename_overwrite 2024-05-08T05:57:59.414 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] holetest 2024-05-08T05:57:59.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:57:59 smithi150 ceph-mon[34315]: pgmap v521: 97 pgs: 97 active+clean; 51 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 90 KiB/s wr, 85 op/s 2024-05-08T05:57:59.552 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_truncate_self 2024-05-08T05:57:59.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:57:59 smithi162 ceph-mon[34800]: pgmap v521: 97 pgs: 97 active+clean; 51 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 90 KiB/s wr, 85 op/s 2024-05-08T05:57:59.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:57:59 smithi112 ceph-mon[30003]: pgmap v521: 97 pgs: 97 active+clean; 51 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 90 KiB/s wr, 85 op/s 2024-05-08T05:57:59.597 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] af_unix 2024-05-08T05:57:59.655 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_stale_pmd 2024-05-08T05:57:59.827 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_cow_race 2024-05-08T05:57:59.845 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_fallocate 2024-05-08T05:57:59.919 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fsync-err 2024-05-08T05:57:59.968 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_write_ro 2024-05-08T05:58:00.139 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_ext4_dax_journal_corruption 2024-05-08T05:58:00.180 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_ext4_dax_inline_corruption 2024-05-08T05:58:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.226+0000 7fa365000700 1 -- 172.21.15.112:0/1308896555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa360108850 msgr2=0x7fa3601103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.226+0000 7fa365000700 1 --2- 172.21.15.112:0/1308896555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa360108850 0x7fa3601103d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa358009fc0 tx=0x7fa35800b330 comp rx=0 tx=0).stop 2024-05-08T05:58:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.226+0000 7fa365000700 1 -- 172.21.15.112:0/1308896555 shutdown_connections 2024-05-08T05:58:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.226+0000 7fa365000700 1 --2- 172.21.15.112:0/1308896555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa360108850 0x7fa3601103d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.226+0000 7fa365000700 1 --2- 172.21.15.112:0/1308896555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa360103750 0x7fa360103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.226+0000 7fa365000700 1 --2- 172.21.15.112:0/1308896555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360104100 0x7fa360108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.226+0000 7fa365000700 1 -- 172.21.15.112:0/1308896555 >> 172.21.15.112:0/1308896555 conn(0x7fa36007a600 msgr2=0x7fa36007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.226+0000 7fa365000700 1 -- 172.21.15.112:0/1308896555 shutdown_connections 2024-05-08T05:58:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.226+0000 7fa365000700 1 -- 172.21.15.112:0/1308896555 wait complete. 2024-05-08T05:58:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.227+0000 7fa365000700 1 Processor -- start 2024-05-08T05:58:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.228+0000 7fa365000700 1 -- start start 2024-05-08T05:58:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.228+0000 7fa365000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360103750 0x7fa36019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.228+0000 7fa365000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa360104100 0x7fa36019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.228+0000 7fa365000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa360108850 0x7fa360199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.228+0000 7fa365000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa360113670 con 0x7fa360103750 2024-05-08T05:58:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.228+0000 7fa365000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3601134f0 con 0x7fa360104100 2024-05-08T05:58:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.228+0000 7fa365000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa3601137f0 con 0x7fa360108850 2024-05-08T05:58:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.229+0000 7fa35de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360103750 0x7fa36019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.229+0000 7fa35de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360103750 0x7fa36019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38644/0 (socket says 172.21.15.112:38644) 2024-05-08T05:58:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.229+0000 7fa35de00700 1 -- 172.21.15.112:0/1394862337 learned_addr learned my addr 172.21.15.112:0/1394862337 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.229+0000 7fa35d400700 1 --2- 172.21.15.112:0/1394862337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa360104100 0x7fa36019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.229+0000 7fa35e800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa360108850 0x7fa360199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.230+0000 7fa35e800700 1 -- 172.21.15.112:0/1394862337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa360104100 msgr2=0x7fa36019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.230+0000 7fa35e800700 1 --2- 172.21.15.112:0/1394862337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa360104100 0x7fa36019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.230+0000 7fa35e800700 1 -- 172.21.15.112:0/1394862337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360103750 msgr2=0x7fa36019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:00.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.230+0000 7fa35e800700 1 --2- 172.21.15.112:0/1394862337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360103750 0x7fa36019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.230+0000 7fa35e800700 1 -- 172.21.15.112:0/1394862337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36019a010 con 0x7fa360108850 2024-05-08T05:58:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.230+0000 7fa35de00700 1 --2- 172.21.15.112:0/1394862337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360103750 0x7fa36019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:58:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.230+0000 7fa35e800700 1 --2- 172.21.15.112:0/1394862337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa360108850 0x7fa360199830 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa358015000 tx=0x7fa35800cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.232+0000 7fa356a00700 1 -- 172.21.15.112:0/1394862337 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358021070 con 0x7fa360108850 2024-05-08T05:58:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.232+0000 7fa356a00700 1 -- 172.21.15.112:0/1394862337 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa35801d450 con 0x7fa360108850 2024-05-08T05:58:00.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.232+0000 7fa365000700 1 -- 172.21.15.112:0/1394862337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36019a2a0 con 0x7fa360108850 2024-05-08T05:58:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.232+0000 7fa356a00700 1 -- 172.21.15.112:0/1394862337 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3580052a0 con 0x7fa360108850 2024-05-08T05:58:00.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.232+0000 7fa365000700 1 -- 172.21.15.112:0/1394862337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3601a64c0 con 0x7fa360108850 2024-05-08T05:58:00.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.238+0000 7fa36fa72b80 1 Processor -- start 2024-05-08T05:58:00.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.239+0000 7fa356a00700 1 -- 172.21.15.112:0/1394862337 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa358024070 con 0x7fa360108850 2024-05-08T05:58:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.239+0000 7fa36fa72b80 1 -- start start 2024-05-08T05:58:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.240+0000 7fa36fa72b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b25bc0 0x561df1b25fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.240+0000 7fa36fa72b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 0x561df1b202d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.240+0000 7fa36fa72b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 0x561df1b20d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.240+0000 7fa36fa72b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561df1a72a20 con 0x561df1b264e0 2024-05-08T05:58:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.240+0000 7fa36fa72b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561df1a10760 con 0x561df1b208f0 2024-05-08T05:58:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.240+0000 7fa36fa72b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561df1a6e9e0 con 0x561df1b25bc0 2024-05-08T05:58:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.240+0000 7fa35d400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b25bc0 0x561df1b25fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.241+0000 7fa35d400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b25bc0 0x561df1b25fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:37420/0 (socket says 172.21.15.112:37420) 2024-05-08T05:58:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.241+0000 7fa35d400700 1 -- 172.21.15.112:0/892030196 learned_addr learned my addr 172.21.15.112:0/892030196 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.241+0000 7fa35de00700 1 --2- 172.21.15.112:0/892030196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 0x561df1b202d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.241+0000 7fa35d400700 1 --2- 172.21.15.112:0/892030196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 0x561df1b20d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.241+0000 7fa35d400700 1 -- 172.21.15.112:0/892030196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 msgr2=0x561df1b20d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:00.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.241+0000 7fa35d400700 1 --2- 172.21.15.112:0/892030196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 0x561df1b20d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.241+0000 7fa35d400700 1 -- 172.21.15.112:0/892030196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 msgr2=0x561df1b202d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:00.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.241+0000 7fa35d400700 1 --2- 172.21.15.112:0/892030196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 0x561df1b202d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.241+0000 7fa35d400700 1 -- 172.21.15.112:0/892030196 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561df19bf8d0 con 0x561df1b25bc0 2024-05-08T05:58:00.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.241+0000 7fa35de00700 1 --2- 172.21.15.112:0/892030196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 0x561df1b202d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:58:00.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.242+0000 7fa35d400700 1 --2- 172.21.15.112:0/892030196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b25bc0 0x561df1b25fa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x561df1b212b0 tx=0x7fa34c00a7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cf994f63760a922 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:00.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.242+0000 7fa356a00700 1 --2- 172.21.15.112:0/1394862337 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3440b2980 0x7fa3440b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.242+0000 7fa356a00700 1 -- 172.21.15.112:0/1394862337 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fa3580e30d0 con 0x7fa360108850 2024-05-08T05:58:00.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.243+0000 7fa34b400700 1 -- 172.21.15.112:0/892030196 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c011e20 con 0x561df1b25bc0 2024-05-08T05:58:00.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.243+0000 7fa34b400700 1 -- 172.21.15.112:0/892030196 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa34c004830 con 0x561df1b25bc0 2024-05-08T05:58:00.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.243+0000 7fa34b400700 1 -- 172.21.15.112:0/892030196 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c005540 con 0x561df1b25bc0 2024-05-08T05:58:00.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.244+0000 7fa35de00700 1 --2- 172.21.15.112:0/1394862337 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3440b2980 0x7fa3440b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.244+0000 7fa36fa72b80 1 -- 172.21.15.112:0/892030196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b25bc0 msgr2=0x561df1b25fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:00.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.244+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/892030196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b25bc0 0x561df1b25fa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x561df1b212b0 tx=0x7fa34c00a7d0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.244+0000 7fa36fa72b80 1 -- 172.21.15.112:0/892030196 shutdown_connections 2024-05-08T05:58:00.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.244+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/892030196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 0x561df1b20d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.244+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/892030196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b25bc0 0x561df1b25fa0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.244+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/892030196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 0x561df1b202d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.244+0000 7fa36fa72b80 1 -- 172.21.15.112:0/892030196 >> 172.21.15.112:0/892030196 conn(0x561df1a986d0 msgr2=0x561df1b1ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:00.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.244+0000 7fa35de00700 1 --2- 172.21.15.112:0/1394862337 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3440b2980 0x7fa3440b4e40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa350009fd0 tx=0x7fa350009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:00.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.245+0000 7fa36fa72b80 1 -- 172.21.15.112:0/892030196 shutdown_connections 2024-05-08T05:58:00.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.245+0000 7fa36fa72b80 1 -- 172.21.15.112:0/892030196 wait complete. 2024-05-08T05:58:00.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.245+0000 7fa36fa72b80 1 -- start start 2024-05-08T05:58:00.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.246+0000 7fa36fa72b80 1 Processor -- start 2024-05-08T05:58:00.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.247+0000 7fa36fa72b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b208f0 0x561df1b2c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.247+0000 7fa36fa72b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b25bc0 0x561df1b29620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.247+0000 7fa36fa72b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 0x561df1b29b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.247+0000 7fa36fa72b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561df1b3df30 con 0x561df1b264e0 2024-05-08T05:58:00.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.247+0000 7fa36fa72b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561df19d8b20 con 0x561df1b25bc0 2024-05-08T05:58:00.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.247+0000 7fa36fa72b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561df1b3e0b0 con 0x561df1b208f0 2024-05-08T05:58:00.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.247+0000 7fa36fa72b80 10 client.0 _ll_register_callbacks cb 0x561df1a19c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:58:00.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.247+0000 7fa35d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b25bc0 0x561df1b29620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.247+0000 7fa35d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b25bc0 0x561df1b29620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:51132/0 (socket says 172.21.15.112:51132) 2024-05-08T05:58:00.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.247+0000 7fa35d400700 1 -- 172.21.15.112:0/788215385 learned_addr learned my addr 172.21.15.112:0/788215385 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:00.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.247+0000 7fa35de00700 1 --2- 172.21.15.112:0/788215385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b208f0 0x561df1b2c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.248+0000 7fa35de00700 1 -- 172.21.15.112:0/788215385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b25bc0 msgr2=0x561df1b29620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:00.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.248+0000 7fa35de00700 1 --2- 172.21.15.112:0/788215385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b25bc0 0x561df1b29620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.248+0000 7fa35de00700 1 -- 172.21.15.112:0/788215385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 msgr2=0x561df1b29b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:00.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.248+0000 7fa35e800700 1 --2- 172.21.15.112:0/788215385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 0x561df1b29b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.248+0000 7fa35de00700 1 --2- 172.21.15.112:0/788215385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 0x561df1b29b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.248+0000 7fa35de00700 1 -- 172.21.15.112:0/788215385 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561df1a38420 con 0x561df1b208f0 2024-05-08T05:58:00.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.248+0000 7fa35e800700 1 --2- 172.21.15.112:0/788215385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 0x561df1b29b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:58:00.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.248+0000 7fa35de00700 1 --2- 172.21.15.112:0/788215385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b208f0 0x561df1b2c280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa350005950 tx=0x7fa3500047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d7a436f75dae2cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:00.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.248+0000 7fa33fe00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:58:00.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.249+0000 7fa33fe00700 1 -- 172.21.15.112:0/788215385 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350004830 con 0x561df1b208f0 2024-05-08T05:58:00.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.249+0000 7fa33fe00700 1 -- 172.21.15.112:0/788215385 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fa3500049d0 con 0x561df1b208f0 2024-05-08T05:58:00.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.249+0000 7fa36fa72b80 1 -- 172.21.15.112:0/788215385 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa34c004830 con 0x561df1b208f0 2024-05-08T05:58:00.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.249+0000 7fa33fe00700 1 -- 172.21.15.112:0/788215385 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa350004bd0 con 0x561df1b208f0 2024-05-08T05:58:00.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.250+0000 7fa36fa72b80 10 client.25477 fetch_fsmap learned FSMap version 27 2024-05-08T05:58:00.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.250+0000 7fa36fa72b80 1 -- 172.21.15.112:0/788215385 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561df1b48880 con 0x561df1b208f0 2024-05-08T05:58:00.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.250+0000 7fa33fe00700 1 -- 172.21.15.112:0/788215385 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa34c004830 con 0x561df1b208f0 2024-05-08T05:58:00.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.251+0000 7fa33fe00700 1 -- 172.21.15.112:0/788215385 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7fa350027070 con 0x561df1b208f0 2024-05-08T05:58:00.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.251+0000 7fa36fa72b80 10 client.25477 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T05:58:00.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.251+0000 7fa36fa72b80 10 client.25477 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T05:58:00.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.251+0000 7fa36fa72b80 20 client.25477 populate_metadata read hostname 'smithi112' 2024-05-08T05:58:00.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.251+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/788215385 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561df1b4ae10 0x561df1b4d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.251+0000 7fa36fa72b80 4 client.25477 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:58:00.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.251+0000 7fa36fa72b80 1 -- 172.21.15.112:0/788215385 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561df1aed140 con 0x561df1b4ae10 2024-05-08T05:58:00.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.252+0000 7fa35d400700 1 --2- 172.21.15.112:0/788215385 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561df1b4ae10 0x561df1b4d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.253+0000 7fa35d400700 1 --2- 172.21.15.112:0/788215385 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561df1b4ae10 0x561df1b4d250 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad23ba275498c4b5 server_cookie=678d8c2edda80367 in_seq=0 out_seq=0 2024-05-08T05:58:00.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.254+0000 7fa33fe00700 10 client.25477 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T05:58:00.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.254+0000 7fa33fe00700 1 -- 172.21.15.112:0/788215385 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561df1aed140 con 0x561df1b4ae10 2024-05-08T05:58:00.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.254+0000 7fa33fe00700 10 client.25477 handle_command_reply: tid=0 2024-05-08T05:58:00.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.255+0000 7fa36fa72b80 1 client.25477 shutdown 2024-05-08T05:58:00.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.255+0000 7fa36fa72b80 1 -- 172.21.15.112:0/788215385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b208f0 msgr2=0x561df1b2c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:00.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.255+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/788215385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b208f0 0x561df1b2c280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa350005950 tx=0x7fa3500047c0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.255+0000 7fa36fa72b80 1 -- 172.21.15.112:0/788215385 shutdown_connections 2024-05-08T05:58:00.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.255+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/788215385 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561df1b4ae10 0x561df1b4d250 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.255+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/788215385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 0x561df1b29b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.255+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/788215385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b208f0 0x561df1b2c280 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.255+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/788215385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b25bc0 0x561df1b29620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.255+0000 7fa36fa72b80 1 -- 172.21.15.112:0/788215385 >> 172.21.15.112:0/788215385 conn(0x561df1a986d0 msgr2=0x561df1b1a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:00.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.255+0000 7fa33fe00700 0 client.25477 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T05:58:00.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.256+0000 7fa36fa72b80 1 -- 172.21.15.112:0/788215385 shutdown_connections 2024-05-08T05:58:00.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.256+0000 7fa36fa72b80 1 -- 172.21.15.112:0/788215385 wait complete. 2024-05-08T05:58:00.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.256+0000 7fa36fa72b80 20 client.25477 trim_cache size 0 max 16384 2024-05-08T05:58:00.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.284+0000 7fa36fa72b80 1 Processor -- start 2024-05-08T05:58:00.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.284+0000 7fa36fa72b80 1 -- start start 2024-05-08T05:58:00.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.284+0000 7fa36fa72b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 0x561df1b202d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.285+0000 7fa36fa72b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b25bc0 0x561df1b46f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.285+0000 7fa36fa72b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b264e0 0x561df1b27020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.285+0000 7fa36fa72b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3601a79d0 con 0x561df1b25bc0 2024-05-08T05:58:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.285+0000 7fa36fa72b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa360074270 con 0x561df1b208f0 2024-05-08T05:58:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.285+0000 7fa36fa72b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa3600720c0 con 0x561df1b264e0 2024-05-08T05:58:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.285+0000 7fa35d400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b25bc0 0x561df1b46f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.285+0000 7fa35d400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b25bc0 0x561df1b46f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38668/0 (socket says 172.21.15.112:38668) 2024-05-08T05:58:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.285+0000 7fa35d400700 1 -- 172.21.15.112:0/2928164073 learned_addr learned my addr 172.21.15.112:0/2928164073 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.285+0000 7fa35e800700 1 --2- 172.21.15.112:0/2928164073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b264e0 0x561df1b27020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.286+0000 7fa35d400700 1 -- 172.21.15.112:0/2928164073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b264e0 msgr2=0x561df1b27020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:00.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.286+0000 7fa35d400700 1 --2- 172.21.15.112:0/2928164073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b264e0 0x561df1b27020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.286+0000 7fa35d400700 1 -- 172.21.15.112:0/2928164073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 msgr2=0x561df1b202d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:58:00.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.286+0000 7fa35d400700 1 --2- 172.21.15.112:0/2928164073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 0x561df1b202d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.286+0000 7fa35d400700 1 -- 172.21.15.112:0/2928164073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561df1b2a560 con 0x561df1b25bc0 2024-05-08T05:58:00.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.286+0000 7fa35e800700 1 --2- 172.21.15.112:0/2928164073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b264e0 0x561df1b27020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:58:00.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.286+0000 7fa35d400700 1 --2- 172.21.15.112:0/2928164073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b25bc0 0x561df1b46f80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005130 tx=0x7fa34c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35609350594d80d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:00.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.286+0000 7fa34aa00700 1 -- 172.21.15.112:0/2928164073 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0051d0 con 0x561df1b25bc0 2024-05-08T05:58:00.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.287+0000 7fa34aa00700 1 -- 172.21.15.112:0/2928164073 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa34c02b430 con 0x561df1b25bc0 2024-05-08T05:58:00.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.287+0000 7fa34aa00700 1 -- 172.21.15.112:0/2928164073 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c003970 con 0x561df1b25bc0 2024-05-08T05:58:00.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.288+0000 7fa36fa72b80 1 -- 172.21.15.112:0/2928164073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b25bc0 msgr2=0x561df1b46f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:00.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.288+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/2928164073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b25bc0 0x561df1b46f80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005130 tx=0x7fa34c013040 comp rx=0 tx=0).stop 2024-05-08T05:58:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.288+0000 7fa36fa72b80 1 -- 172.21.15.112:0/2928164073 shutdown_connections 2024-05-08T05:58:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.288+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/2928164073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b264e0 0x561df1b27020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.288+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/2928164073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 0x561df1b202d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.288+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/2928164073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b25bc0 0x561df1b46f80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.288+0000 7fa36fa72b80 1 -- 172.21.15.112:0/2928164073 >> 172.21.15.112:0/2928164073 conn(0x561df1a986d0 msgr2=0x561df1b26910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.288+0000 7fa36fa72b80 1 -- 172.21.15.112:0/2928164073 shutdown_connections 2024-05-08T05:58:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.288+0000 7fa36fa72b80 1 -- 172.21.15.112:0/2928164073 wait complete. 2024-05-08T05:58:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.288+0000 7fa36fa72b80 1 -- start start 2024-05-08T05:58:00.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.289+0000 7fa36fa72b80 1 Processor -- start 2024-05-08T05:58:00.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.289+0000 7fa36fa72b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 0x561df1b53120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.289+0000 7fa36fa72b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b25bc0 0x561df1b43300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.289+0000 7fa36fa72b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 0x561df1b43860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.289+0000 7fa36fa72b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa360072730 con 0x561df1b264e0 2024-05-08T05:58:00.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.289+0000 7fa36fa72b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa360073ba0 con 0x561df1b208f0 2024-05-08T05:58:00.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.289+0000 7fa36fa72b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa360072e00 con 0x561df1b25bc0 2024-05-08T05:58:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.289+0000 7fa36fa72b80 10 client.0 _ll_register_callbacks cb 0x561df1a19c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:58:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.289+0000 7fa35de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 0x561df1b53120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.289+0000 7fa35de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 0x561df1b53120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:51150/0 (socket says 172.21.15.112:51150) 2024-05-08T05:58:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.289+0000 7fa35de00700 1 -- 172.21.15.112:0/613007510 learned_addr learned my addr 172.21.15.112:0/613007510 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.289+0000 7fa35d400700 1 --2- 172.21.15.112:0/613007510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b25bc0 0x561df1b43300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.290+0000 7fa35e800700 1 --2- 172.21.15.112:0/613007510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 0x561df1b43860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.290+0000 7fa35de00700 1 -- 172.21.15.112:0/613007510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b25bc0 msgr2=0x561df1b43300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.290+0000 7fa35de00700 1 --2- 172.21.15.112:0/613007510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b25bc0 0x561df1b43300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.290+0000 7fa35de00700 1 -- 172.21.15.112:0/613007510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 msgr2=0x561df1b43860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.290+0000 7fa35de00700 1 --2- 172.21.15.112:0/613007510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 0x561df1b43860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.290+0000 7fa35de00700 1 -- 172.21.15.112:0/613007510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561df1b51c80 con 0x561df1b208f0 2024-05-08T05:58:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.290+0000 7fa35e800700 1 --2- 172.21.15.112:0/613007510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 0x561df1b43860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:58:00.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.290+0000 7fa35d400700 1 --2- 172.21.15.112:0/613007510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b25bc0 0x561df1b43300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:58:00.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.290+0000 7fa35de00700 1 --2- 172.21.15.112:0/613007510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 0x561df1b53120 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa350005170 tx=0x7fa3500057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5ce302fd045e9ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:00.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.290+0000 7fa33fe00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:58:00.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.291+0000 7fa33fe00700 1 -- 172.21.15.112:0/613007510 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350004830 con 0x561df1b208f0 2024-05-08T05:58:00.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.291+0000 7fa33fe00700 1 -- 172.21.15.112:0/613007510 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fa3500049d0 con 0x561df1b208f0 2024-05-08T05:58:00.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.291+0000 7fa36fa72b80 1 -- 172.21.15.112:0/613007510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa34c02b430 con 0x561df1b208f0 2024-05-08T05:58:00.293 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_ofd_locks 2024-05-08T05:58:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.291+0000 7fa33fe00700 1 -- 172.21.15.112:0/613007510 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa35001f630 con 0x561df1b208f0 2024-05-08T05:58:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.291+0000 7fa36fa72b80 10 client.25454 fetch_fsmap learned FSMap version 27 2024-05-08T05:58:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.291+0000 7fa36fa72b80 1 -- 172.21.15.112:0/613007510 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561df1bb4b10 con 0x561df1b208f0 2024-05-08T05:58:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.292+0000 7fa33fe00700 1 -- 172.21.15.112:0/613007510 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa34c02b430 con 0x561df1b208f0 2024-05-08T05:58:00.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.292+0000 7fa33fe00700 1 -- 172.21.15.112:0/613007510 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7fa350027040 con 0x561df1b208f0 2024-05-08T05:58:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.292+0000 7fa36fa72b80 10 client.25454 fetch_fsmap finished waiting for FSMap version 27 2024-05-08T05:58:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.292+0000 7fa36fa72b80 10 client.25454 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T05:58:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.292+0000 7fa36fa72b80 20 client.25454 populate_metadata read hostname 'smithi112' 2024-05-08T05:58:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.293+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/613007510 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561df1bb5190 0x561df1bb75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.293+0000 7fa36fa72b80 4 client.25454 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:58:00.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.293+0000 7fa36fa72b80 1 -- 172.21.15.112:0/613007510 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561df1a80400 con 0x561df1bb5190 2024-05-08T05:58:00.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.295+0000 7fa35d400700 1 --2- 172.21.15.112:0/613007510 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561df1bb5190 0x561df1bb75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:00.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.296+0000 7fa35d400700 1 --2- 172.21.15.112:0/613007510 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561df1bb5190 0x561df1bb75b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a7f7aa5e49dc969 server_cookie=867282ddf613c3a0 in_seq=0 out_seq=0 2024-05-08T05:58:00.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.296+0000 7fa33fe00700 10 client.25454 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T05:58:00.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.296+0000 7fa33fe00700 1 -- 172.21.15.112:0/613007510 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561df1a80400 con 0x561df1bb5190 2024-05-08T05:58:00.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.296+0000 7fa33fe00700 10 client.25454 handle_command_reply: tid=0 2024-05-08T05:58:00.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.296+0000 7fa36fa72b80 1 client.25454 shutdown 2024-05-08T05:58:00.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.296+0000 7fa36fa72b80 1 -- 172.21.15.112:0/613007510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 msgr2=0x561df1b53120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:00.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.296+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/613007510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 0x561df1b53120 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa350005170 tx=0x7fa3500057a0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.297+0000 7fa36fa72b80 1 -- 172.21.15.112:0/613007510 shutdown_connections 2024-05-08T05:58:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.297+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/613007510 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x561df1bb5190 0x561df1bb75b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.297+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/613007510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561df1b264e0 0x561df1b43860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.297+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/613007510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561df1b208f0 0x561df1b53120 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.297+0000 7fa36fa72b80 1 --2- 172.21.15.112:0/613007510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561df1b25bc0 0x561df1b43300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.297+0000 7fa36fa72b80 1 -- 172.21.15.112:0/613007510 >> 172.21.15.112:0/613007510 conn(0x561df1a986d0 msgr2=0x561df1b287a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.297+0000 7fa33fe00700 0 client.25454 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T05:58:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.298+0000 7fa36fa72b80 1 -- 172.21.15.112:0/613007510 shutdown_connections 2024-05-08T05:58:00.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.298+0000 7fa36fa72b80 1 -- 172.21.15.112:0/613007510 wait complete. 2024-05-08T05:58:00.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.298+0000 7fa36fa72b80 20 client.25454 trim_cache size 0 max 16384 2024-05-08T05:58:00.299 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:58:00.301 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_collision 2024-05-08T05:58:00.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.299+0000 7fa365000700 1 -- 172.21.15.112:0/1394862337 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3440b2980 msgr2=0x7fa3440b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:00.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.299+0000 7fa365000700 1 --2- 172.21.15.112:0/1394862337 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3440b2980 0x7fa3440b4e40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa350009fd0 tx=0x7fa350009310 comp rx=0 tx=0).stop 2024-05-08T05:58:00.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.299+0000 7fa365000700 1 -- 172.21.15.112:0/1394862337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa360108850 msgr2=0x7fa360199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:00.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.299+0000 7fa365000700 1 --2- 172.21.15.112:0/1394862337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa360108850 0x7fa360199830 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa358015000 tx=0x7fa35800cca0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.299+0000 7fa365000700 1 -- 172.21.15.112:0/1394862337 shutdown_connections 2024-05-08T05:58:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.299+0000 7fa365000700 1 --2- 172.21.15.112:0/1394862337 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa3440b2980 0x7fa3440b4e40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.299+0000 7fa365000700 1 --2- 172.21.15.112:0/1394862337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa360108850 0x7fa360199830 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.299+0000 7fa365000700 1 --2- 172.21.15.112:0/1394862337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360103750 0x7fa36019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.299+0000 7fa365000700 1 --2- 172.21.15.112:0/1394862337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa360104100 0x7fa36019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.299+0000 7fa365000700 1 -- 172.21.15.112:0/1394862337 >> 172.21.15.112:0/1394862337 conn(0x7fa36007a600 msgr2=0x7fa3601014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.299+0000 7fa365000700 1 -- 172.21.15.112:0/1394862337 shutdown_connections 2024-05-08T05:58:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:00.300+0000 7fa365000700 1 -- 172.21.15.112:0/1394862337 wait complete. 2024-05-08T05:58:00.509 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] mmap-write-concurrent 2024-05-08T05:58:00.523 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_get_file_time 2024-05-08T05:58:00.651 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_create_short_dirs 2024-05-08T05:58:00.714 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_create_long_dirs 2024-05-08T05:58:00.826 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_enospc 2024-05-08T05:58:00.839 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_writev_overlap 2024-05-08T05:58:01.013 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] checkpoint_journal 2024-05-08T05:58:01.056 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] mmap-rw-fault 2024-05-08T05:58:01.176 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] allocstale 2024-05-08T05:58:01.230 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_cow_memory_failure 2024-05-08T05:58:01.367 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fake-dump-rootino 2024-05-08T05:58:01.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:01 smithi150 ceph-mon[34315]: pgmap v522: 97 pgs: 97 active+clean; 53 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 270 KiB/s wr, 96 op/s 2024-05-08T05:58:01.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:01 smithi112 ceph-mon[30003]: pgmap v522: 97 pgs: 97 active+clean; 53 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 270 KiB/s wr, 96 op/s 2024-05-08T05:58:01.514 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dio-buf-fault 2024-05-08T05:58:01.524 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-08T05:58:01.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:01 smithi162 ceph-mon[34800]: pgmap v522: 97 pgs: 97 active+clean; 53 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 270 KiB/s wr, 96 op/s 2024-05-08T05:58:01.599 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] rewinddir-test 2024-05-08T05:58:01.615 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] readdir-while-renames 2024-05-08T05:58:01.692 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] xfsctl 2024-05-08T05:58:01.875 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] bstat 2024-05-08T05:58:01.939 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mtab 2024-05-08T05:58:01.953 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] getdevicesize 2024-05-08T05:58:02.054 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:58:02.087 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] preallo_rw_pattern_reader 2024-05-08T05:58:02.266 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] preallo_rw_pattern_writer 2024-05-08T05:58:02.285 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] ftrunc 2024-05-08T05:58:02.301 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] trunc 2024-05-08T05:58:02.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:02 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:58:02.457 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:02 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:58:02.467 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fs_perms 2024-05-08T05:58:02.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:02 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:58:02.585 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] testx 2024-05-08T05:58:02.600 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] looptest 2024-05-08T05:58:02.655 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] locktest 2024-05-08T05:58:02.813 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] unwritten_mmap 2024-05-08T05:58:02.868 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] bulkstat_unlink_test 2024-05-08T05:58:02.989 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] deduperace 2024-05-08T05:58:03.149 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] bulkstat_unlink_test_modified 2024-05-08T05:58:03.172 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_dir_offset 2024-05-08T05:58:03.216 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_futimens 2024-05-08T05:58:03.378 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_immutable 2024-05-08T05:58:03.477 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] stale_handle 2024-05-08T05:58:03.505 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] pwrite_mmap_blocked 2024-05-08T05:58:03.520 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_dir_offset2 2024-05-08T05:58:03.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:03 smithi112 ceph-mon[30003]: pgmap v523: 97 pgs: 97 active+clean; 54 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 285 KiB/s wr, 95 op/s 2024-05-08T05:58:03.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:03 smithi162 ceph-mon[34800]: pgmap v523: 97 pgs: 97 active+clean; 54 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 285 KiB/s wr, 95 op/s 2024-05-08T05:58:03.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:03 smithi150 ceph-mon[34315]: pgmap v523: 97 pgs: 97 active+clean; 54 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 285 KiB/s wr, 95 op/s 2024-05-08T05:58:03.805 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] seek_sanity_test 2024-05-08T05:58:03.847 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] seek_copy_test 2024-05-08T05:58:03.872 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_readdir_1 2024-05-08T05:58:04.165 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_readdir_2 2024-05-08T05:58:04.202 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fsync-tester 2024-05-08T05:58:04.374 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] nsexec 2024-05-08T05:58:04.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.494+0000 7f2449000700 1 -- 172.21.15.112:0/2357769841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2444075ff0 msgr2=0x7f2444114130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:04.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.494+0000 7f2449000700 1 --2- 172.21.15.112:0/2357769841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2444075ff0 0x7f2444114130 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2430009990 tx=0x7f2430009ca0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.494+0000 7f2449000700 1 -- 172.21.15.112:0/2357769841 shutdown_connections 2024-05-08T05:58:04.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.494+0000 7f2449000700 1 --2- 172.21.15.112:0/2357769841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2444114670 0x7f2444116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.494+0000 7f2449000700 1 --2- 172.21.15.112:0/2357769841 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440756d0 0x7f2444075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.494+0000 7f2449000700 1 --2- 172.21.15.112:0/2357769841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2444075ff0 0x7f2444114130 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.494+0000 7f2449000700 1 -- 172.21.15.112:0/2357769841 >> 172.21.15.112:0/2357769841 conn(0x7f244407a600 msgr2=0x7f244407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:04.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.495+0000 7f2449000700 1 -- 172.21.15.112:0/2357769841 shutdown_connections 2024-05-08T05:58:04.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.496+0000 7f2449000700 1 -- 172.21.15.112:0/2357769841 wait complete. 2024-05-08T05:58:04.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.497+0000 7f2449000700 1 Processor -- start 2024-05-08T05:58:04.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.497+0000 7f2449000700 1 -- start start 2024-05-08T05:58:04.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.497+0000 7f2449000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440756d0 0x7f24441a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.497+0000 7f2449000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2444075ff0 0x7f24441a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.497+0000 7f2449000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2444114670 0x7f24441a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.498+0000 7f2449000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2444119cc0 con 0x7f2444114670 2024-05-08T05:58:04.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.498+0000 7f2449000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2444119b40 con 0x7f2444075ff0 2024-05-08T05:58:04.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.498+0000 7f2449000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2444119e40 con 0x7f24440756d0 2024-05-08T05:58:04.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.498+0000 7f2441e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440756d0 0x7f24441a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:04.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.498+0000 7f2441e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440756d0 0x7f24441a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:36138/0 (socket says 172.21.15.112:36138) 2024-05-08T05:58:04.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.498+0000 7f2441e00700 1 -- 172.21.15.112:0/1700941822 learned_addr learned my addr 172.21.15.112:0/1700941822 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:04.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.498+0000 7f2441400700 1 --2- 172.21.15.112:0/1700941822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2444075ff0 0x7f24441a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:04.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.499+0000 7f2441e00700 1 -- 172.21.15.112:0/1700941822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2444075ff0 msgr2=0x7f24441a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:04.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.499+0000 7f2441e00700 1 --2- 172.21.15.112:0/1700941822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2444075ff0 0x7f24441a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.499+0000 7f2441e00700 1 -- 172.21.15.112:0/1700941822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2444114670 msgr2=0x7f24441a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:04.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.499+0000 7f2441e00700 1 --2- 172.21.15.112:0/1700941822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2444114670 0x7f24441a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.499+0000 7f2441e00700 1 -- 172.21.15.112:0/1700941822 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24441a5200 con 0x7f24440756d0 2024-05-08T05:58:04.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.499+0000 7f2441e00700 1 --2- 172.21.15.112:0/1700941822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440756d0 0x7f24441a01f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f243400e9a0 tx=0x7f243400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.500+0000 7f243aa00700 1 -- 172.21.15.112:0/1700941822 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243400cbd0 con 0x7f24440756d0 2024-05-08T05:58:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.500+0000 7f243aa00700 1 -- 172.21.15.112:0/1700941822 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f243400cd70 con 0x7f24440756d0 2024-05-08T05:58:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.500+0000 7f2449000700 1 -- 172.21.15.112:0/1700941822 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24441a54f0 con 0x7f24440756d0 2024-05-08T05:58:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.500+0000 7f243aa00700 1 -- 172.21.15.112:0/1700941822 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434018680 con 0x7f24440756d0 2024-05-08T05:58:04.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.500+0000 7f2449000700 1 -- 172.21.15.112:0/1700941822 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24441acd70 con 0x7f24440756d0 2024-05-08T05:58:04.512 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] cloner 2024-05-08T05:58:04.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.508+0000 7f2453a05b80 1 Processor -- start 2024-05-08T05:58:04.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.508+0000 7f243aa00700 1 -- 172.21.15.112:0/1700941822 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2434010430 con 0x7f24440756d0 2024-05-08T05:58:04.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.510+0000 7f243aa00700 1 --2- 172.21.15.112:0/1700941822 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f24280b2980 0x7f24280b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.511+0000 7f243aa00700 1 -- 172.21.15.112:0/1700941822 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f24340e2440 con 0x7f24440756d0 2024-05-08T05:58:04.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.511+0000 7f2453a05b80 1 -- start start 2024-05-08T05:58:04.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.511+0000 7f2453a05b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7060 0x557f2fcc7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.511+0000 7f2453a05b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7980 0x557f2fccb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.511+0000 7f2453a05b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fccb9d0 0x557f2fccbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.511+0000 7f2453a05b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557f2fc19a20 con 0x557f2fcc7060 2024-05-08T05:58:04.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.511+0000 7f2453a05b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557f2fbb7760 con 0x557f2fcc7980 2024-05-08T05:58:04.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.511+0000 7f2453a05b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x557f2fc159e0 con 0x557f2fccb9d0 2024-05-08T05:58:04.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.512+0000 7f2441400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7980 0x557f2fccb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:04.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.512+0000 7f2441400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7980 0x557f2fccb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:33318/0 (socket says 172.21.15.112:33318) 2024-05-08T05:58:04.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.512+0000 7f2441400700 1 -- 172.21.15.112:0/1130342103 learned_addr learned my addr 172.21.15.112:0/1130342103 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:04.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.512+0000 7f2442800700 1 --2- 172.21.15.112:0/1130342103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fccb9d0 0x557f2fccbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:04.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.512+0000 7f2441e00700 1 --2- 172.21.15.112:0/1130342103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7060 0x557f2fcc7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:04.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.512+0000 7f2441400700 1 -- 172.21.15.112:0/1130342103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fccb9d0 msgr2=0x557f2fccbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:04.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.512+0000 7f2441400700 1 --2- 172.21.15.112:0/1130342103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fccb9d0 0x557f2fccbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.512+0000 7f2441400700 1 -- 172.21.15.112:0/1130342103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7060 msgr2=0x557f2fcc7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:04.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.512+0000 7f2441400700 1 --2- 172.21.15.112:0/1130342103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7060 0x557f2fcc7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.512+0000 7f2441400700 1 -- 172.21.15.112:0/1130342103 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f2fb668d0 con 0x557f2fcc7980 2024-05-08T05:58:04.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.512+0000 7f2442800700 1 --2- 172.21.15.112:0/1700941822 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f24280b2980 0x7f24280b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:04.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.513+0000 7f2441400700 1 --2- 172.21.15.112:0/1130342103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7980 0x557f2fccb490 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2430015000 tx=0x7f24300096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1361d51eaa9395 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:04.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.514+0000 7f242f400700 1 -- 172.21.15.112:0/1130342103 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430027070 con 0x557f2fcc7980 2024-05-08T05:58:04.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.514+0000 7f242f400700 1 -- 172.21.15.112:0/1130342103 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24300056d0 con 0x557f2fcc7980 2024-05-08T05:58:04.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.514+0000 7f242f400700 1 -- 172.21.15.112:0/1130342103 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430003aa0 con 0x557f2fcc7980 2024-05-08T05:58:04.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.515+0000 7f2453a05b80 1 -- 172.21.15.112:0/1130342103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7980 msgr2=0x557f2fccb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:04.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.515+0000 7f2453a05b80 1 --2- 172.21.15.112:0/1130342103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7980 0x557f2fccb490 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2430015000 tx=0x7f24300096a0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.515+0000 7f2442800700 1 --2- 172.21.15.112:0/1700941822 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f24280b2980 0x7f24280b4e40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f243c0060b0 tx=0x7f243c009b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.515+0000 7f2453a05b80 1 -- 172.21.15.112:0/1130342103 shutdown_connections 2024-05-08T05:58:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.515+0000 7f2453a05b80 1 --2- 172.21.15.112:0/1130342103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fccb9d0 0x557f2fccbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.515+0000 7f2453a05b80 1 --2- 172.21.15.112:0/1130342103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7060 0x557f2fcc7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.515+0000 7f2453a05b80 1 --2- 172.21.15.112:0/1130342103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7980 0x557f2fccb490 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.515+0000 7f2453a05b80 1 -- 172.21.15.112:0/1130342103 >> 172.21.15.112:0/1130342103 conn(0x557f2fbf5b60 msgr2=0x557f2fcc17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.516+0000 7f2453a05b80 1 -- 172.21.15.112:0/1130342103 shutdown_connections 2024-05-08T05:58:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.516+0000 7f2453a05b80 1 -- 172.21.15.112:0/1130342103 wait complete. 2024-05-08T05:58:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.516+0000 7f2453a05b80 1 -- start start 2024-05-08T05:58:04.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.517+0000 7f2453a05b80 1 Processor -- start 2024-05-08T05:58:04.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.518+0000 7f2453a05b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7060 0x557f2fcd2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.518+0000 7f2453a05b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7980 0x557f2fcd0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.518+0000 7f2453a05b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fccb9d0 0x557f2fcd0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.518+0000 7f2453a05b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557f2fcd3b80 con 0x557f2fcc7980 2024-05-08T05:58:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.518+0000 7f2453a05b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557f2fb7fb20 con 0x557f2fcc7060 2024-05-08T05:58:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.518+0000 7f2453a05b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x557f2fcd3d00 con 0x557f2fccb9d0 2024-05-08T05:58:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.518+0000 7f2453a05b80 10 client.0 _ll_register_callbacks cb 0x557f2fbc0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:58:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.518+0000 7f2441400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7980 0x557f2fcd0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.518+0000 7f2441400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7980 0x557f2fcd0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60608/0 (socket says 172.21.15.112:60608) 2024-05-08T05:58:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.518+0000 7f2441400700 1 -- 172.21.15.112:0/750442305 learned_addr learned my addr 172.21.15.112:0/750442305 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.519+0000 7f2442800700 1 --2- 172.21.15.112:0/750442305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fccb9d0 0x557f2fcd0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.519+0000 7f2441e00700 1 --2- 172.21.15.112:0/750442305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7060 0x557f2fcd2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.519+0000 7f2442800700 1 -- 172.21.15.112:0/750442305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7060 msgr2=0x557f2fcd2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.519+0000 7f2442800700 1 --2- 172.21.15.112:0/750442305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7060 0x557f2fcd2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.519+0000 7f2442800700 1 -- 172.21.15.112:0/750442305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7980 msgr2=0x557f2fcd0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.519+0000 7f2442800700 1 --2- 172.21.15.112:0/750442305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7980 0x557f2fcd0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.519+0000 7f2442800700 1 -- 172.21.15.112:0/750442305 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f2fbdf420 con 0x557f2fccb9d0 2024-05-08T05:58:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.520+0000 7f2442800700 1 --2- 172.21.15.112:0/750442305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fccb9d0 0x557f2fcd0790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f244406bd30 tx=0x7f243c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1afe93e71346531d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.520+0000 7f2423e00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:58:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.520+0000 7f2423e00700 1 -- 172.21.15.112:0/750442305 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c004db0 con 0x557f2fccb9d0 2024-05-08T05:58:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.520+0000 7f2423e00700 1 -- 172.21.15.112:0/750442305 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f243c01b620 con 0x557f2fccb9d0 2024-05-08T05:58:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.520+0000 7f2453a05b80 1 -- 172.21.15.112:0/750442305 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24300056d0 con 0x557f2fccb9d0 2024-05-08T05:58:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.521+0000 7f2423e00700 1 -- 172.21.15.112:0/750442305 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f243c02a030 con 0x557f2fccb9d0 2024-05-08T05:58:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.521+0000 7f2453a05b80 10 client.25495 fetch_fsmap learned FSMap version 28 2024-05-08T05:58:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.521+0000 7f2453a05b80 1 -- 172.21.15.112:0/750442305 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f2fcde450 con 0x557f2fccb9d0 2024-05-08T05:58:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.522+0000 7f2423e00700 1 -- 172.21.15.112:0/750442305 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24300056d0 con 0x557f2fccb9d0 2024-05-08T05:58:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.522+0000 7f2423e00700 1 -- 172.21.15.112:0/750442305 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f243c02f850 con 0x557f2fccb9d0 2024-05-08T05:58:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.522+0000 7f2453a05b80 10 client.25495 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T05:58:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.522+0000 7f2453a05b80 10 client.25495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:58:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.522+0000 7f2453a05b80 20 client.25495 populate_metadata read hostname 'smithi112' 2024-05-08T05:58:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.522+0000 7f2453a05b80 1 --2- 172.21.15.112:0/750442305 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x557f2fce0a90 0x557f2fce2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.522+0000 7f2453a05b80 4 client.25495 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:58:04.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.522+0000 7f2453a05b80 1 -- 172.21.15.112:0/750442305 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f2fc940a0 con 0x557f2fce0a90 2024-05-08T05:58:04.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.523+0000 7f2441e00700 1 --2- 172.21.15.112:0/750442305 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x557f2fce0a90 0x557f2fce2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:04.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.524+0000 7f2441e00700 1 --2- 172.21.15.112:0/750442305 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x557f2fce0a90 0x557f2fce2ee0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1ae3e1ba6295d9b server_cookie=48fbc253ed3879be in_seq=0 out_seq=0 2024-05-08T05:58:04.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.524+0000 7f2423e00700 10 client.25495 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:04.731 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] renameat2 2024-05-08T05:58:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.526+0000 7f2423e00700 1 -- 172.21.15.112:0/750442305 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f2fc940a0 con 0x557f2fce0a90 2024-05-08T05:58:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.526+0000 7f2423e00700 10 client.25495 handle_command_reply: tid=0 2024-05-08T05:58:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.526+0000 7f2453a05b80 1 client.25495 shutdown 2024-05-08T05:58:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.528+0000 7f2453a05b80 1 -- 172.21.15.112:0/750442305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fccb9d0 msgr2=0x557f2fcd0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.528+0000 7f2453a05b80 1 --2- 172.21.15.112:0/750442305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fccb9d0 0x557f2fcd0790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f244406bd30 tx=0x7f243c009920 comp rx=0 tx=0).stop 2024-05-08T05:58:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.529+0000 7f2453a05b80 1 -- 172.21.15.112:0/750442305 shutdown_connections 2024-05-08T05:58:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.529+0000 7f2453a05b80 1 --2- 172.21.15.112:0/750442305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fccb9d0 0x557f2fcd0790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.529+0000 7f2453a05b80 1 --2- 172.21.15.112:0/750442305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7060 0x557f2fcd2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.529+0000 7f2453a05b80 1 --2- 172.21.15.112:0/750442305 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x557f2fce0a90 0x557f2fce2ee0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.529+0000 7f2453a05b80 1 --2- 172.21.15.112:0/750442305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7980 0x557f2fcd0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.529+0000 7f2453a05b80 1 -- 172.21.15.112:0/750442305 >> 172.21.15.112:0/750442305 conn(0x557f2fbf5b60 msgr2=0x557f2fcbc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.529+0000 7f2423e00700 0 client.25495 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.529+0000 7f2453a05b80 1 -- 172.21.15.112:0/750442305 shutdown_connections 2024-05-08T05:58:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.529+0000 7f2453a05b80 1 -- 172.21.15.112:0/750442305 wait complete. 2024-05-08T05:58:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.529+0000 7f2453a05b80 20 client.25495 trim_cache size 0 max 16384 2024-05-08T05:58:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.552+0000 7f2453a05b80 1 Processor -- start 2024-05-08T05:58:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.553+0000 7f2453a05b80 1 -- start start 2024-05-08T05:58:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.553+0000 7f2453a05b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fbf5b60 0x557f2fccf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.553+0000 7f2453a05b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7060 0x557f2fcdcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.553+0000 7f2453a05b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7980 0x557f2fce87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.553+0000 7f2453a05b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24441b6c10 con 0x557f2fcc7060 2024-05-08T05:58:04.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.553+0000 7f2453a05b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f24441b4440 con 0x557f2fcc7980 2024-05-08T05:58:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.553+0000 7f2453a05b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f24441b2290 con 0x557f2fbf5b60 2024-05-08T05:58:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.554+0000 7f2441400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7060 0x557f2fcdcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.554+0000 7f2441400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7060 0x557f2fcdcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60620/0 (socket says 172.21.15.112:60620) 2024-05-08T05:58:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.554+0000 7f2441e00700 1 --2- 172.21.15.112:0/171441973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fbf5b60 0x557f2fccf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.554+0000 7f2441400700 1 -- 172.21.15.112:0/171441973 learned_addr learned my addr 172.21.15.112:0/171441973 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.554+0000 7f2441400700 1 -- 172.21.15.112:0/171441973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fbf5b60 msgr2=0x557f2fccf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.554+0000 7f2441400700 1 --2- 172.21.15.112:0/171441973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fbf5b60 0x557f2fccf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.554+0000 7f2441400700 1 -- 172.21.15.112:0/171441973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7980 msgr2=0x557f2fce87f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:58:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.554+0000 7f2441400700 1 --2- 172.21.15.112:0/171441973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7980 0x557f2fce87f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.554+0000 7f2441400700 1 -- 172.21.15.112:0/171441973 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f2fcd0b10 con 0x557f2fcc7060 2024-05-08T05:58:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.555+0000 7f2441400700 1 --2- 172.21.15.112:0/171441973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7060 0x557f2fcdcc10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2430005950 tx=0x7f2430005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb13abdda4b9dad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.555+0000 7f242ea00700 1 -- 172.21.15.112:0/171441973 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430032070 con 0x557f2fcc7060 2024-05-08T05:58:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.555+0000 7f242ea00700 1 -- 172.21.15.112:0/171441973 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2430028040 con 0x557f2fcc7060 2024-05-08T05:58:04.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.555+0000 7f242ea00700 1 -- 172.21.15.112:0/171441973 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430027070 con 0x557f2fcc7060 2024-05-08T05:58:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.556+0000 7f2453a05b80 1 -- 172.21.15.112:0/171441973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7060 msgr2=0x557f2fcdcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.556+0000 7f2453a05b80 1 --2- 172.21.15.112:0/171441973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7060 0x557f2fcdcc10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2430005950 tx=0x7f2430005660 comp rx=0 tx=0).stop 2024-05-08T05:58:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.557+0000 7f2453a05b80 1 -- 172.21.15.112:0/171441973 shutdown_connections 2024-05-08T05:58:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.557+0000 7f2453a05b80 1 --2- 172.21.15.112:0/171441973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7980 0x557f2fce87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.557+0000 7f2453a05b80 1 --2- 172.21.15.112:0/171441973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fbf5b60 0x557f2fccf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.557+0000 7f2453a05b80 1 --2- 172.21.15.112:0/171441973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fcc7060 0x557f2fcdcc10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.557+0000 7f2453a05b80 1 -- 172.21.15.112:0/171441973 >> 172.21.15.112:0/171441973 conn(0x557f2fcbb610 msgr2=0x557f2fcbc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.557+0000 7f2453a05b80 1 -- 172.21.15.112:0/171441973 shutdown_connections 2024-05-08T05:58:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.558+0000 7f2453a05b80 1 -- 172.21.15.112:0/171441973 wait complete. 2024-05-08T05:58:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.558+0000 7f2453a05b80 1 -- start start 2024-05-08T05:58:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.558+0000 7f2453a05b80 1 Processor -- start 2024-05-08T05:58:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.559+0000 7f2453a05b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fbf5b60 0x557f2fccd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.559+0000 7f2453a05b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fcc7060 0x557f2fccd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.559+0000 7f2453a05b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7980 0x557f2fccdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.559+0000 7f2453a05b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24441b2900 con 0x557f2fbf5b60 2024-05-08T05:58:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.559+0000 7f2453a05b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f24441b3d70 con 0x557f2fcc7980 2024-05-08T05:58:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.559+0000 7f2453a05b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f24441b2fd0 con 0x557f2fcc7060 2024-05-08T05:58:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.559+0000 7f2453a05b80 10 client.0 _ll_register_callbacks cb 0x557f2fbc0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:58:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.560+0000 7f2441e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fbf5b60 0x557f2fccd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.560+0000 7f2441e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fbf5b60 0x557f2fccd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60634/0 (socket says 172.21.15.112:60634) 2024-05-08T05:58:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.560+0000 7f2441e00700 1 -- 172.21.15.112:0/3057046063 learned_addr learned my addr 172.21.15.112:0/3057046063 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.560+0000 7f2441400700 1 --2- 172.21.15.112:0/3057046063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fcc7060 0x557f2fccd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.561+0000 7f2441e00700 1 -- 172.21.15.112:0/3057046063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fcc7060 msgr2=0x557f2fccd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.561+0000 7f2441e00700 1 --2- 172.21.15.112:0/3057046063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fcc7060 0x557f2fccd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.561+0000 7f2441e00700 1 -- 172.21.15.112:0/3057046063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7980 msgr2=0x557f2fccdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.561+0000 7f2441e00700 1 --2- 172.21.15.112:0/3057046063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7980 0x557f2fccdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.561+0000 7f2441e00700 1 -- 172.21.15.112:0/3057046063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f2fd5b3b0 con 0x557f2fbf5b60 2024-05-08T05:58:04.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.561+0000 7f2441400700 1 --2- 172.21.15.112:0/3057046063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fcc7060 0x557f2fccd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:58:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.561+0000 7f2441e00700 1 --2- 172.21.15.112:0/3057046063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fbf5b60 0x557f2fccd0c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f24340a9960 tx=0x7f2434010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156cc40cd9c3aa63 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.561+0000 7f2423e00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:58:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.562+0000 7f2423e00700 1 -- 172.21.15.112:0/3057046063 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24340a94e0 con 0x557f2fbf5b60 2024-05-08T05:58:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.562+0000 7f2423e00700 1 -- 172.21.15.112:0/3057046063 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f24340cc2d0 con 0x557f2fbf5b60 2024-05-08T05:58:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.562+0000 7f2453a05b80 1 -- 172.21.15.112:0/3057046063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2430028040 con 0x557f2fbf5b60 2024-05-08T05:58:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.563+0000 7f2423e00700 1 -- 172.21.15.112:0/3057046063 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f24340aa0d0 con 0x557f2fbf5b60 2024-05-08T05:58:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.563+0000 7f2453a05b80 10 client.15351 fetch_fsmap learned FSMap version 28 2024-05-08T05:58:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.563+0000 7f2453a05b80 1 -- 172.21.15.112:0/3057046063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f2fd5c690 con 0x557f2fbf5b60 2024-05-08T05:58:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.563+0000 7f2423e00700 1 -- 172.21.15.112:0/3057046063 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2430028040 con 0x557f2fbf5b60 2024-05-08T05:58:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.564+0000 7f2423e00700 1 -- 172.21.15.112:0/3057046063 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2434010720 con 0x557f2fbf5b60 2024-05-08T05:58:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.564+0000 7f2453a05b80 10 client.15351 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T05:58:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.564+0000 7f2453a05b80 10 client.15351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:58:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.564+0000 7f2453a05b80 20 client.15351 populate_metadata read hostname 'smithi112' 2024-05-08T05:58:04.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.564+0000 7f2453a05b80 1 --2- 172.21.15.112:0/3057046063 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x557f2fccb9d0 0x557f2fd5ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.564+0000 7f2453a05b80 4 client.15351 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:58:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.564+0000 7f2453a05b80 1 -- 172.21.15.112:0/3057046063 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f2fc27400 con 0x557f2fccb9d0 2024-05-08T05:58:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.565+0000 7f2441400700 1 --2- 172.21.15.112:0/3057046063 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x557f2fccb9d0 0x557f2fd5ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.566+0000 7f2441400700 1 --2- 172.21.15.112:0/3057046063 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x557f2fccb9d0 0x557f2fd5ee40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b84209658de87208 server_cookie=a6990f1f7afe35af in_seq=0 out_seq=0 2024-05-08T05:58:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.566+0000 7f2423e00700 10 client.15351 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.566+0000 7f2423e00700 1 -- 172.21.15.112:0/3057046063 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f2fc27400 con 0x557f2fccb9d0 2024-05-08T05:58:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.566+0000 7f2423e00700 10 client.15351 handle_command_reply: tid=0 2024-05-08T05:58:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.566+0000 7f2453a05b80 1 client.15351 shutdown 2024-05-08T05:58:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.567+0000 7f2453a05b80 1 -- 172.21.15.112:0/3057046063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fbf5b60 msgr2=0x557f2fccd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.567+0000 7f2453a05b80 1 --2- 172.21.15.112:0/3057046063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fbf5b60 0x557f2fccd0c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f24340a9960 tx=0x7f2434010c10 comp rx=0 tx=0).stop 2024-05-08T05:58:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.568+0000 7f2453a05b80 1 -- 172.21.15.112:0/3057046063 shutdown_connections 2024-05-08T05:58:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.568+0000 7f2453a05b80 1 --2- 172.21.15.112:0/3057046063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f2fcc7980 0x557f2fccdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.568+0000 7f2453a05b80 1 --2- 172.21.15.112:0/3057046063 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x557f2fccb9d0 0x557f2fd5ee40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.568+0000 7f2453a05b80 1 --2- 172.21.15.112:0/3057046063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f2fbf5b60 0x557f2fccd0c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.568+0000 7f2453a05b80 1 --2- 172.21.15.112:0/3057046063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557f2fcc7060 0x557f2fccd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.568+0000 7f2453a05b80 1 -- 172.21.15.112:0/3057046063 >> 172.21.15.112:0/3057046063 conn(0x557f2fcbb610 msgr2=0x557f2fcc5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:04.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.568+0000 7f2423e00700 0 client.15351 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:04.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.571+0000 7f2453a05b80 1 -- 172.21.15.112:0/3057046063 shutdown_connections 2024-05-08T05:58:04.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.571+0000 7f2453a05b80 1 -- 172.21.15.112:0/3057046063 wait complete. 2024-05-08T05:58:04.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.571+0000 7f2453a05b80 20 client.15351 trim_cache size 0 max 16384 2024-05-08T05:58:04.742 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:58:04.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.572+0000 7f2449000700 1 -- 172.21.15.112:0/1700941822 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f24280b2980 msgr2=0x7f24280b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:04.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.572+0000 7f2449000700 1 --2- 172.21.15.112:0/1700941822 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f24280b2980 0x7f24280b4e40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f243c0060b0 tx=0x7f243c009b60 comp rx=0 tx=0).stop 2024-05-08T05:58:04.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.572+0000 7f2449000700 1 -- 172.21.15.112:0/1700941822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440756d0 msgr2=0x7f24441a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:04.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.572+0000 7f2449000700 1 --2- 172.21.15.112:0/1700941822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440756d0 0x7f24441a01f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f243400e9a0 tx=0x7f243400ecb0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.573+0000 7f2449000700 1 -- 172.21.15.112:0/1700941822 shutdown_connections 2024-05-08T05:58:04.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.573+0000 7f2449000700 1 --2- 172.21.15.112:0/1700941822 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f24280b2980 0x7f24280b4e40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.573+0000 7f2449000700 1 --2- 172.21.15.112:0/1700941822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2444114670 0x7f24441a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.573+0000 7f2449000700 1 --2- 172.21.15.112:0/1700941822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24440756d0 0x7f24441a01f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.573+0000 7f2449000700 1 --2- 172.21.15.112:0/1700941822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2444075ff0 0x7f24441a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:04.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.573+0000 7f2449000700 1 -- 172.21.15.112:0/1700941822 >> 172.21.15.112:0/1700941822 conn(0x7f244407a600 msgr2=0x7f2444110110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:04.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.573+0000 7f2449000700 1 -- 172.21.15.112:0/1700941822 shutdown_connections 2024-05-08T05:58:04.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:04.575+0000 7f2449000700 1 -- 172.21.15.112:0/1700941822 wait complete. 2024-05-08T05:58:04.744 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_getcwd 2024-05-08T05:58:04.746 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] e4compact 2024-05-08T05:58:04.860 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] test-nextquota 2024-05-08T05:58:04.891 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] punch-alternating 2024-05-08T05:58:05.019 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] attr-list-by-handle-cursor-test 2024-05-08T05:58:05.116 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] listxattr 2024-05-08T05:58:05.204 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dio-interleaved 2024-05-08T05:58:05.245 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_dir_type 2024-05-08T05:58:05.361 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dio-invalidate-cache 2024-05-08T05:58:05.401 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] stat_test 2024-05-08T05:58:05.534 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_encrypted_d_revalidate 2024-05-08T05:58:05.553 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] attr_replace_test 2024-05-08T05:58:05.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:05 smithi112 ceph-mon[30003]: pgmap v524: 97 pgs: 97 active+clean; 53 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 286 KiB/s wr, 76 op/s 2024-05-08T05:58:05.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:05 smithi162 ceph-mon[34800]: pgmap v524: 97 pgs: 97 active+clean; 53 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 286 KiB/s wr, 76 op/s 2024-05-08T05:58:05.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:05 smithi150 ceph-mon[34315]: pgmap v524: 97 pgs: 97 active+clean; 53 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 286 KiB/s wr, 76 op/s 2024-05-08T05:58:05.730 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] swapon 2024-05-08T05:58:05.841 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] mkswap 2024-05-08T05:58:05.879 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_attr_corruption 2024-05-08T05:58:05.892 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_open_tmpfiles 2024-05-08T05:58:06.154 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fscrypt-crypt-util 2024-05-08T05:58:06.247 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] bulkstat_null_ocount 2024-05-08T05:58:06.250 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] splice-test 2024-05-08T05:58:06.251 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] chprojid_fail 2024-05-08T05:58:06.576 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] detached_mounts_propagation 2024-05-08T05:58:06.593 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] ext4_resize 2024-05-08T05:58:06.624 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_readdir_3 2024-05-08T05:58:06.658 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T05:58:06.659 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T05:58:06.934 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] splice2pipe 2024-05-08T05:58:07.033 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] uuid_ioctl 2024-05-08T05:58:07.071 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_snapshot_deleted_subvolume 2024-05-08T05:58:07.184 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fiemap-fault 2024-05-08T05:58:07.234 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:58:07.326 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fiemap-tester 2024-05-08T05:58:07.389 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_stripealign 2024-05-08T05:58:07.423 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] open_by_handle 2024-05-08T05:58:07.530 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_dio 2024-05-08T05:58:07.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:07 smithi112 ceph-mon[30003]: pgmap v525: 97 pgs: 97 active+clean; 54 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 360 KiB/s wr, 76 op/s 2024-05-08T05:58:07.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:07 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:58:07.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:07 smithi162 ceph-mon[34800]: pgmap v525: 97 pgs: 97 active+clean; 54 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 360 KiB/s wr, 76 op/s 2024-05-08T05:58:07.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:07 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:58:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:07 smithi150 ceph-mon[34315]: pgmap v525: 97 pgs: 97 active+clean; 54 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 360 KiB/s wr, 76 op/s 2024-05-08T05:58:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:07 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:58:07.725 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] xfsfind 2024-05-08T05:58:07.855 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_reflink_read_race 2024-05-08T05:58:07.877 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fssum 2024-05-08T05:58:07.891 INFO:tasks.workunit.client.0.smithi112.stdout:Building log-writes 2024-05-08T05:58:07.910 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] replay-log 2024-05-08T05:58:08.135 INFO:tasks.workunit.client.0.smithi112.stdout:Building perf 2024-05-08T05:58:08.145 INFO:tasks.workunit.client.0.smithi112.stdout:Building aio-dio-regress 2024-05-08T05:58:08.160 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aiocp 2024-05-08T05:58:08.313 INFO:tasks.workunit.client.0.smithi112.stdout:Building vfs 2024-05-08T05:58:08.332 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] vfstest 2024-05-08T05:58:08.617 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] mount-idmapped 2024-05-08T05:58:08.632 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-append-write-fallocate-race 2024-05-08T05:58:08.637 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-append-write-read-race 2024-05-08T05:58:08.961 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-cow-race 2024-05-08T05:58:08.978 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-cycle-write 2024-05-08T05:58:09.258 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-eof-race 2024-05-08T05:58:09.355 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-extend-stat 2024-05-08T05:58:09.389 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:09 smithi112 ceph-mon[30003]: pgmap v526: 97 pgs: 97 active+clean; 54 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 329 KiB/s wr, 47 op/s 2024-05-08T05:58:09.434 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-fcntl-race 2024-05-08T05:58:09.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:09 smithi162 ceph-mon[34800]: pgmap v526: 97 pgs: 97 active+clean; 54 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 329 KiB/s wr, 47 op/s 2024-05-08T05:58:09.622 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-hole-filling-race 2024-05-08T05:58:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:09 smithi150 ceph-mon[34315]: pgmap v526: 97 pgs: 97 active+clean; 54 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 329 KiB/s wr, 47 op/s 2024-05-08T05:58:09.710 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-invalidate-failure 2024-05-08T05:58:09.761 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-invalidate-readahead 2024-05-08T05:58:09.912 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aiodio_sparse2 2024-05-08T05:58:10.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.995+0000 7f8077e00700 1 -- 172.21.15.112:0/431289566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f807006b180 msgr2=0x7f807006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.995+0000 7f8077e00700 1 --2- 172.21.15.112:0/431289566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f807006b180 0x7f807006b600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f805c009990 tx=0x7f805c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.996+0000 7f8077e00700 1 -- 172.21.15.112:0/431289566 shutdown_connections 2024-05-08T05:58:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.996+0000 7f8077e00700 1 --2- 172.21.15.112:0/431289566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807006bb40 0x7f8070110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.996+0000 7f8077e00700 1 --2- 172.21.15.112:0/431289566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8070109510 0x7f80701098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.996+0000 7f8077e00700 1 --2- 172.21.15.112:0/431289566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f807006b180 0x7f807006b600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.996+0000 7f8077e00700 1 -- 172.21.15.112:0/431289566 >> 172.21.15.112:0/431289566 conn(0x7f8070079350 msgr2=0x7f8070079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.996+0000 7f8077e00700 1 -- 172.21.15.112:0/431289566 shutdown_connections 2024-05-08T05:58:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.996+0000 7f8077e00700 1 -- 172.21.15.112:0/431289566 wait complete. 2024-05-08T05:58:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.997+0000 7f8077e00700 1 Processor -- start 2024-05-08T05:58:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.997+0000 7f8077e00700 1 -- start start 2024-05-08T05:58:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.997+0000 7f8077e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f807006b180 0x7f807019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.997+0000 7f8077e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807006bb40 0x7f807019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.997+0000 7f8077e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8070109510 0x7f80701a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.997+0000 7f8077e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8070113690 con 0x7f807006b180 2024-05-08T05:58:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.997+0000 7f8077e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8070113510 con 0x7f8070109510 2024-05-08T05:58:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.997+0000 7f8077e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8070113810 con 0x7f807006bb40 2024-05-08T05:58:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.998+0000 7f8074a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807006bb40 0x7f807019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.998+0000 7f8074a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807006bb40 0x7f807019c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:36182/0 (socket says 172.21.15.112:36182) 2024-05-08T05:58:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.998+0000 7f8074a00700 1 -- 172.21.15.112:0/450907325 learned_addr learned my addr 172.21.15.112:0/450907325 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.998+0000 7f8075400700 1 --2- 172.21.15.112:0/450907325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f807006b180 0x7f807019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.998+0000 7f8075e00700 1 --2- 172.21.15.112:0/450907325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8070109510 0x7f80701a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.998+0000 7f8074a00700 1 -- 172.21.15.112:0/450907325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8070109510 msgr2=0x7f80701a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.998+0000 7f8074a00700 1 --2- 172.21.15.112:0/450907325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8070109510 0x7f80701a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.998+0000 7f8074a00700 1 -- 172.21.15.112:0/450907325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f807006b180 msgr2=0x7f807019bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.998+0000 7f8074a00700 1 --2- 172.21.15.112:0/450907325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f807006b180 0x7f807019bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.998+0000 7f8074a00700 1 -- 172.21.15.112:0/450907325 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80701a0db0 con 0x7f807006bb40 2024-05-08T05:58:10.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.998+0000 7f8075e00700 1 --2- 172.21.15.112:0/450907325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8070109510 0x7f80701a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:58:10.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.999+0000 7f8074a00700 1 --2- 172.21.15.112:0/450907325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807006bb40 0x7f807019c2e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f805c006010 tx=0x7f805c00bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:10.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.999+0000 7f806e000700 1 -- 172.21.15.112:0/450907325 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c027070 con 0x7f807006bb40 2024-05-08T05:58:10.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.999+0000 7f8077e00700 1 -- 172.21.15.112:0/450907325 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80701a1040 con 0x7f807006bb40 2024-05-08T05:58:10.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.999+0000 7f806e000700 1 -- 172.21.15.112:0/450907325 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f805c005da0 con 0x7f807006bb40 2024-05-08T05:58:10.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.999+0000 7f806e000700 1 -- 172.21.15.112:0/450907325 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c0173f0 con 0x7f807006bb40 2024-05-08T05:58:10.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:09.999+0000 7f8077e00700 1 -- 172.21.15.112:0/450907325 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80701a8920 con 0x7f807006bb40 2024-05-08T05:58:10.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.004+0000 7f806e000700 1 -- 172.21.15.112:0/450907325 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f805c02a070 con 0x7f807006bb40 2024-05-08T05:58:10.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.004+0000 7f80826e2b80 1 Processor -- start 2024-05-08T05:58:10.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.005+0000 7f80826e2b80 1 -- start start 2024-05-08T05:58:10.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.006+0000 7f806e000700 1 --2- 172.21.15.112:0/450907325 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f80580b2880 0x7f80580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.006+0000 7f806e000700 1 -- 172.21.15.112:0/450907325 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f805c0e2ef0 con 0x7f807006bb40 2024-05-08T05:58:10.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.006+0000 7f8075e00700 1 --2- 172.21.15.112:0/450907325 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f80580b2880 0x7f80580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.006+0000 7f80826e2b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 0x561da2a3d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.006+0000 7f80826e2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a38b60 0x561da2a38fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.006+0000 7f80826e2b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a39520 0x561da2a45270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.006+0000 7f80826e2b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561da2982aa0 con 0x561da2a3d350 2024-05-08T05:58:10.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.006+0000 7f80826e2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561da29207e0 con 0x561da2a38b60 2024-05-08T05:58:10.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.006+0000 7f80826e2b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561da297ea60 con 0x561da2a39520 2024-05-08T05:58:10.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.007+0000 7f8074a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a38b60 0x561da2a38fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.007+0000 7f8074a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a38b60 0x561da2a38fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:33372/0 (socket says 172.21.15.112:33372) 2024-05-08T05:58:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.007+0000 7f8074a00700 1 -- 172.21.15.112:0/198092770 learned_addr learned my addr 172.21.15.112:0/198092770 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.007+0000 7f8075e00700 1 --2- 172.21.15.112:0/198092770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a39520 0x561da2a45270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.007+0000 7f8074a00700 1 -- 172.21.15.112:0/198092770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a39520 msgr2=0x561da2a45270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.007+0000 7f8074a00700 1 --2- 172.21.15.112:0/198092770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a39520 0x561da2a45270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.007+0000 7f8074a00700 1 -- 172.21.15.112:0/198092770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 msgr2=0x561da2a3d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.007+0000 7f8074a00700 1 --2- 172.21.15.112:0/198092770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 0x561da2a3d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.007+0000 7f8074a00700 1 -- 172.21.15.112:0/198092770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561da28cf950 con 0x561da2a38b60 2024-05-08T05:58:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.007+0000 7f8075e00700 1 --2- 172.21.15.112:0/450907325 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f80580b2880 0x7f80580b4d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8068009410 tx=0x7f8068009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.008+0000 7f8074a00700 1 --2- 172.21.15.112:0/198092770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a38b60 0x561da2a38fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f805c0098e0 tx=0x7f805c022a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da098381e530501 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.008+0000 7f8062a00700 1 -- 172.21.15.112:0/198092770 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c0aa570 con 0x561da2a38b60 2024-05-08T05:58:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.008+0000 7f8062a00700 1 -- 172.21.15.112:0/198092770 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f805c0ae920 con 0x561da2a38b60 2024-05-08T05:58:10.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.008+0000 7f8062a00700 1 -- 172.21.15.112:0/198092770 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c0b2eb0 con 0x561da2a38b60 2024-05-08T05:58:10.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.009+0000 7f80826e2b80 1 -- 172.21.15.112:0/198092770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a38b60 msgr2=0x561da2a38fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:10.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.009+0000 7f80826e2b80 1 --2- 172.21.15.112:0/198092770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a38b60 0x561da2a38fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f805c0098e0 tx=0x7f805c022a30 comp rx=0 tx=0).stop 2024-05-08T05:58:10.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.009+0000 7f80826e2b80 1 -- 172.21.15.112:0/198092770 shutdown_connections 2024-05-08T05:58:10.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.009+0000 7f80826e2b80 1 --2- 172.21.15.112:0/198092770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a39520 0x561da2a45270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.009+0000 7f80826e2b80 1 --2- 172.21.15.112:0/198092770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 0x561da2a3d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.009+0000 7f80826e2b80 1 --2- 172.21.15.112:0/198092770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a38b60 0x561da2a38fe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.009+0000 7f80826e2b80 1 -- 172.21.15.112:0/198092770 >> 172.21.15.112:0/198092770 conn(0x561da29a8750 msgr2=0x561da2a2ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:10.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.009+0000 7f80826e2b80 1 -- 172.21.15.112:0/198092770 shutdown_connections 2024-05-08T05:58:10.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.009+0000 7f80826e2b80 1 -- 172.21.15.112:0/198092770 wait complete. 2024-05-08T05:58:10.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.010+0000 7f80826e2b80 1 -- start start 2024-05-08T05:58:10.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.011+0000 7f80826e2b80 1 Processor -- start 2024-05-08T05:58:10.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.012+0000 7f80826e2b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a38b60 0x561da2a44f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.012+0000 7f80826e2b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a39520 0x561da2a42300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.012+0000 7f80826e2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a3d350 0x561da2a42840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.012+0000 7f80826e2b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561da2a3de40 con 0x561da2a38b60 2024-05-08T05:58:10.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.012+0000 7f80826e2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561da28e8ba0 con 0x561da2a3d350 2024-05-08T05:58:10.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.012+0000 7f8075400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a38b60 0x561da2a44f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.012+0000 7f8075e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a3d350 0x561da2a42840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.012+0000 7f8075e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a3d350 0x561da2a42840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:33376/0 (socket says 172.21.15.112:33376) 2024-05-08T05:58:10.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.012+0000 7f8075e00700 1 -- 172.21.15.112:0/3745863022 learned_addr learned my addr 172.21.15.112:0/3745863022 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:10.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.013+0000 7f80826e2b80 1 -- 172.21.15.112:0/3745863022 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561da2a457b0 con 0x561da2a39520 2024-05-08T05:58:10.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.013+0000 7f80826e2b80 10 client.0 _ll_register_callbacks cb 0x561da2929cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:58:10.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.013+0000 7f8075e00700 1 -- 172.21.15.112:0/3745863022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a39520 msgr2=0x561da2a42300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:58:10.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.013+0000 7f8075e00700 1 --2- 172.21.15.112:0/3745863022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a39520 0x561da2a42300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.013+0000 7f8075e00700 1 -- 172.21.15.112:0/3745863022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a38b60 msgr2=0x561da2a44f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:10.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.013+0000 7f8075e00700 1 --2- 172.21.15.112:0/3745863022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a38b60 0x561da2a44f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.013+0000 7f8075e00700 1 -- 172.21.15.112:0/3745863022 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561da29484a0 con 0x561da2a3d350 2024-05-08T05:58:10.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.013+0000 7f8075400700 1 --2- 172.21.15.112:0/3745863022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a38b60 0x561da2a44f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:58:10.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.014+0000 7f8075e00700 1 --2- 172.21.15.112:0/3745863022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a3d350 0x561da2a42840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8068017c50 tx=0x7f8068004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9e68244114e35b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:10.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.014+0000 7f8057400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:58:10.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.014+0000 7f8057400700 1 -- 172.21.15.112:0/3745863022 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068027020 con 0x561da2a3d350 2024-05-08T05:58:10.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.014+0000 7f8057400700 1 -- 172.21.15.112:0/3745863022 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f8068018d60 con 0x561da2a3d350 2024-05-08T05:58:10.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.014+0000 7f80826e2b80 1 -- 172.21.15.112:0/3745863022 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f805c0ae920 con 0x561da2a3d350 2024-05-08T05:58:10.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.015+0000 7f8057400700 1 -- 172.21.15.112:0/3745863022 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8068032050 con 0x561da2a3d350 2024-05-08T05:58:10.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.015+0000 7f80826e2b80 10 client.25478 fetch_fsmap learned FSMap version 29 2024-05-08T05:58:10.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.015+0000 7f80826e2b80 1 -- 172.21.15.112:0/3745863022 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561da2a4fe60 con 0x561da2a3d350 2024-05-08T05:58:10.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.016+0000 7f8057400700 1 -- 172.21.15.112:0/3745863022 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f805c0ae920 con 0x561da2a3d350 2024-05-08T05:58:10.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.016+0000 7f8057400700 1 -- 172.21.15.112:0/3745863022 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f8068022500 con 0x561da2a3d350 2024-05-08T05:58:10.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.017+0000 7f80826e2b80 10 client.25478 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:58:10.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.017+0000 7f80826e2b80 10 client.25478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:58:10.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.017+0000 7f80826e2b80 20 client.25478 populate_metadata read hostname 'smithi112' 2024-05-08T05:58:10.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.017+0000 7f80826e2b80 1 --2- 172.21.15.112:0/3745863022 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x561da2a52540 0x561da2a54930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.017+0000 7f80826e2b80 4 client.25478 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:58:10.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.017+0000 7f80826e2b80 1 -- 172.21.15.112:0/3745863022 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561da29fd1c0 con 0x561da2a52540 2024-05-08T05:58:10.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.017+0000 7f8075400700 1 --2- 172.21.15.112:0/3745863022 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x561da2a52540 0x561da2a54930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.018+0000 7f8075400700 1 --2- 172.21.15.112:0/3745863022 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x561da2a52540 0x561da2a54930 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb267e604bceb68e server_cookie=701995feff2893f1 in_seq=0 out_seq=0 2024-05-08T05:58:10.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.018+0000 7f8057400700 10 client.25478 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:10.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.019+0000 7f8057400700 1 -- 172.21.15.112:0/3745863022 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561da29fd1c0 con 0x561da2a52540 2024-05-08T05:58:10.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.019+0000 7f8057400700 10 client.25478 handle_command_reply: tid=0 2024-05-08T05:58:10.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.019+0000 7f80826e2b80 1 client.25478 shutdown 2024-05-08T05:58:10.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.020+0000 7f80826e2b80 1 -- 172.21.15.112:0/3745863022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a3d350 msgr2=0x561da2a42840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:10.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.020+0000 7f80826e2b80 1 --2- 172.21.15.112:0/3745863022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a3d350 0x561da2a42840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8068017c50 tx=0x7f8068004b40 comp rx=0 tx=0).stop 2024-05-08T05:58:10.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.020+0000 7f80826e2b80 1 -- 172.21.15.112:0/3745863022 shutdown_connections 2024-05-08T05:58:10.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.020+0000 7f80826e2b80 1 --2- 172.21.15.112:0/3745863022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a3d350 0x561da2a42840 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.020+0000 7f80826e2b80 1 --2- 172.21.15.112:0/3745863022 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x561da2a52540 0x561da2a54930 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.020+0000 7f80826e2b80 1 --2- 172.21.15.112:0/3745863022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a38b60 0x561da2a44f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.020+0000 7f80826e2b80 1 --2- 172.21.15.112:0/3745863022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a39520 0x561da2a42300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.020+0000 7f80826e2b80 1 -- 172.21.15.112:0/3745863022 >> 172.21.15.112:0/3745863022 conn(0x561da29a8750 msgr2=0x561da2a3e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:10.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.020+0000 7f8057400700 0 client.25478 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:10.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.020+0000 7f80826e2b80 1 -- 172.21.15.112:0/3745863022 shutdown_connections 2024-05-08T05:58:10.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.020+0000 7f80826e2b80 1 -- 172.21.15.112:0/3745863022 wait complete. 2024-05-08T05:58:10.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.021+0000 7f80826e2b80 20 client.25478 trim_cache size 0 max 16384 2024-05-08T05:58:10.035 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-subblock-eof-read 2024-05-08T05:58:10.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.043+0000 7f80826e2b80 1 Processor -- start 2024-05-08T05:58:10.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.043+0000 7f80826e2b80 1 -- start start 2024-05-08T05:58:10.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.044+0000 7f80826e2b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a38b60 0x561da2a4e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.044+0000 7f80826e2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a39520 0x561da2a2a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.044+0000 7f80826e2b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 0x561da2a5a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.044+0000 7f80826e2b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80701ae1e0 con 0x561da2a3d350 2024-05-08T05:58:10.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.044+0000 7f80826e2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f80701ab9e0 con 0x561da2a39520 2024-05-08T05:58:10.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.044+0000 7f80826e2b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f807010e1b0 con 0x561da2a38b60 2024-05-08T05:58:10.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.044+0000 7f8074a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a39520 0x561da2a2a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.044+0000 7f8074a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a39520 0x561da2a2a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:33380/0 (socket says 172.21.15.112:33380) 2024-05-08T05:58:10.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.044+0000 7f8074a00700 1 -- 172.21.15.112:0/1174857562 learned_addr learned my addr 172.21.15.112:0/1174857562 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:10.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.044+0000 7f8075e00700 1 --2- 172.21.15.112:0/1174857562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 0x561da2a5a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.045+0000 7f8075400700 1 --2- 172.21.15.112:0/1174857562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a38b60 0x561da2a4e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.045+0000 7f8074a00700 1 -- 172.21.15.112:0/1174857562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a38b60 msgr2=0x561da2a4e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:10.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.045+0000 7f8074a00700 1 --2- 172.21.15.112:0/1174857562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a38b60 0x561da2a4e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.045+0000 7f8074a00700 1 -- 172.21.15.112:0/1174857562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 msgr2=0x561da2a5a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:10.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.045+0000 7f8074a00700 1 --2- 172.21.15.112:0/1174857562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 0x561da2a5a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.045+0000 7f8074a00700 1 -- 172.21.15.112:0/1174857562 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561da2a54f90 con 0x561da2a39520 2024-05-08T05:58:10.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.045+0000 7f8075e00700 1 --2- 172.21.15.112:0/1174857562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 0x561da2a5a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:58:10.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.046+0000 7f8074a00700 1 --2- 172.21.15.112:0/1174857562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a39520 0x561da2a2a990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f805c0a65f0 tx=0x7f805c0aa3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cead7fec86a0645a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:10.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.046+0000 7f8062000700 1 -- 172.21.15.112:0/1174857562 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c0aa570 con 0x561da2a39520 2024-05-08T05:58:10.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.046+0000 7f8062000700 1 -- 172.21.15.112:0/1174857562 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f805c0ae920 con 0x561da2a39520 2024-05-08T05:58:10.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.046+0000 7f8062000700 1 -- 172.21.15.112:0/1174857562 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c0b2ff0 con 0x561da2a39520 2024-05-08T05:58:10.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.047+0000 7f80826e2b80 1 -- 172.21.15.112:0/1174857562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a39520 msgr2=0x561da2a2a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:10.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.047+0000 7f80826e2b80 1 --2- 172.21.15.112:0/1174857562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a39520 0x561da2a2a990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f805c0a65f0 tx=0x7f805c0aa3d0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.047+0000 7f80826e2b80 1 -- 172.21.15.112:0/1174857562 shutdown_connections 2024-05-08T05:58:10.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.047+0000 7f80826e2b80 1 --2- 172.21.15.112:0/1174857562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 0x561da2a5a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.047+0000 7f80826e2b80 1 --2- 172.21.15.112:0/1174857562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a38b60 0x561da2a4e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.047+0000 7f80826e2b80 1 --2- 172.21.15.112:0/1174857562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a39520 0x561da2a2a990 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f805c0a65f0 tx=0x7f805c0aa3d0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.047+0000 7f80826e2b80 1 -- 172.21.15.112:0/1174857562 >> 172.21.15.112:0/1174857562 conn(0x561da29a8750 msgr2=0x561da2a2fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:10.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.047+0000 7f80826e2b80 1 -- 172.21.15.112:0/1174857562 shutdown_connections 2024-05-08T05:58:10.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.048+0000 7f80826e2b80 1 -- 172.21.15.112:0/1174857562 wait complete. 2024-05-08T05:58:10.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.048+0000 7f80826e2b80 1 -- start start 2024-05-08T05:58:10.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.048+0000 7f80826e2b80 1 Processor -- start 2024-05-08T05:58:10.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.049+0000 7f80826e2b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a38b60 0x561da2a3b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.049+0000 7f80826e2b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 0x561da2a4a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.049+0000 7f80826e2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a4ae20 0x561da2a39f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.049+0000 7f80826e2b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80701a9ea0 con 0x561da2a3d350 2024-05-08T05:58:10.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.049+0000 7f80826e2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f80701ab310 con 0x561da2a4ae20 2024-05-08T05:58:10.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.049+0000 7f80826e2b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f80701aa570 con 0x561da2a38b60 2024-05-08T05:58:10.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.049+0000 7f80826e2b80 10 client.0 _ll_register_callbacks cb 0x561da2929cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:58:10.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.050+0000 7f8074a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 0x561da2a4a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.050+0000 7f8075e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a4ae20 0x561da2a39f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.050+0000 7f8074a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 0x561da2a4a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60706/0 (socket says 172.21.15.112:60706) 2024-05-08T05:58:10.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.050+0000 7f8074a00700 1 -- 172.21.15.112:0/1911807240 learned_addr learned my addr 172.21.15.112:0/1911807240 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:10.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.050+0000 7f8075400700 1 --2- 172.21.15.112:0/1911807240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a38b60 0x561da2a3b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.050+0000 7f8074a00700 1 -- 172.21.15.112:0/1911807240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a38b60 msgr2=0x561da2a3b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:10.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.050+0000 7f8074a00700 1 --2- 172.21.15.112:0/1911807240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a38b60 0x561da2a3b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.050+0000 7f8074a00700 1 -- 172.21.15.112:0/1911807240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a4ae20 msgr2=0x561da2a39f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:10.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.050+0000 7f8074a00700 1 --2- 172.21.15.112:0/1911807240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a4ae20 0x561da2a39f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.050+0000 7f8074a00700 1 -- 172.21.15.112:0/1911807240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561da2a3d980 con 0x561da2a3d350 2024-05-08T05:58:10.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.051+0000 7f8075400700 1 --2- 172.21.15.112:0/1911807240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a38b60 0x561da2a3b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:58:10.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.051+0000 7f8074a00700 1 --2- 172.21.15.112:0/1911807240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 0x561da2a4a8e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f805c0a77e0 tx=0x7f805c00b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69a10cedf2842b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:10.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.051+0000 7f8057400700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:58:10.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.052+0000 7f8057400700 1 -- 172.21.15.112:0/1911807240 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c0aa9e0 con 0x561da2a3d350 2024-05-08T05:58:10.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.052+0000 7f80826e2b80 1 -- 172.21.15.112:0/1911807240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f805c0ae920 con 0x561da2a3d350 2024-05-08T05:58:10.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.052+0000 7f8057400700 1 -- 172.21.15.112:0/1911807240 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f805c022470 con 0x561da2a3d350 2024-05-08T05:58:10.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.052+0000 7f8057400700 1 -- 172.21.15.112:0/1911807240 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f805c00fb40 con 0x561da2a3d350 2024-05-08T05:58:10.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.052+0000 7f80826e2b80 10 client.15366 fetch_fsmap learned FSMap version 29 2024-05-08T05:58:10.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.052+0000 7f80826e2b80 1 -- 172.21.15.112:0/1911807240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561da2acf310 con 0x561da2a3d350 2024-05-08T05:58:10.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.053+0000 7f8057400700 1 -- 172.21.15.112:0/1911807240 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f805c0ae920 con 0x561da2a3d350 2024-05-08T05:58:10.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.053+0000 7f8057400700 1 -- 172.21.15.112:0/1911807240 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f805c090ae0 con 0x561da2a3d350 2024-05-08T05:58:10.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.053+0000 7f80826e2b80 10 client.15366 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:58:10.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.053+0000 7f80826e2b80 10 client.15366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:58:10.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.053+0000 7f80826e2b80 20 client.15366 populate_metadata read hostname 'smithi112' 2024-05-08T05:58:10.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.053+0000 7f80826e2b80 1 --2- 172.21.15.112:0/1911807240 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x561da2acf550 0x561da2ad1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:10.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.054+0000 7f80826e2b80 4 client.15366 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T05:58:10.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.054+0000 7f80826e2b80 1 -- 172.21.15.112:0/1911807240 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561da2990480 con 0x561da2acf550 2024-05-08T05:58:10.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.054+0000 7f8075400700 1 --2- 172.21.15.112:0/1911807240 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x561da2acf550 0x561da2ad1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:10.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.055+0000 7f8075400700 1 --2- 172.21.15.112:0/1911807240 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x561da2acf550 0x561da2ad1970 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b09c4d7508e90c6 server_cookie=963fb5b78869d334 in_seq=0 out_seq=0 2024-05-08T05:58:10.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.055+0000 7f8057400700 10 client.15366 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:10.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.059+0000 7f8057400700 1 -- 172.21.15.112:0/1911807240 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561da2990480 con 0x561da2acf550 2024-05-08T05:58:10.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.059+0000 7f8057400700 10 client.15366 handle_command_reply: tid=0 2024-05-08T05:58:10.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.059+0000 7f80826e2b80 1 client.15366 shutdown 2024-05-08T05:58:10.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.059+0000 7f80826e2b80 1 -- 172.21.15.112:0/1911807240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 msgr2=0x561da2a4a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:10.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.059+0000 7f80826e2b80 1 --2- 172.21.15.112:0/1911807240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 0x561da2a4a8e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f805c0a77e0 tx=0x7f805c00b610 comp rx=0 tx=0).stop 2024-05-08T05:58:10.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.059+0000 7f80826e2b80 1 -- 172.21.15.112:0/1911807240 shutdown_connections 2024-05-08T05:58:10.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.060+0000 7f80826e2b80 1 --2- 172.21.15.112:0/1911807240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561da2a4ae20 0x561da2a39f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.060+0000 7f80826e2b80 1 --2- 172.21.15.112:0/1911807240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561da2a38b60 0x561da2a3b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.060+0000 7f80826e2b80 1 --2- 172.21.15.112:0/1911807240 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x561da2acf550 0x561da2ad1970 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.060+0000 7f80826e2b80 1 --2- 172.21.15.112:0/1911807240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561da2a3d350 0x561da2a4a8e0 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f805c0a77e0 tx=0x7f805c00b610 comp rx=0 tx=0).stop 2024-05-08T05:58:10.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.060+0000 7f80826e2b80 1 -- 172.21.15.112:0/1911807240 >> 172.21.15.112:0/1911807240 conn(0x561da29a8750 msgr2=0x561da2a438d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:10.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.060+0000 7f8057400700 0 client.15366 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:10.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.060+0000 7f80826e2b80 1 -- 172.21.15.112:0/1911807240 shutdown_connections 2024-05-08T05:58:10.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.060+0000 7f80826e2b80 1 -- 172.21.15.112:0/1911807240 wait complete. 2024-05-08T05:58:10.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.061+0000 7f80826e2b80 20 client.15366 trim_cache size 0 max 16384 2024-05-08T05:58:10.061 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:58:10.062 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-08T05:58:10.062 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "45688eea-deb8-4e01-a574-5e050c80e33f", 2024-05-08T05:58:10.062 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-08T05:58:10.062 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:58:10.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.063+0000 7f8077e00700 1 -- 172.21.15.112:0/450907325 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f80580b2880 msgr2=0x7f80580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:10.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.063+0000 7f8077e00700 1 --2- 172.21.15.112:0/450907325 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f80580b2880 0x7f80580b4d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8068009410 tx=0x7f8068009920 comp rx=0 tx=0).stop 2024-05-08T05:58:10.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.063+0000 7f8077e00700 1 -- 172.21.15.112:0/450907325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807006bb40 msgr2=0x7f807019c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:10.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.063+0000 7f8077e00700 1 --2- 172.21.15.112:0/450907325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807006bb40 0x7f807019c2e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f805c006010 tx=0x7f805c00bf20 comp rx=0 tx=0).stop 2024-05-08T05:58:10.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.065+0000 7f8077e00700 1 -- 172.21.15.112:0/450907325 shutdown_connections 2024-05-08T05:58:10.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.065+0000 7f8077e00700 1 --2- 172.21.15.112:0/450907325 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f80580b2880 0x7f80580b4d40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.065+0000 7f8077e00700 1 --2- 172.21.15.112:0/450907325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8070109510 0x7f80701a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.065+0000 7f8077e00700 1 --2- 172.21.15.112:0/450907325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f807006b180 0x7f807019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.065+0000 7f8077e00700 1 --2- 172.21.15.112:0/450907325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f807006bb40 0x7f807019c2e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:10.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.065+0000 7f8077e00700 1 -- 172.21.15.112:0/450907325 >> 172.21.15.112:0/450907325 conn(0x7f8070079350 msgr2=0x7f807010e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:10.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.066+0000 7f8077e00700 1 -- 172.21.15.112:0/450907325 shutdown_connections 2024-05-08T05:58:10.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:10.066+0000 7f8077e00700 1 -- 172.21.15.112:0/450907325 wait complete. 2024-05-08T05:58:10.175 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-write-verify 2024-05-08T05:58:10.477 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-05-08T05:58:10.500 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-05-08T05:58:10.672 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-last-ref-held-by-io 2024-05-08T05:58:11.231 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T05:58:11.366 INFO:tasks.workunit.client.0.smithi112.stderr:+ cd .. 2024-05-08T05:58:11.366 INFO:tasks.workunit.client.0.smithi112.stderr:+ cp xfstests-dev/ltp/fsx . 2024-05-08T05:58:11.371 INFO:tasks.workunit.client.0.smithi112.stderr:+ OPTIONS=-z 2024-05-08T05:58:11.371 INFO:tasks.workunit.client.0.smithi112.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-05-08T05:58:11.373 INFO:tasks.workunit.client.0.smithi112.stdout:Seed set to 1 2024-05-08T05:58:11.407 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-05-08T05:58:11.407 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-05-08T05:58:11.411 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-05-08T05:58:11.411 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-05-08T05:58:11.411 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support clone range, disabling! 2024-05-08T05:58:11.416 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support dedupe range, disabling! 2024-05-08T05:58:11.421 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support exchange range, disabling! 2024-05-08T05:58:11.421 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero size read 2024-05-08T05:58:11.422 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero size read 2024-05-08T05:58:11.422 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x81d54 2024-05-08T05:58:11.434 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xb86d7 2024-05-08T05:58:11.451 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xe627e 2024-05-08T05:58:11.509 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0xf3194 2024-05-08T05:58:11.536 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xf3ceb 2024-05-08T05:58:11.544 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0xf47e4 2024-05-08T05:58:11.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:11 smithi112 ceph-mon[30003]: pgmap v527: 97 pgs: 97 active+clean; 57 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 564 KiB/s wr, 53 op/s 2024-05-08T05:58:11.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:11 smithi112 ceph-mon[30003]: scrub queued for path: / 2024-05-08T05:58:11.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:11 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:58:11.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:11 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T05:58:11.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:11 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T05:58:11.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:11 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T05:58:11.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:11 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:58:11.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:11 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:58:11.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:11 smithi162 ceph-mon[34800]: pgmap v527: 97 pgs: 97 active+clean; 57 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 564 KiB/s wr, 53 op/s 2024-05-08T05:58:11.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:11 smithi162 ceph-mon[34800]: scrub queued for path: / 2024-05-08T05:58:11.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:11 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:58:11.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:11 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T05:58:11.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:11 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T05:58:11.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:11 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T05:58:11.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:11 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:58:11.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:11 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:58:11.666 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xfaccc 2024-05-08T05:58:11.670 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:58:11.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:11 smithi150 ceph-mon[34315]: pgmap v527: 97 pgs: 97 active+clean; 57 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 564 KiB/s wr, 53 op/s 2024-05-08T05:58:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:11 smithi150 ceph-mon[34315]: scrub queued for path: / 2024-05-08T05:58:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:11 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:58:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:11 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T05:58:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:11 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T05:58:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:11 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T05:58:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:11 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:58:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:11 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:58:12.169 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xfe423 2024-05-08T05:58:12.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:12 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:58:12.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:12 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:58:12.685 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:12 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:58:13.456 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:13 smithi112 ceph-mon[30003]: pgmap v528: 97 pgs: 97 active+clean; 58 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 427 KiB/s wr, 39 op/s 2024-05-08T05:58:13.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:13 smithi162 ceph-mon[34800]: pgmap v528: 97 pgs: 97 active+clean; 58 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 427 KiB/s wr, 39 op/s 2024-05-08T05:58:13.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:13 smithi150 ceph-mon[34315]: pgmap v528: 97 pgs: 97 active+clean; 58 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 427 KiB/s wr, 39 op/s 2024-05-08T05:58:14.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.187+0000 7fe705e00700 1 -- 172.21.15.112:0/4050018596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe70006bad0 msgr2=0x7fe70010fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:14.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.187+0000 7fe705e00700 1 --2- 172.21.15.112:0/4050018596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe70006bad0 0x7fe70010fa10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009990 tx=0x7fe6ec009ca0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.187+0000 7fe705e00700 1 -- 172.21.15.112:0/4050018596 shutdown_connections 2024-05-08T05:58:14.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.187+0000 7fe705e00700 1 --2- 172.21.15.112:0/4050018596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe70010ff50 0x7fe700112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.187+0000 7fe705e00700 1 --2- 172.21.15.112:0/4050018596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70006b1b0 0x7fe70006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.187+0000 7fe705e00700 1 --2- 172.21.15.112:0/4050018596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe70006bad0 0x7fe70010fa10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.187+0000 7fe705e00700 1 -- 172.21.15.112:0/4050018596 >> 172.21.15.112:0/4050018596 conn(0x7fe700079350 msgr2=0x7fe700079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:14.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.187+0000 7fe705e00700 1 -- 172.21.15.112:0/4050018596 shutdown_connections 2024-05-08T05:58:14.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.187+0000 7fe705e00700 1 -- 172.21.15.112:0/4050018596 wait complete. 2024-05-08T05:58:14.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.189+0000 7fe705e00700 1 Processor -- start 2024-05-08T05:58:14.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.189+0000 7fe705e00700 1 -- start start 2024-05-08T05:58:14.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.189+0000 7fe705e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe70006b1b0 0x7fe700074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.189+0000 7fe705e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe70006bad0 0x7fe70006de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.189+0000 7fe705e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70010ff50 0x7fe70006e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.189+0000 7fe705e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7001158b0 con 0x7fe70006b1b0 2024-05-08T05:58:14.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.189+0000 7fe705e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe700115730 con 0x7fe70006bad0 2024-05-08T05:58:14.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.190+0000 7fe705e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe700115a30 con 0x7fe70010ff50 2024-05-08T05:58:14.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.190+0000 7fe6fea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe70006b1b0 0x7fe700074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.190+0000 7fe6fea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe70006b1b0 0x7fe700074d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60796/0 (socket says 172.21.15.112:60796) 2024-05-08T05:58:14.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.190+0000 7fe6fea00700 1 -- 172.21.15.112:0/3993546806 learned_addr learned my addr 172.21.15.112:0/3993546806 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:14.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.190+0000 7fe6ff400700 1 --2- 172.21.15.112:0/3993546806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70010ff50 0x7fe70006e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.190+0000 7fe6ff400700 1 -- 172.21.15.112:0/3993546806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe70006bad0 msgr2=0x7fe70006de40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T05:58:14.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.190+0000 7fe6ff400700 1 --2- 172.21.15.112:0/3993546806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe70006bad0 0x7fe70006de40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.190+0000 7fe6ff400700 1 -- 172.21.15.112:0/3993546806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe70006b1b0 msgr2=0x7fe700074d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:14.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.191+0000 7fe6ff400700 1 --2- 172.21.15.112:0/3993546806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe70006b1b0 0x7fe700074d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.191+0000 7fe6ff400700 1 -- 172.21.15.112:0/3993546806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70006ea20 con 0x7fe70010ff50 2024-05-08T05:58:14.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.191+0000 7fe6fea00700 1 --2- 172.21.15.112:0/3993546806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe70006b1b0 0x7fe700074d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:58:14.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.191+0000 7fe6ff400700 1 --2- 172.21.15.112:0/3993546806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70010ff50 0x7fe70006e380 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800ef90 tx=0x7fe6f800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:14.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.191+0000 7fe704a00700 1 -- 172.21.15.112:0/3993546806 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80092e0 con 0x7fe70010ff50 2024-05-08T05:58:14.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.191+0000 7fe704a00700 1 -- 172.21.15.112:0/3993546806 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f8004bb0 con 0x7fe70010ff50 2024-05-08T05:58:14.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.191+0000 7fe704a00700 1 -- 172.21.15.112:0/3993546806 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8004280 con 0x7fe70010ff50 2024-05-08T05:58:14.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.191+0000 7fe705e00700 1 -- 172.21.15.112:0/3993546806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe700072840 con 0x7fe70010ff50 2024-05-08T05:58:14.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.191+0000 7fe705e00700 1 -- 172.21.15.112:0/3993546806 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe700072d30 con 0x7fe70010ff50 2024-05-08T05:58:14.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.195+0000 7fe704a00700 1 -- 172.21.15.112:0/3993546806 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe6f801b030 con 0x7fe70010ff50 2024-05-08T05:58:14.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.197+0000 7fe7107b9b80 1 Processor -- start 2024-05-08T05:58:14.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.197+0000 7fe704a00700 1 --2- 172.21.15.112:0/3993546806 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6e40b2980 0x7fe6e40b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.197+0000 7fe7107b9b80 1 -- start start 2024-05-08T05:58:14.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.197+0000 7fe704a00700 1 -- 172.21.15.112:0/3993546806 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fe6f80e1c90 con 0x7fe70010ff50 2024-05-08T05:58:14.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.197+0000 7fe7107b9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072d6870 0x562f072d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.197+0000 7fe7107b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072d7190 0x562f072daca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.197+0000 7fe7107b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072db1e0 0x562f072db640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.197+0000 7fe7107b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562f07220a20 con 0x562f072d7190 2024-05-08T05:58:14.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.197+0000 7fe7107b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562f071be760 con 0x562f072db1e0 2024-05-08T05:58:14.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.197+0000 7fe7107b9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562f0721c9e0 con 0x562f072d6870 2024-05-08T05:58:14.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.198+0000 7fe6fea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072d6870 0x562f072d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.198+0000 7fe6fea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072d6870 0x562f072d6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60276/0 (socket says 172.21.15.112:60276) 2024-05-08T05:58:14.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.198+0000 7fe6fea00700 1 -- 172.21.15.112:0/3465736274 learned_addr learned my addr 172.21.15.112:0/3465736274 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:14.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.198+0000 7fe6fe000700 1 --2- 172.21.15.112:0/3993546806 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6e40b2980 0x7fe6e40b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.198+0000 7fe6ff400700 1 --2- 172.21.15.112:0/3465736274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072db1e0 0x562f072db640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.198+0000 7fe6fea00700 1 -- 172.21.15.112:0/3465736274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072db1e0 msgr2=0x562f072db640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:14.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.198+0000 7fe6fea00700 1 --2- 172.21.15.112:0/3465736274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072db1e0 0x562f072db640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.198+0000 7fe6fea00700 1 -- 172.21.15.112:0/3465736274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072d7190 msgr2=0x562f072daca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:58:14.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.198+0000 7fe6fea00700 1 --2- 172.21.15.112:0/3465736274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072d7190 0x562f072daca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.198+0000 7fe6fea00700 1 -- 172.21.15.112:0/3465736274 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f0716d8d0 con 0x562f072d6870 2024-05-08T05:58:14.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.198+0000 7fe6fea00700 1 --2- 172.21.15.112:0/3465736274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072d6870 0x562f072d6c50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00117e0 tx=0x7fe6f0011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edaec657a407e1bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:14.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.199+0000 7fe6f4c00700 1 -- 172.21.15.112:0/3465736274 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f001e070 con 0x562f072d6870 2024-05-08T05:58:14.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.199+0000 7fe6f4c00700 1 -- 172.21.15.112:0/3465736274 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f000f930 con 0x562f072d6870 2024-05-08T05:58:14.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.199+0000 7fe6f4c00700 1 -- 172.21.15.112:0/3465736274 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0019ab0 con 0x562f072d6870 2024-05-08T05:58:14.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.200+0000 7fe7107b9b80 1 -- 172.21.15.112:0/3465736274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072d6870 msgr2=0x562f072d6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:14.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.200+0000 7fe6fe000700 1 --2- 172.21.15.112:0/3993546806 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6e40b2980 0x7fe6e40b4e40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec006010 tx=0x7fe6ec00b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:14.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.200+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/3465736274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072d6870 0x562f072d6c50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00117e0 tx=0x7fe6f0011af0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.200+0000 7fe7107b9b80 1 -- 172.21.15.112:0/3465736274 shutdown_connections 2024-05-08T05:58:14.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.200+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/3465736274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072db1e0 0x562f072db640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.200+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/3465736274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072d6870 0x562f072d6c50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.200+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/3465736274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072d7190 0x562f072daca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.200+0000 7fe7107b9b80 1 -- 172.21.15.112:0/3465736274 >> 172.21.15.112:0/3465736274 conn(0x562f071fcb60 msgr2=0x562f072c87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:14.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.200+0000 7fe7107b9b80 1 -- 172.21.15.112:0/3465736274 shutdown_connections 2024-05-08T05:58:14.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.200+0000 7fe7107b9b80 1 -- 172.21.15.112:0/3465736274 wait complete. 2024-05-08T05:58:14.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.201+0000 7fe7107b9b80 1 -- start start 2024-05-08T05:58:14.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.201+0000 7fe7107b9b80 1 Processor -- start 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.202+0000 7fe7107b9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072d6870 0x562f072e2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.202+0000 7fe7107b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072d7190 0x562f072df9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.202+0000 7fe7107b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072db1e0 0x562f072dff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.202+0000 7fe7107b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562f072e33f0 con 0x562f072db1e0 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.202+0000 7fe7107b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562f07186b20 con 0x562f072d7190 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.202+0000 7fe7107b9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562f072e3570 con 0x562f072d6870 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.202+0000 7fe7107b9b80 10 client.0 _ll_register_callbacks cb 0x562f071c7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.203+0000 7fe6fea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072d6870 0x562f072e2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.203+0000 7fe6fe000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072d7190 0x562f072df9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.203+0000 7fe6ff400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072db1e0 0x562f072dff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.203+0000 7fe6fe000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072d7190 0x562f072df9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:58966/0 (socket says 172.21.15.112:58966) 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.203+0000 7fe6fe000700 1 -- 172.21.15.112:0/1502858550 learned_addr learned my addr 172.21.15.112:0/1502858550 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.203+0000 7fe6fe000700 1 -- 172.21.15.112:0/1502858550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072d6870 msgr2=0x562f072e2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.203+0000 7fe6fe000700 1 --2- 172.21.15.112:0/1502858550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072d6870 0x562f072e2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.204+0000 7fe6fe000700 1 -- 172.21.15.112:0/1502858550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072db1e0 msgr2=0x562f072dff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:14.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.204+0000 7fe6fe000700 1 --2- 172.21.15.112:0/1502858550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072db1e0 0x562f072dff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.204+0000 7fe6fe000700 1 -- 172.21.15.112:0/1502858550 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f071e6420 con 0x562f072d7190 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.204+0000 7fe6fe000700 1 --2- 172.21.15.112:0/1502858550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072d7190 0x562f072df9f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009990 tx=0x7fe6ec009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6c60598aa1eb87e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.204+0000 7fe6e9600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.204+0000 7fe6e9600700 1 -- 172.21.15.112:0/1502858550 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec003a50 con 0x562f072d7190 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.205+0000 7fe6e9600700 1 -- 172.21.15.112:0/1502858550 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fe6ec01e030 con 0x562f072d7190 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.205+0000 7fe7107b9b80 1 -- 172.21.15.112:0/1502858550 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6f000f930 con 0x562f072d7190 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.205+0000 7fe6e9600700 1 -- 172.21.15.112:0/1502858550 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6ec003bf0 con 0x562f072d7190 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.205+0000 7fe7107b9b80 10 client.25493 fetch_fsmap learned FSMap version 29 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.205+0000 7fe7107b9b80 1 -- 172.21.15.112:0/1502858550 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562f072edc80 con 0x562f072d7190 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.206+0000 7fe6e9600700 1 -- 172.21.15.112:0/1502858550 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f000f930 con 0x562f072d7190 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.206+0000 7fe6e9600700 1 -- 172.21.15.112:0/1502858550 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fe6ec021410 con 0x562f072d7190 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.206+0000 7fe7107b9b80 10 client.25493 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.206+0000 7fe7107b9b80 10 client.25493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.206+0000 7fe7107b9b80 20 client.25493 populate_metadata read hostname 'smithi112' 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.206+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/1502858550 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562f072f02c0 0x562f072f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.206+0000 7fe7107b9b80 4 client.25493 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.206+0000 7fe7107b9b80 1 -- 172.21.15.112:0/1502858550 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f0729b0a0 con 0x562f072f02c0 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.206+0000 7fe6fea00700 1 --2- 172.21.15.112:0/1502858550 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562f072f02c0 0x562f072f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.207+0000 7fe6fea00700 1 --2- 172.21.15.112:0/1502858550 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562f072f02c0 0x562f072f26d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=867fc66d437b7803 server_cookie=644f11c6f4f5a7a9 in_seq=0 out_seq=0 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.207+0000 7fe6e9600700 10 client.25493 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.208+0000 7fe6e9600700 1 -- 172.21.15.112:0/1502858550 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562f0729b0a0 con 0x562f072f02c0 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.208+0000 7fe6e9600700 10 client.25493 handle_command_reply: tid=0 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.208+0000 7fe7107b9b80 1 client.25493 shutdown 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.208+0000 7fe7107b9b80 1 -- 172.21.15.112:0/1502858550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072d7190 msgr2=0x562f072df9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.208+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/1502858550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072d7190 0x562f072df9f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009990 tx=0x7fe6ec009ca0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.208+0000 7fe7107b9b80 1 -- 172.21.15.112:0/1502858550 shutdown_connections 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.208+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/1502858550 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562f072f02c0 0x562f072f26d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.209+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/1502858550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072db1e0 0x562f072dff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.209+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/1502858550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072d6870 0x562f072e2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.209+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/1502858550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072d7190 0x562f072df9f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.209+0000 7fe6e9600700 0 client.25493 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:14.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.209+0000 7fe7107b9b80 1 -- 172.21.15.112:0/1502858550 >> 172.21.15.112:0/1502858550 conn(0x562f071fcb60 msgr2=0x562f072c32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:14.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.209+0000 7fe7107b9b80 1 -- 172.21.15.112:0/1502858550 shutdown_connections 2024-05-08T05:58:14.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.209+0000 7fe7107b9b80 1 -- 172.21.15.112:0/1502858550 wait complete. 2024-05-08T05:58:14.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.209+0000 7fe7107b9b80 20 client.25493 trim_cache size 0 max 16384 2024-05-08T05:58:14.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.241+0000 7fe7107b9b80 1 Processor -- start 2024-05-08T05:58:14.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.242+0000 7fe7107b9b80 1 -- start start 2024-05-08T05:58:14.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.242+0000 7fe7107b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072d6870 0x562f072e0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.242+0000 7fe7107b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072d7190 0x562f072c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.242+0000 7fe7107b9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072db1e0 0x562f072f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.242+0000 7fe7107b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e000fd80 con 0x562f072d6870 2024-05-08T05:58:14.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.242+0000 7fe7107b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e000d580 con 0x562f072d7190 2024-05-08T05:58:14.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.242+0000 7fe7107b9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e000b3b0 con 0x562f072db1e0 2024-05-08T05:58:14.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.242+0000 7fe6fe000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072d7190 0x562f072c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.242+0000 7fe6fe000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072d7190 0x562f072c3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:58970/0 (socket says 172.21.15.112:58970) 2024-05-08T05:58:14.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.242+0000 7fe6fe000700 1 -- 172.21.15.112:0/2904855008 learned_addr learned my addr 172.21.15.112:0/2904855008 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:14.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.242+0000 7fe6fea00700 1 --2- 172.21.15.112:0/2904855008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072d6870 0x562f072e0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.243+0000 7fe6ff400700 1 --2- 172.21.15.112:0/2904855008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072db1e0 0x562f072f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.243+0000 7fe6fe000700 1 -- 172.21.15.112:0/2904855008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072db1e0 msgr2=0x562f072f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:14.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.243+0000 7fe6fe000700 1 --2- 172.21.15.112:0/2904855008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072db1e0 0x562f072f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.243+0000 7fe6fe000700 1 -- 172.21.15.112:0/2904855008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072d6870 msgr2=0x562f072e0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:14.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.243+0000 7fe6fe000700 1 --2- 172.21.15.112:0/2904855008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072d6870 0x562f072e0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.243+0000 7fe6fe000700 1 -- 172.21.15.112:0/2904855008 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f072e29c0 con 0x562f072d7190 2024-05-08T05:58:14.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.243+0000 7fe6fe000700 1 --2- 172.21.15.112:0/2904855008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072d7190 0x562f072c3130 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec005850 tx=0x7fe6ec005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593ce51f74dabbbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:14.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.244+0000 7fe6ebe00700 1 -- 172.21.15.112:0/2904855008 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec003d90 con 0x562f072d7190 2024-05-08T05:58:14.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.244+0000 7fe6ebe00700 1 -- 172.21.15.112:0/2904855008 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6ec0213f0 con 0x562f072d7190 2024-05-08T05:58:14.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.244+0000 7fe6ebe00700 1 -- 172.21.15.112:0/2904855008 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec042900 con 0x562f072d7190 2024-05-08T05:58:14.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.245+0000 7fe7107b9b80 1 -- 172.21.15.112:0/2904855008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072d7190 msgr2=0x562f072c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:14.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.245+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/2904855008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072d7190 0x562f072c3130 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec005850 tx=0x7fe6ec005f10 comp rx=0 tx=0).stop 2024-05-08T05:58:14.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.245+0000 7fe7107b9b80 1 -- 172.21.15.112:0/2904855008 shutdown_connections 2024-05-08T05:58:14.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.245+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/2904855008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072db1e0 0x562f072f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.245+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/2904855008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072d6870 0x562f072e0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.245+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/2904855008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072d7190 0x562f072c3130 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec005850 tx=0x7fe6ec005f10 comp rx=0 tx=0).stop 2024-05-08T05:58:14.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.245+0000 7fe7107b9b80 1 -- 172.21.15.112:0/2904855008 >> 172.21.15.112:0/2904855008 conn(0x562f071fcb60 msgr2=0x562f072c2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:14.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.246+0000 7fe7107b9b80 1 -- 172.21.15.112:0/2904855008 shutdown_connections 2024-05-08T05:58:14.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.246+0000 7fe7107b9b80 1 -- 172.21.15.112:0/2904855008 wait complete. 2024-05-08T05:58:14.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.246+0000 7fe7107b9b80 1 -- start start 2024-05-08T05:58:14.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.247+0000 7fe7107b9b80 1 Processor -- start 2024-05-08T05:58:14.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.247+0000 7fe7107b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072d6870 0x562f072e6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.247+0000 7fe7107b9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072db1e0 0x562f072e6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.247+0000 7fe7107b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072e7090 0x562f07359ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.247+0000 7fe7107b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e000ba40 con 0x562f072d6870 2024-05-08T05:58:14.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.247+0000 7fe7107b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e000ceb0 con 0x562f072e7090 2024-05-08T05:58:14.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.247+0000 7fe7107b9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e000c110 con 0x562f072db1e0 2024-05-08T05:58:14.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.247+0000 7fe7107b9b80 10 client.0 _ll_register_callbacks cb 0x562f071c7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:58:14.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.248+0000 7fe6fe000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072db1e0 0x562f072e6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.248+0000 7fe6fe000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072db1e0 0x562f072e6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60300/0 (socket says 172.21.15.112:60300) 2024-05-08T05:58:14.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.248+0000 7fe6fe000700 1 -- 172.21.15.112:0/2062569078 learned_addr learned my addr 172.21.15.112:0/2062569078 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:14.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.248+0000 7fe6fea00700 1 --2- 172.21.15.112:0/2062569078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072d6870 0x562f072e6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.248+0000 7fe6fe000700 1 -- 172.21.15.112:0/2062569078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072e7090 msgr2=0x562f07359ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.248+0000 7fe6fe000700 1 --2- 172.21.15.112:0/2062569078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072e7090 0x562f07359ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.248+0000 7fe6fe000700 1 -- 172.21.15.112:0/2062569078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072d6870 msgr2=0x562f072e6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.248+0000 7fe6fe000700 1 --2- 172.21.15.112:0/2062569078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072d6870 0x562f072e6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.248+0000 7fe6fe000700 1 -- 172.21.15.112:0/2062569078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f0735a220 con 0x562f072db1e0 2024-05-08T05:58:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.249+0000 7fe6fe000700 1 --2- 172.21.15.112:0/2062569078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072db1e0 0x562f072e6b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009990 tx=0x7fe6ec005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a44671cd88a6473 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.249+0000 7fe6e9600700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:58:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.249+0000 7fe6e9600700 1 -- 172.21.15.112:0/2062569078 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec026070 con 0x562f072db1e0 2024-05-08T05:58:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.249+0000 7fe6e9600700 1 -- 172.21.15.112:0/2062569078 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fe6ec01e040 con 0x562f072db1e0 2024-05-08T05:58:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.249+0000 7fe7107b9b80 1 -- 172.21.15.112:0/2062569078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6ec0213f0 con 0x562f072db1e0 2024-05-08T05:58:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.249+0000 7fe6e9600700 1 -- 172.21.15.112:0/2062569078 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6ec051050 con 0x562f072db1e0 2024-05-08T05:58:14.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.250+0000 7fe7107b9b80 10 client.25531 fetch_fsmap learned FSMap version 29 2024-05-08T05:58:14.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.250+0000 7fe7107b9b80 1 -- 172.21.15.112:0/2062569078 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562f0735b4f0 con 0x562f072db1e0 2024-05-08T05:58:14.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.250+0000 7fe6e9600700 1 -- 172.21.15.112:0/2062569078 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6ec0213f0 con 0x562f072db1e0 2024-05-08T05:58:14.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.250+0000 7fe6e9600700 1 -- 172.21.15.112:0/2062569078 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fe6ec02f540 con 0x562f072db1e0 2024-05-08T05:58:14.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.250+0000 7fe7107b9b80 10 client.25531 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:58:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.250+0000 7fe7107b9b80 10 client.25531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:58:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.250+0000 7fe7107b9b80 20 client.25531 populate_metadata read hostname 'smithi112' 2024-05-08T05:58:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.250+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/2062569078 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562f0735bae0 0x562f0735df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.250+0000 7fe7107b9b80 4 client.25531 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:58:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.250+0000 7fe7107b9b80 1 -- 172.21.15.112:0/2062569078 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562f0722e400 con 0x562f0735bae0 2024-05-08T05:58:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.251+0000 7fe6fea00700 1 --2- 172.21.15.112:0/2062569078 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562f0735bae0 0x562f0735df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.252+0000 7fe6e9600700 10 client.25531 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.252+0000 7fe6fea00700 1 --2- 172.21.15.112:0/2062569078 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562f0735bae0 0x562f0735df00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4baf207c7e406db8 server_cookie=b9cf339f3d525664 in_seq=0 out_seq=0 2024-05-08T05:58:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.253+0000 7fe6e9600700 1 -- 172.21.15.112:0/2062569078 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562f0722e400 con 0x562f0735bae0 2024-05-08T05:58:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.253+0000 7fe6e9600700 10 client.25531 handle_command_reply: tid=0 2024-05-08T05:58:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.253+0000 7fe7107b9b80 1 client.25531 shutdown 2024-05-08T05:58:14.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.253+0000 7fe7107b9b80 1 -- 172.21.15.112:0/2062569078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072db1e0 msgr2=0x562f072e6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:14.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.253+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/2062569078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072db1e0 0x562f072e6b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009990 tx=0x7fe6ec005d50 comp rx=0 tx=0).stop 2024-05-08T05:58:14.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.253+0000 7fe7107b9b80 1 -- 172.21.15.112:0/2062569078 shutdown_connections 2024-05-08T05:58:14.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.253+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/2062569078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f072e7090 0x562f07359ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.253+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/2062569078 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562f0735bae0 0x562f0735df00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.253+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/2062569078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f072d6870 0x562f072e6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.253+0000 7fe7107b9b80 1 --2- 172.21.15.112:0/2062569078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f072db1e0 0x562f072e6b50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.253+0000 7fe7107b9b80 1 -- 172.21.15.112:0/2062569078 >> 172.21.15.112:0/2062569078 conn(0x562f071fcb60 msgr2=0x562f072c32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.254+0000 7fe6e9600700 0 client.25531 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.254+0000 7fe7107b9b80 1 -- 172.21.15.112:0/2062569078 shutdown_connections 2024-05-08T05:58:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.254+0000 7fe7107b9b80 1 -- 172.21.15.112:0/2062569078 wait complete. 2024-05-08T05:58:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.254+0000 7fe7107b9b80 20 client.25531 trim_cache size 0 max 16384 2024-05-08T05:58:14.257 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:58:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T05:58:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-05-08T05:58:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "45688eea-deb8-4e01-a574-5e050c80e33f": { 2024-05-08T05:58:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-05-08T05:58:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "45688eea-deb8-4e01-a574-5e050c80e33f", 2024-05-08T05:58:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-05-08T05:58:14.257 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T05:58:14.258 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T05:58:14.258 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:58:14.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.257+0000 7fe705e00700 1 -- 172.21.15.112:0/3993546806 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6e40b2980 msgr2=0x7fe6e40b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.257+0000 7fe705e00700 1 --2- 172.21.15.112:0/3993546806 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6e40b2980 0x7fe6e40b4e40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec006010 tx=0x7fe6ec00b610 comp rx=0 tx=0).stop 2024-05-08T05:58:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.257+0000 7fe705e00700 1 -- 172.21.15.112:0/3993546806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70010ff50 msgr2=0x7fe70006e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.257+0000 7fe705e00700 1 --2- 172.21.15.112:0/3993546806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70010ff50 0x7fe70006e380 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800ef90 tx=0x7fe6f800c3a0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.258+0000 7fe705e00700 1 -- 172.21.15.112:0/3993546806 shutdown_connections 2024-05-08T05:58:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.258+0000 7fe705e00700 1 --2- 172.21.15.112:0/3993546806 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe6e40b2980 0x7fe6e40b4e40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.258+0000 7fe705e00700 1 --2- 172.21.15.112:0/3993546806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70010ff50 0x7fe70006e380 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.258+0000 7fe705e00700 1 --2- 172.21.15.112:0/3993546806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe70006b1b0 0x7fe700074d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.258+0000 7fe705e00700 1 --2- 172.21.15.112:0/3993546806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe70006bad0 0x7fe70006de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.258+0000 7fe705e00700 1 -- 172.21.15.112:0/3993546806 >> 172.21.15.112:0/3993546806 conn(0x7fe700079350 msgr2=0x7fe7001114a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.258+0000 7fe705e00700 1 -- 172.21.15.112:0/3993546806 shutdown_connections 2024-05-08T05:58:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:14.259+0000 7fe705e00700 1 -- 172.21.15.112:0/3993546806 wait complete. 2024-05-08T05:58:15.163 INFO:tasks.cephfs.filesystem:scrub status for tag:45688eea-deb8-4e01-a574-5e050c80e33f - {'path': '/', 'tag': '45688eea-deb8-4e01-a574-5e050c80e33f', 'options': 'recursive,force'} 2024-05-08T05:58:15.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:15 smithi112 ceph-mon[30003]: pgmap v529: 97 pgs: 97 active+clean; 59 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 965 KiB/s wr, 98 op/s 2024-05-08T05:58:15.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:15 smithi162 ceph-mon[34800]: pgmap v529: 97 pgs: 97 active+clean; 59 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 965 KiB/s wr, 98 op/s 2024-05-08T05:58:15.626 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xfe492 2024-05-08T05:58:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:15 smithi150 ceph-mon[34315]: pgmap v529: 97 pgs: 97 active+clean; 59 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 965 KiB/s wr, 98 op/s 2024-05-08T05:58:17.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:17 smithi112 ceph-mon[30003]: pgmap v530: 97 pgs: 97 active+clean; 63 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-08T05:58:17.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:17 smithi162 ceph-mon[34800]: pgmap v530: 97 pgs: 97 active+clean; 63 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-08T05:58:17.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:17 smithi150 ceph-mon[34315]: pgmap v530: 97 pgs: 97 active+clean; 63 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-08T05:58:19.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:19 smithi112 ceph-mon[30003]: pgmap v531: 97 pgs: 97 active+clean; 63 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-05-08T05:58:19.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:19 smithi162 ceph-mon[34800]: pgmap v531: 97 pgs: 97 active+clean; 63 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-05-08T05:58:19.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:19 smithi150 ceph-mon[34315]: pgmap v531: 97 pgs: 97 active+clean; 63 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2024-05-08T05:58:20.965 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xff038 2024-05-08T05:58:21.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:21 smithi112 ceph-mon[30003]: pgmap v532: 97 pgs: 97 active+clean; 65 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 300 op/s 2024-05-08T05:58:21.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:21 smithi162 ceph-mon[34800]: pgmap v532: 97 pgs: 97 active+clean; 65 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 300 op/s 2024-05-08T05:58:21.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:21 smithi150 ceph-mon[34315]: pgmap v532: 97 pgs: 97 active+clean; 65 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 300 op/s 2024-05-08T05:58:23.303 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0xffcac 2024-05-08T05:58:23.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:23 smithi112 ceph-mon[30003]: pgmap v533: 97 pgs: 97 active+clean; 65 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 288 op/s 2024-05-08T05:58:23.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:23 smithi162 ceph-mon[34800]: pgmap v533: 97 pgs: 97 active+clean; 65 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 288 op/s 2024-05-08T05:58:23.612 INFO:tasks.workunit.client.0.smithi112.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-05-08T05:58:23.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:23 smithi150 ceph-mon[34315]: pgmap v533: 97 pgs: 97 active+clean; 65 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 288 op/s 2024-05-08T05:58:24.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:24 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T05:58:24.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:24 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T05:58:24.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:24 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T05:58:24.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:24 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T05:58:24.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:24 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T05:58:24.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:24 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T05:58:24.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:24 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T05:58:24.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:24 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T05:58:24.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:24 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T05:58:25.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:25 smithi112 ceph-mon[30003]: pgmap v534: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 360 op/s 2024-05-08T05:58:25.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:25 smithi162 ceph-mon[34800]: pgmap v534: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 360 op/s 2024-05-08T05:58:25.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:25 smithi150 ceph-mon[34315]: pgmap v534: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 360 op/s 2024-05-08T05:58:27.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:27 smithi112 ceph-mon[30003]: pgmap v535: 97 pgs: 97 active+clean; 65 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 325 op/s 2024-05-08T05:58:27.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:27 smithi162 ceph-mon[34800]: pgmap v535: 97 pgs: 97 active+clean; 65 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 325 op/s 2024-05-08T05:58:27.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:27 smithi150 ceph-mon[34315]: pgmap v535: 97 pgs: 97 active+clean; 65 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 325 op/s 2024-05-08T05:58:29.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:29 smithi112 ceph-mon[30003]: pgmap v536: 97 pgs: 97 active+clean; 66 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 305 op/s 2024-05-08T05:58:29.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:29 smithi162 ceph-mon[34800]: pgmap v536: 97 pgs: 97 active+clean; 66 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 305 op/s 2024-05-08T05:58:29.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:29 smithi150 ceph-mon[34315]: pgmap v536: 97 pgs: 97 active+clean; 66 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 305 op/s 2024-05-08T05:58:30.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:30 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:30.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:30 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:30.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:30 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:31.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:31 smithi112 ceph-mon[30003]: pgmap v537: 97 pgs: 97 active+clean; 66 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 386 op/s 2024-05-08T05:58:31.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:31.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:31.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:58:31.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:31 smithi162 ceph-mon[34800]: pgmap v537: 97 pgs: 97 active+clean; 66 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 386 op/s 2024-05-08T05:58:31.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:31.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:31.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:58:31.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:31 smithi150 ceph-mon[34315]: pgmap v537: 97 pgs: 97 active+clean; 66 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 386 op/s 2024-05-08T05:58:31.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:31.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:31.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:58:33.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:33 smithi112 ceph-mon[30003]: pgmap v538: 97 pgs: 97 active+clean; 67 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 268 op/s 2024-05-08T05:58:33.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:33 smithi162 ceph-mon[34800]: pgmap v538: 97 pgs: 97 active+clean; 67 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 268 op/s 2024-05-08T05:58:33.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:33 smithi150 ceph-mon[34315]: pgmap v538: 97 pgs: 97 active+clean; 67 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 268 op/s 2024-05-08T05:58:35.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:35 smithi112 ceph-mon[30003]: pgmap v539: 97 pgs: 97 active+clean; 67 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 352 op/s 2024-05-08T05:58:35.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:35 smithi162 ceph-mon[34800]: pgmap v539: 97 pgs: 97 active+clean; 67 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 352 op/s 2024-05-08T05:58:35.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:35 smithi150 ceph-mon[34315]: pgmap v539: 97 pgs: 97 active+clean; 67 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 352 op/s 2024-05-08T05:58:37.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:37 smithi112 ceph-mon[30003]: pgmap v540: 97 pgs: 97 active+clean; 68 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 279 op/s 2024-05-08T05:58:37.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:37 smithi162 ceph-mon[34800]: pgmap v540: 97 pgs: 97 active+clean; 68 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 279 op/s 2024-05-08T05:58:37.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:37 smithi150 ceph-mon[34315]: pgmap v540: 97 pgs: 97 active+clean; 68 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 279 op/s 2024-05-08T05:58:39.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:39 smithi112 ceph-mon[30003]: pgmap v541: 97 pgs: 97 active+clean; 69 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 294 op/s 2024-05-08T05:58:39.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:39 smithi162 ceph-mon[34800]: pgmap v541: 97 pgs: 97 active+clean; 69 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 294 op/s 2024-05-08T05:58:39.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:39 smithi150 ceph-mon[34315]: pgmap v541: 97 pgs: 97 active+clean; 69 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 294 op/s 2024-05-08T05:58:41.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:41 smithi112 ceph-mon[30003]: pgmap v542: 97 pgs: 97 active+clean; 69 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 332 op/s 2024-05-08T05:58:41.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:41 smithi162 ceph-mon[34800]: pgmap v542: 97 pgs: 97 active+clean; 69 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 332 op/s 2024-05-08T05:58:41.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:41 smithi150 ceph-mon[34315]: pgmap v542: 97 pgs: 97 active+clean; 69 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 332 op/s 2024-05-08T05:58:43.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:43 smithi112 ceph-mon[30003]: pgmap v543: 97 pgs: 97 active+clean; 70 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 287 op/s 2024-05-08T05:58:43.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:43 smithi162 ceph-mon[34800]: pgmap v543: 97 pgs: 97 active+clean; 70 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 287 op/s 2024-05-08T05:58:43.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:43 smithi150 ceph-mon[34315]: pgmap v543: 97 pgs: 97 active+clean; 70 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 287 op/s 2024-05-08T05:58:45.165 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T05:58:45.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:45 smithi112 ceph-mon[30003]: pgmap v544: 97 pgs: 97 active+clean; 70 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 308 op/s 2024-05-08T05:58:45.658 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:58:45.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:45 smithi150 ceph-mon[34315]: pgmap v544: 97 pgs: 97 active+clean; 70 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 308 op/s 2024-05-08T05:58:45.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:45 smithi162 ceph-mon[34800]: pgmap v544: 97 pgs: 97 active+clean; 70 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 308 op/s 2024-05-08T05:58:47.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:47 smithi112 ceph-mon[30003]: pgmap v545: 97 pgs: 97 active+clean; 71 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-05-08T05:58:47.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:47 smithi150 ceph-mon[34315]: pgmap v545: 97 pgs: 97 active+clean; 71 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-05-08T05:58:47.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:47 smithi162 ceph-mon[34800]: pgmap v545: 97 pgs: 97 active+clean; 71 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-05-08T05:58:48.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.150+0000 7f590b000700 1 -- 172.21.15.112:0/1516916964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59041056b0 msgr2=0x7f5904105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:48.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.150+0000 7f590b000700 1 --2- 172.21.15.112:0/1516916964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59041056b0 0x7f5904105b30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009990 tx=0x7f58f0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.151+0000 7f590b000700 1 -- 172.21.15.112:0/1516916964 shutdown_connections 2024-05-08T05:58:48.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.151+0000 7f590b000700 1 --2- 172.21.15.112:0/1516916964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5904106070 0x7f5904112580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.151+0000 7f590b000700 1 --2- 172.21.15.112:0/1516916964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f590410b6b0 0x7f590410ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.151+0000 7f590b000700 1 --2- 172.21.15.112:0/1516916964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59041056b0 0x7f5904105b30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.151+0000 7f590b000700 1 -- 172.21.15.112:0/1516916964 >> 172.21.15.112:0/1516916964 conn(0x7f59041011a0 msgr2=0x7f59041035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:48.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.151+0000 7f590b000700 1 -- 172.21.15.112:0/1516916964 shutdown_connections 2024-05-08T05:58:48.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.151+0000 7f590b000700 1 -- 172.21.15.112:0/1516916964 wait complete. 2024-05-08T05:58:48.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.152+0000 7f590b000700 1 Processor -- start 2024-05-08T05:58:48.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.152+0000 7f590b000700 1 -- start start 2024-05-08T05:58:48.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.153+0000 7f590b000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59041056b0 0x7f590419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.153+0000 7f590b000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5904106070 0x7f590419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.153+0000 7f590b000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f590410b6b0 0x7f59041a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.153+0000 7f590b000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5904115940 con 0x7f59041056b0 2024-05-08T05:58:48.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.153+0000 7f590b000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59041157c0 con 0x7f5904106070 2024-05-08T05:58:48.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.153+0000 7f590b000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5904115ac0 con 0x7f590410b6b0 2024-05-08T05:58:48.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.153+0000 7f5903e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59041056b0 0x7f590419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.154+0000 7f5903e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59041056b0 0x7f590419bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43004/0 (socket says 172.21.15.112:43004) 2024-05-08T05:58:48.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.154+0000 7f5903e00700 1 -- 172.21.15.112:0/31510201 learned_addr learned my addr 172.21.15.112:0/31510201 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:48.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.154+0000 7f5909000700 1 --2- 172.21.15.112:0/31510201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f590410b6b0 0x7f59041a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.154+0000 7f5909000700 1 -- 172.21.15.112:0/31510201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5904106070 msgr2=0x7f590419c2d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T05:58:48.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.154+0000 7f5909000700 1 --2- 172.21.15.112:0/31510201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5904106070 0x7f590419c2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.154+0000 7f5909000700 1 -- 172.21.15.112:0/31510201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59041056b0 msgr2=0x7f590419bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:48.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.154+0000 7f5909000700 1 --2- 172.21.15.112:0/31510201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59041056b0 0x7f590419bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.154+0000 7f5909000700 1 -- 172.21.15.112:0/31510201 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59041a0da0 con 0x7f590410b6b0 2024-05-08T05:58:48.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.154+0000 7f5909000700 1 --2- 172.21.15.112:0/31510201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f590410b6b0 0x7f59041a0660 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f58fc007f10 tx=0x7f58fc00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:48.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.155+0000 7f5900c00700 1 -- 172.21.15.112:0/31510201 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc00ddd0 con 0x7f590410b6b0 2024-05-08T05:58:48.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.155+0000 7f590b000700 1 -- 172.21.15.112:0/31510201 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59041a1090 con 0x7f590410b6b0 2024-05-08T05:58:48.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.155+0000 7f5900c00700 1 -- 172.21.15.112:0/31510201 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58fc009e20 con 0x7f590410b6b0 2024-05-08T05:58:48.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.155+0000 7f5900c00700 1 -- 172.21.15.112:0/31510201 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc015400 con 0x7f590410b6b0 2024-05-08T05:58:48.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.155+0000 7f590b000700 1 -- 172.21.15.112:0/31510201 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59041a8910 con 0x7f590410b6b0 2024-05-08T05:58:48.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.159+0000 7f5900c00700 1 -- 172.21.15.112:0/31510201 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f58fc005020 con 0x7f590410b6b0 2024-05-08T05:58:48.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.161+0000 7f5900c00700 1 --2- 172.21.15.112:0/31510201 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f58e80b2900 0x7f58e80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.161+0000 7f5915922b80 1 Processor -- start 2024-05-08T05:58:48.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.161+0000 7f5915922b80 1 -- start start 2024-05-08T05:58:48.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.161+0000 7f5900c00700 1 -- 172.21.15.112:0/31510201 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f58fc0e1bd0 con 0x7f590410b6b0 2024-05-08T05:58:48.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.161+0000 7f5903e00700 1 --2- 172.21.15.112:0/31510201 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f58e80b2900 0x7f58e80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.161+0000 7f5915922b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 0x55b4f29a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.162+0000 7f5915922b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f29a34e0 0x55b4f299d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.162+0000 7f5915922b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f299d8f0 0x55b4f299dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.162+0000 7f5915922b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b4f28efa20 con 0x55b4f29a2bc0 2024-05-08T05:58:48.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.162+0000 7f5915922b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b4f288d760 con 0x55b4f299d8f0 2024-05-08T05:58:48.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.162+0000 7f5915922b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55b4f28eb9e0 con 0x55b4f29a34e0 2024-05-08T05:58:48.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.162+0000 7f5903e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 0x55b4f29a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.162+0000 7f5903e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 0x55b4f29a2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43012/0 (socket says 172.21.15.112:43012) 2024-05-08T05:58:48.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.162+0000 7f5903e00700 1 -- 172.21.15.112:0/2871917615 learned_addr learned my addr 172.21.15.112:0/2871917615 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:48.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.162+0000 7f5909000700 1 --2- 172.21.15.112:0/2871917615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f299d8f0 0x55b4f299dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.162+0000 7f5903400700 1 --2- 172.21.15.112:0/2871917615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f29a34e0 0x55b4f299d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.163+0000 7f5903e00700 1 -- 172.21.15.112:0/2871917615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f29a34e0 msgr2=0x55b4f299d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:48.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.163+0000 7f5903e00700 1 --2- 172.21.15.112:0/2871917615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f29a34e0 0x55b4f299d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.163+0000 7f5903e00700 1 -- 172.21.15.112:0/2871917615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f299d8f0 msgr2=0x55b4f299dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:48.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.163+0000 7f5903e00700 1 --2- 172.21.15.112:0/2871917615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f299d8f0 0x55b4f299dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.163+0000 7f5903e00700 1 -- 172.21.15.112:0/2871917615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4f283c8d0 con 0x55b4f29a2bc0 2024-05-08T05:58:48.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.163+0000 7f5903e00700 1 --2- 172.21.15.112:0/31510201 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f58e80b2900 0x7f58e80b4dc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f58f4007c60 tx=0x7f58f40106b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:48.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.163+0000 7f5903e00700 1 --2- 172.21.15.112:0/2871917615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 0x55b4f29a2fa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f58f4010da0 tx=0x7f58f401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a12a74db538d5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:48.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.164+0000 7f58f9600700 1 -- 172.21.15.112:0/2871917615 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4010e90 con 0x55b4f29a2bc0 2024-05-08T05:58:48.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.164+0000 7f58f9600700 1 -- 172.21.15.112:0/2871917615 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f4011440 con 0x55b4f29a2bc0 2024-05-08T05:58:48.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.164+0000 7f58f9600700 1 -- 172.21.15.112:0/2871917615 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f401e660 con 0x55b4f29a2bc0 2024-05-08T05:58:48.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.165+0000 7f5915922b80 1 -- 172.21.15.112:0/2871917615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 msgr2=0x55b4f29a2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:48.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.165+0000 7f5915922b80 1 --2- 172.21.15.112:0/2871917615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 0x55b4f29a2fa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f58f4010da0 tx=0x7f58f401b000 comp rx=0 tx=0).stop 2024-05-08T05:58:48.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.165+0000 7f5915922b80 1 -- 172.21.15.112:0/2871917615 shutdown_connections 2024-05-08T05:58:48.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.165+0000 7f5915922b80 1 --2- 172.21.15.112:0/2871917615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f299d8f0 0x55b4f299dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.165+0000 7f5915922b80 1 --2- 172.21.15.112:0/2871917615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 0x55b4f29a2fa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.165+0000 7f5915922b80 1 --2- 172.21.15.112:0/2871917615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f29a34e0 0x55b4f299d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.165+0000 7f5915922b80 1 -- 172.21.15.112:0/2871917615 >> 172.21.15.112:0/2871917615 conn(0x55b4f29156d0 msgr2=0x55b4f2997b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:48.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.165+0000 7f5915922b80 1 -- 172.21.15.112:0/2871917615 shutdown_connections 2024-05-08T05:58:48.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.165+0000 7f5915922b80 1 -- 172.21.15.112:0/2871917615 wait complete. 2024-05-08T05:58:48.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.166+0000 7f5915922b80 1 -- start start 2024-05-08T05:58:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.166+0000 7f5915922b80 1 Processor -- start 2024-05-08T05:58:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.166+0000 7f5915922b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 0x55b4f29a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5915922b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 0x55b4f29a6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5915922b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29a34e0 0x55b4f29a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5915922b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b4f29a9f20 con 0x55b4f29a2bc0 2024-05-08T05:58:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5915922b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b4f2855b20 con 0x55b4f29a34e0 2024-05-08T05:58:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5915922b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55b4f29aa0a0 con 0x55b4f299d8f0 2024-05-08T05:58:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5915922b80 10 client.0 _ll_register_callbacks cb 0x55b4f2896c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:58:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5909000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29a34e0 0x55b4f29a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5903400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 0x55b4f29a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5909000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29a34e0 0x55b4f29a6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:38060/0 (socket says 172.21.15.112:38060) 2024-05-08T05:58:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5909000700 1 -- 172.21.15.112:0/3720306106 learned_addr learned my addr 172.21.15.112:0/3720306106 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:48.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5903e00700 1 --2- 172.21.15.112:0/3720306106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 0x55b4f29a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5909000700 1 -- 172.21.15.112:0/3720306106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 msgr2=0x55b4f29a91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5909000700 1 --2- 172.21.15.112:0/3720306106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 0x55b4f29a91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5909000700 1 -- 172.21.15.112:0/3720306106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 msgr2=0x55b4f29a6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5909000700 1 --2- 172.21.15.112:0/3720306106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 0x55b4f29a6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5909000700 1 -- 172.21.15.112:0/3720306106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4f28b5420 con 0x55b4f29a34e0 2024-05-08T05:58:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.167+0000 7f5903e00700 1 --2- 172.21.15.112:0/3720306106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 0x55b4f29a91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:58:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.168+0000 7f5909000700 1 --2- 172.21.15.112:0/3720306106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29a34e0 0x55b4f29a6ab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0df6a0 tx=0x7f58fc0a9930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7c2621f5669e969 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.168+0000 7f58ee000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:58:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.168+0000 7f58ee000700 1 -- 172.21.15.112:0/3720306106 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc0b2020 con 0x55b4f29a34e0 2024-05-08T05:58:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.168+0000 7f58ee000700 1 -- 172.21.15.112:0/3720306106 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f58fc005310 con 0x55b4f29a34e0 2024-05-08T05:58:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.168+0000 7f5915922b80 1 -- 172.21.15.112:0/3720306106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58f4011440 con 0x55b4f29a34e0 2024-05-08T05:58:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.168+0000 7f58ee000700 1 -- 172.21.15.112:0/3720306106 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f58fc0ad160 con 0x55b4f29a34e0 2024-05-08T05:58:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.168+0000 7f5915922b80 10 client.25505 fetch_fsmap learned FSMap version 29 2024-05-08T05:58:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.168+0000 7f5915922b80 1 -- 172.21.15.112:0/3720306106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4f29b4790 con 0x55b4f29a34e0 2024-05-08T05:58:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.169+0000 7f58ee000700 1 -- 172.21.15.112:0/3720306106 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58fc0c1070 con 0x55b4f29a34e0 2024-05-08T05:58:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.169+0000 7f58ee000700 1 -- 172.21.15.112:0/3720306106 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f58fc0b1b90 con 0x55b4f29a34e0 2024-05-08T05:58:48.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.169+0000 7f5915922b80 10 client.25505 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:58:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.169+0000 7f5915922b80 10 client.25505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:58:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.169+0000 7f5915922b80 20 client.25505 populate_metadata read hostname 'smithi112' 2024-05-08T05:58:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.169+0000 7f5915922b80 1 --2- 172.21.15.112:0/3720306106 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55b4f29b6d20 0x55b4f29b9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.169+0000 7f5915922b80 4 client.25505 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:58:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.169+0000 7f5915922b80 1 -- 172.21.15.112:0/3720306106 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4f296a140 con 0x55b4f29b6d20 2024-05-08T05:58:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.170+0000 7f5903e00700 1 --2- 172.21.15.112:0/3720306106 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55b4f29b6d20 0x55b4f29b9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.170+0000 7f5903e00700 1 --2- 172.21.15.112:0/3720306106 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55b4f29b6d20 0x55b4f29b9180 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=910f9c27eb6dfbe7 server_cookie=7d9b2774b40c0fe8 in_seq=0 out_seq=0 2024-05-08T05:58:48.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.170+0000 7f58ee000700 10 client.25505 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:48.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.171+0000 7f58ee000700 1 -- 172.21.15.112:0/3720306106 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b4f296a140 con 0x55b4f29b6d20 2024-05-08T05:58:48.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.171+0000 7f58ee000700 10 client.25505 handle_command_reply: tid=0 2024-05-08T05:58:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.171+0000 7f5915922b80 1 client.25505 shutdown 2024-05-08T05:58:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.171+0000 7f5915922b80 1 -- 172.21.15.112:0/3720306106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29a34e0 msgr2=0x55b4f29a6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.171+0000 7f5915922b80 1 --2- 172.21.15.112:0/3720306106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29a34e0 0x55b4f29a6ab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0df6a0 tx=0x7f58fc0a9930 comp rx=0 tx=0).stop 2024-05-08T05:58:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.171+0000 7f5915922b80 1 -- 172.21.15.112:0/3720306106 shutdown_connections 2024-05-08T05:58:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.171+0000 7f5915922b80 1 --2- 172.21.15.112:0/3720306106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29a34e0 0x55b4f29a6ab0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.171+0000 7f5915922b80 1 --2- 172.21.15.112:0/3720306106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 0x55b4f29a91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.171+0000 7f5915922b80 1 --2- 172.21.15.112:0/3720306106 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55b4f29b6d20 0x55b4f29b9180 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.171+0000 7f5915922b80 1 --2- 172.21.15.112:0/3720306106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 0x55b4f29a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.172+0000 7f5915922b80 1 -- 172.21.15.112:0/3720306106 >> 172.21.15.112:0/3720306106 conn(0x55b4f29156d0 msgr2=0x55b4f2997190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.172+0000 7f58ee000700 0 client.25505 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.172+0000 7f5915922b80 1 -- 172.21.15.112:0/3720306106 shutdown_connections 2024-05-08T05:58:48.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.172+0000 7f5915922b80 1 -- 172.21.15.112:0/3720306106 wait complete. 2024-05-08T05:58:48.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.172+0000 7f5915922b80 20 client.25505 trim_cache size 0 max 16384 2024-05-08T05:58:48.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.198+0000 7f5915922b80 1 Processor -- start 2024-05-08T05:58:48.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.198+0000 7f5915922b80 1 -- start start 2024-05-08T05:58:48.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.199+0000 7f5915922b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29156d0 0x55b4f29b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.199+0000 7f5915922b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 0x55b4f29a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.199+0000 7f5915922b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 0x55b4f2997910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.199+0000 7f5915922b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59041b2780 con 0x55b4f29a2bc0 2024-05-08T05:58:48.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.199+0000 7f5915922b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59041aff80 con 0x55b4f29156d0 2024-05-08T05:58:48.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.199+0000 7f5915922b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f59041addd0 con 0x55b4f299d8f0 2024-05-08T05:58:48.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.199+0000 7f5903e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29156d0 0x55b4f29b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.199+0000 7f5903400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 0x55b4f29a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.199+0000 7f5903e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29156d0 0x55b4f29b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:38066/0 (socket says 172.21.15.112:38066) 2024-05-08T05:58:48.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.199+0000 7f5903e00700 1 -- 172.21.15.112:0/3337618242 learned_addr learned my addr 172.21.15.112:0/3337618242 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:48.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.200+0000 7f5903400700 1 -- 172.21.15.112:0/3337618242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29156d0 msgr2=0x55b4f29b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:48.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.200+0000 7f5903400700 1 --2- 172.21.15.112:0/3337618242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29156d0 0x55b4f29b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.200+0000 7f5903400700 1 -- 172.21.15.112:0/3337618242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 msgr2=0x55b4f2997910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:48.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.200+0000 7f5903400700 1 --2- 172.21.15.112:0/3337618242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 0x55b4f2997910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.200+0000 7f5903400700 1 -- 172.21.15.112:0/3337618242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4f29a10c0 con 0x55b4f299d8f0 2024-05-08T05:58:48.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.200+0000 7f5903400700 1 --2- 172.21.15.112:0/3337618242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 0x55b4f29a0a80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f58f00098e0 tx=0x7f58f000b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16c5a1d736f267ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:48.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.200+0000 7f58f8c00700 1 -- 172.21.15.112:0/3337618242 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0029070 con 0x55b4f299d8f0 2024-05-08T05:58:48.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.200+0000 7f58f8c00700 1 -- 172.21.15.112:0/3337618242 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f00050c0 con 0x55b4f299d8f0 2024-05-08T05:58:48.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.200+0000 7f58f8c00700 1 -- 172.21.15.112:0/3337618242 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f000ba30 con 0x55b4f299d8f0 2024-05-08T05:58:48.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.201+0000 7f5915922b80 1 -- 172.21.15.112:0/3337618242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 msgr2=0x55b4f29a0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:48.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.201+0000 7f5915922b80 1 --2- 172.21.15.112:0/3337618242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 0x55b4f29a0a80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f58f00098e0 tx=0x7f58f000b540 comp rx=0 tx=0).stop 2024-05-08T05:58:48.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.201+0000 7f5915922b80 1 -- 172.21.15.112:0/3337618242 shutdown_connections 2024-05-08T05:58:48.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.201+0000 7f5915922b80 1 --2- 172.21.15.112:0/3337618242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29a2bc0 0x55b4f2997910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.201+0000 7f5915922b80 1 --2- 172.21.15.112:0/3337618242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29156d0 0x55b4f29b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.201+0000 7f5915922b80 1 --2- 172.21.15.112:0/3337618242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 0x55b4f29a0a80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.201+0000 7f5915922b80 1 -- 172.21.15.112:0/3337618242 >> 172.21.15.112:0/3337618242 conn(0x55b4f2994750 msgr2=0x55b4f29a1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.202+0000 7f5915922b80 1 -- 172.21.15.112:0/3337618242 shutdown_connections 2024-05-08T05:58:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.202+0000 7f5915922b80 1 -- 172.21.15.112:0/3337618242 wait complete. 2024-05-08T05:58:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.202+0000 7f5915922b80 1 -- start start 2024-05-08T05:58:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.203+0000 7f5915922b80 1 Processor -- start 2024-05-08T05:58:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.203+0000 7f5915922b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29156d0 0x55b4f29af330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.203+0000 7f5915922b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 0x55b4f29af870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.203+0000 7f5915922b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29a2bc0 0x55b4f29ac300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.203+0000 7f5915922b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59041ae440 con 0x55b4f29156d0 2024-05-08T05:58:48.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.203+0000 7f5915922b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59041af8b0 con 0x55b4f29a2bc0 2024-05-08T05:58:48.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.203+0000 7f5915922b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f59041aeb10 con 0x55b4f299d8f0 2024-05-08T05:58:48.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.203+0000 7f5915922b80 10 client.0 _ll_register_callbacks cb 0x55b4f2896c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:58:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.204+0000 7f5909000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29a2bc0 0x55b4f29ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.204+0000 7f5909000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29a2bc0 0x55b4f29ac300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:38078/0 (socket says 172.21.15.112:38078) 2024-05-08T05:58:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.204+0000 7f5909000700 1 -- 172.21.15.112:0/3386148889 learned_addr learned my addr 172.21.15.112:0/3386148889 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.204+0000 7f5903e00700 1 --2- 172.21.15.112:0/3386148889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29156d0 0x55b4f29af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.204+0000 7f5903400700 1 --2- 172.21.15.112:0/3386148889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 0x55b4f29af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.204+0000 7f5909000700 1 -- 172.21.15.112:0/3386148889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 msgr2=0x55b4f29af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.204+0000 7f5909000700 1 --2- 172.21.15.112:0/3386148889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 0x55b4f29af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.204+0000 7f5909000700 1 -- 172.21.15.112:0/3386148889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29156d0 msgr2=0x55b4f29af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.204+0000 7f5909000700 1 --2- 172.21.15.112:0/3386148889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29156d0 0x55b4f29af330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.204+0000 7f5909000700 1 -- 172.21.15.112:0/3386148889 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4f29ac840 con 0x55b4f29a2bc0 2024-05-08T05:58:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.204+0000 7f5909000700 1 --2- 172.21.15.112:0/3386148889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29a2bc0 0x55b4f29ac300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0be520 tx=0x7f58fc0ad630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d188d4da967cbe14 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.204+0000 7f58ee000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:58:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.205+0000 7f58ee000700 1 -- 172.21.15.112:0/3386148889 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc0a98c0 con 0x55b4f29a2bc0 2024-05-08T05:58:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.205+0000 7f58ee000700 1 -- 172.21.15.112:0/3386148889 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f58fc005c00 con 0x55b4f29a2bc0 2024-05-08T05:58:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.205+0000 7f5915922b80 1 -- 172.21.15.112:0/3386148889 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58f00050c0 con 0x55b4f29a2bc0 2024-05-08T05:58:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.205+0000 7f58ee000700 1 -- 172.21.15.112:0/3386148889 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f58fc005e00 con 0x55b4f29a2bc0 2024-05-08T05:58:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.205+0000 7f5915922b80 10 client.25511 fetch_fsmap learned FSMap version 29 2024-05-08T05:58:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.205+0000 7f5915922b80 1 -- 172.21.15.112:0/3386148889 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4f2a2a5b0 con 0x55b4f29a2bc0 2024-05-08T05:58:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.206+0000 7f58ee000700 1 -- 172.21.15.112:0/3386148889 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f00050c0 con 0x55b4f29a2bc0 2024-05-08T05:58:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.206+0000 7f58ee000700 1 -- 172.21.15.112:0/3386148889 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f58fc0a4bc0 con 0x55b4f29a2bc0 2024-05-08T05:58:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.206+0000 7f5915922b80 10 client.25511 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:58:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.206+0000 7f5915922b80 10 client.25511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:58:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.206+0000 7f5915922b80 20 client.25511 populate_metadata read hostname 'smithi112' 2024-05-08T05:58:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.206+0000 7f5915922b80 1 --2- 172.21.15.112:0/3386148889 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55b4f29a34e0 0x55b4f2a2cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.206+0000 7f5915922b80 4 client.25511 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:58:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.206+0000 7f5915922b80 1 -- 172.21.15.112:0/3386148889 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b4f28fd400 con 0x55b4f29a34e0 2024-05-08T05:58:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.206+0000 7f5903e00700 1 --2- 172.21.15.112:0/3386148889 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55b4f29a34e0 0x55b4f2a2cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.207+0000 7f5903e00700 1 --2- 172.21.15.112:0/3386148889 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55b4f29a34e0 0x55b4f2a2cc50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d19671fc11d44e51 server_cookie=dc5dd94eb957b05e in_seq=0 out_seq=0 2024-05-08T05:58:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.207+0000 7f58ee000700 10 client.25511 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.207+0000 7f58ee000700 1 -- 172.21.15.112:0/3386148889 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b4f28fd400 con 0x55b4f29a34e0 2024-05-08T05:58:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.207+0000 7f58ee000700 10 client.25511 handle_command_reply: tid=0 2024-05-08T05:58:48.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.207+0000 7f5915922b80 1 client.25511 shutdown 2024-05-08T05:58:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.209+0000 7f5915922b80 1 -- 172.21.15.112:0/3386148889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29a2bc0 msgr2=0x55b4f29ac300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.209+0000 7f5915922b80 1 --2- 172.21.15.112:0/3386148889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29a2bc0 0x55b4f29ac300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0be520 tx=0x7f58fc0ad630 comp rx=0 tx=0).stop 2024-05-08T05:58:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.210+0000 7f5915922b80 1 -- 172.21.15.112:0/3386148889 shutdown_connections 2024-05-08T05:58:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.210+0000 7f5915922b80 1 --2- 172.21.15.112:0/3386148889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b4f29a2bc0 0x55b4f29ac300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.210+0000 7f5915922b80 1 --2- 172.21.15.112:0/3386148889 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55b4f29a34e0 0x55b4f2a2cc50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.210+0000 7f5915922b80 1 --2- 172.21.15.112:0/3386148889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4f29156d0 0x55b4f29af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.210+0000 7f5915922b80 1 --2- 172.21.15.112:0/3386148889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b4f299d8f0 0x55b4f29af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.210+0000 7f58ee000700 0 client.25511 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:58:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.210+0000 7f5915922b80 1 -- 172.21.15.112:0/3386148889 >> 172.21.15.112:0/3386148889 conn(0x55b4f2994750 msgr2=0x55b4f29a4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.210+0000 7f5915922b80 1 -- 172.21.15.112:0/3386148889 shutdown_connections 2024-05-08T05:58:48.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.210+0000 7f5915922b80 1 -- 172.21.15.112:0/3386148889 wait complete. 2024-05-08T05:58:48.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.210+0000 7f5915922b80 20 client.25511 trim_cache size 0 max 16384 2024-05-08T05:58:48.221 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:58:48.221 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-08T05:58:48.221 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-08T05:58:48.221 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:58:48.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.211+0000 7f590b000700 1 -- 172.21.15.112:0/31510201 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f58e80b2900 msgr2=0x7f58e80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:48.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.211+0000 7f590b000700 1 --2- 172.21.15.112:0/31510201 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f58e80b2900 0x7f58e80b4dc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f58f4007c60 tx=0x7f58f40106b0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.211+0000 7f590b000700 1 -- 172.21.15.112:0/31510201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f590410b6b0 msgr2=0x7f59041a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.211+0000 7f590b000700 1 --2- 172.21.15.112:0/31510201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f590410b6b0 0x7f59041a0660 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f58fc007f10 tx=0x7f58fc00d3b0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.211+0000 7f590b000700 1 -- 172.21.15.112:0/31510201 shutdown_connections 2024-05-08T05:58:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.211+0000 7f590b000700 1 --2- 172.21.15.112:0/31510201 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f58e80b2900 0x7f58e80b4dc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.211+0000 7f590b000700 1 --2- 172.21.15.112:0/31510201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f590410b6b0 0x7f59041a0660 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.211+0000 7f590b000700 1 --2- 172.21.15.112:0/31510201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59041056b0 0x7f590419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.211+0000 7f590b000700 1 --2- 172.21.15.112:0/31510201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5904106070 0x7f590419c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.211+0000 7f590b000700 1 -- 172.21.15.112:0/31510201 >> 172.21.15.112:0/31510201 conn(0x7f59041011a0 msgr2=0x7f5904101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:48.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.213+0000 7f590b000700 1 -- 172.21.15.112:0/31510201 shutdown_connections 2024-05-08T05:58:48.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:48.213+0000 7f590b000700 1 -- 172.21.15.112:0/31510201 wait complete. 2024-05-08T05:58:49.190 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:58:49.190 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:58:49.682 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:58:49.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:49 smithi150 ceph-mon[34315]: pgmap v546: 97 pgs: 97 active+clean; 72 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 266 op/s 2024-05-08T05:58:49.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:49 smithi112 ceph-mon[30003]: pgmap v546: 97 pgs: 97 active+clean; 72 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 266 op/s 2024-05-08T05:58:49.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:49 smithi162 ceph-mon[34800]: pgmap v546: 97 pgs: 97 active+clean; 72 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 266 op/s 2024-05-08T05:58:51.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:51 smithi150 ceph-mon[34315]: pgmap v547: 97 pgs: 97 active+clean; 72 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 303 op/s 2024-05-08T05:58:51.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:51 smithi112 ceph-mon[30003]: pgmap v547: 97 pgs: 97 active+clean; 72 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 303 op/s 2024-05-08T05:58:51.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:51 smithi162 ceph-mon[34800]: pgmap v547: 97 pgs: 97 active+clean; 72 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 303 op/s 2024-05-08T05:58:53.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.056+0000 7f4708a00700 1 -- 172.21.15.112:0/3336837237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470406ba40 msgr2=0x7f470406be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:53.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.056+0000 7f4708a00700 1 --2- 172.21.15.112:0/3336837237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470406ba40 0x7f470406be20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009990 tx=0x7f46f4009ca0 comp rx=0 tx=0).stop 2024-05-08T05:58:53.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.056+0000 7f4708a00700 1 -- 172.21.15.112:0/3336837237 shutdown_connections 2024-05-08T05:58:53.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.056+0000 7f4708a00700 1 --2- 172.21.15.112:0/3336837237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f470410ff40 0x7f4704112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:53.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.056+0000 7f4708a00700 1 --2- 172.21.15.112:0/3336837237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470406ba40 0x7f470406be20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:53.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.056+0000 7f4708a00700 1 --2- 172.21.15.112:0/3336837237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f470406c360 0x7f470410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:53.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.056+0000 7f4708a00700 1 -- 172.21.15.112:0/3336837237 >> 172.21.15.112:0/3336837237 conn(0x7f4704079310 msgr2=0x7f4704079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:53.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.056+0000 7f4708a00700 1 -- 172.21.15.112:0/3336837237 shutdown_connections 2024-05-08T05:58:53.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.056+0000 7f4708a00700 1 -- 172.21.15.112:0/3336837237 wait complete. 2024-05-08T05:58:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.057+0000 7f4708a00700 1 Processor -- start 2024-05-08T05:58:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.058+0000 7f4708a00700 1 -- start start 2024-05-08T05:58:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.058+0000 7f4708a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f470406ba40 0x7f470419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.058+0000 7f4708a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470406c360 0x7f470419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.058+0000 7f4708a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f470410ff40 0x7f47041a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.058+0000 7f4708a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47041158a0 con 0x7f470406ba40 2024-05-08T05:58:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.058+0000 7f4708a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4704115720 con 0x7f470410ff40 2024-05-08T05:58:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.058+0000 7f4708a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4704115a20 con 0x7f470406c360 2024-05-08T05:58:53.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.058+0000 7f4701e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f470406ba40 0x7f470419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:53.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.059+0000 7f4701e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f470406ba40 0x7f470419bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43044/0 (socket says 172.21.15.112:43044) 2024-05-08T05:58:53.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.059+0000 7f4701e00700 1 -- 172.21.15.112:0/2666508148 learned_addr learned my addr 172.21.15.112:0/2666508148 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:53.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.059+0000 7f4702800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f470410ff40 0x7f47041a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:53.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.059+0000 7f4702800700 1 -- 172.21.15.112:0/2666508148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470406c360 msgr2=0x7f470419c2f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T05:58:53.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.059+0000 7f4702800700 1 --2- 172.21.15.112:0/2666508148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470406c360 0x7f470419c2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:53.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.059+0000 7f4702800700 1 -- 172.21.15.112:0/2666508148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f470406ba40 msgr2=0x7f470419bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:53.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.059+0000 7f4702800700 1 --2- 172.21.15.112:0/2666508148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f470406ba40 0x7f470419bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:53.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.059+0000 7f4702800700 1 -- 172.21.15.112:0/2666508148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47041a0dc0 con 0x7f470410ff40 2024-05-08T05:58:53.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.059+0000 7f4702800700 1 --2- 172.21.15.112:0/2666508148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f470410ff40 0x7f47041a0680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f470406af40 tx=0x7f46fc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:53.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.059+0000 7f46faa00700 1 -- 172.21.15.112:0/2666508148 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc00ca50 con 0x7f470410ff40 2024-05-08T05:58:53.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.060+0000 7f4708a00700 1 -- 172.21.15.112:0/2666508148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47041a10b0 con 0x7f470410ff40 2024-05-08T05:58:53.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.060+0000 7f46faa00700 1 -- 172.21.15.112:0/2666508148 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46fc00cbf0 con 0x7f470410ff40 2024-05-08T05:58:53.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.060+0000 7f46faa00700 1 -- 172.21.15.112:0/2666508148 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc004810 con 0x7f470410ff40 2024-05-08T05:58:53.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.060+0000 7f4708a00700 1 -- 172.21.15.112:0/2666508148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47041a8930 con 0x7f470410ff40 2024-05-08T05:58:53.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.061+0000 7f4708a00700 1 -- 172.21.15.112:0/2666508148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f470406a190 con 0x7f470410ff40 2024-05-08T05:58:53.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.063+0000 7f46faa00700 1 -- 172.21.15.112:0/2666508148 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f46fc012050 con 0x7f470410ff40 2024-05-08T05:58:53.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.065+0000 7f46faa00700 1 --2- 172.21.15.112:0/2666508148 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f46e80b2880 0x7f46e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:53.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.065+0000 7f46faa00700 1 -- 172.21.15.112:0/2666508148 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f46fc0e1be0 con 0x7f470410ff40 2024-05-08T05:58:53.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.065+0000 7f4701e00700 1 --2- 172.21.15.112:0/2666508148 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f46e80b2880 0x7f46e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:53.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.066+0000 7f4701e00700 1 --2- 172.21.15.112:0/2666508148 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f46e80b2880 0x7f46e80b4d40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f46f4006010 tx=0x7f46f4005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:53.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.069+0000 7f46faa00700 1 -- 172.21.15.112:0/2666508148 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f46fc022060 con 0x7f470410ff40 2024-05-08T05:58:53.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.425+0000 7f4708a00700 1 -- 172.21.15.112:0/2666508148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f470402e510 con 0x7f470410ff40 2024-05-08T05:58:53.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.426+0000 7f46faa00700 1 -- 172.21.15.112:0/2666508148 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11898 (secure 0 0 0) 0x7f46fc010040 con 0x7f470410ff40 2024-05-08T05:58:53.426 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:58:53.426 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T05:58:06.021913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24518,24511]},"id":1}]} 2024-05-08T05:58:53.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.429+0000 7f4708a00700 1 -- 172.21.15.112:0/2666508148 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f46e80b2880 msgr2=0x7f46e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:53.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.429+0000 7f4708a00700 1 --2- 172.21.15.112:0/2666508148 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f46e80b2880 0x7f46e80b4d40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f46f4006010 tx=0x7f46f4005ab0 comp rx=0 tx=0).stop 2024-05-08T05:58:53.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.429+0000 7f4708a00700 1 -- 172.21.15.112:0/2666508148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f470410ff40 msgr2=0x7f47041a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:53.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.429+0000 7f4708a00700 1 --2- 172.21.15.112:0/2666508148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f470410ff40 0x7f47041a0680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f470406af40 tx=0x7f46fc00c3a0 comp rx=0 tx=0).stop 2024-05-08T05:58:53.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.429+0000 7f4708a00700 1 -- 172.21.15.112:0/2666508148 shutdown_connections 2024-05-08T05:58:53.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.429+0000 7f4708a00700 1 --2- 172.21.15.112:0/2666508148 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f46e80b2880 0x7f46e80b4d40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:53.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.430+0000 7f4708a00700 1 --2- 172.21.15.112:0/2666508148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f470410ff40 0x7f47041a0680 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:53.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.430+0000 7f4708a00700 1 --2- 172.21.15.112:0/2666508148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f470406ba40 0x7f470419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:53.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.430+0000 7f4708a00700 1 --2- 172.21.15.112:0/2666508148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470406c360 0x7f470419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:53.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.430+0000 7f4708a00700 1 -- 172.21.15.112:0/2666508148 >> 172.21.15.112:0/2666508148 conn(0x7f4704079310 msgr2=0x7f47041113f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:53.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.430+0000 7f4708a00700 1 -- 172.21.15.112:0/2666508148 shutdown_connections 2024-05-08T05:58:53.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:53.430+0000 7f4708a00700 1 -- 172.21.15.112:0/2666508148 wait complete. 2024-05-08T05:58:53.431 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 29 2024-05-08T05:58:53.618 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:53 smithi162 ceph-mon[34800]: pgmap v548: 97 pgs: 97 active+clean; 73 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 278 op/s 2024-05-08T05:58:53.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:53 smithi150 ceph-mon[34315]: pgmap v548: 97 pgs: 97 active+clean; 73 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 278 op/s 2024-05-08T05:58:53.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:53 smithi112 ceph-mon[30003]: pgmap v548: 97 pgs: 97 active+clean; 73 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 278 op/s 2024-05-08T05:58:54.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-08T05:58:54.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:54 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/2666508148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:58:54.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:54 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2666508148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:58:54.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:54 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/2666508148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:58:54.925 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:58:55.618 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:55 smithi112 ceph-mon[30003]: pgmap v549: 97 pgs: 97 active+clean; 73 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 322 op/s 2024-05-08T05:58:55.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:55 smithi150 ceph-mon[34315]: pgmap v549: 97 pgs: 97 active+clean; 73 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 322 op/s 2024-05-08T05:58:55.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:55 smithi162 ceph-mon[34800]: pgmap v549: 97 pgs: 97 active+clean; 73 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 322 op/s 2024-05-08T05:58:57.178 INFO:tasks.workunit.client.0.smithi112.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-05-08T05:58:57.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:57 smithi112 ceph-mon[30003]: pgmap v550: 97 pgs: 97 active+clean; 73 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 256 op/s 2024-05-08T05:58:57.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:57 smithi150 ceph-mon[34315]: pgmap v550: 97 pgs: 97 active+clean; 73 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 256 op/s 2024-05-08T05:58:57.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:57 smithi162 ceph-mon[34800]: pgmap v550: 97 pgs: 97 active+clean; 73 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 256 op/s 2024-05-08T05:58:58.031 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xffcdd 2024-05-08T05:58:58.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.173+0000 7f0634a00700 1 -- 172.21.15.112:0/332886030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0630103d90 msgr2=0x7f06301102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:58.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.173+0000 7f0634a00700 1 --2- 172.21.15.112:0/332886030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0630103d90 0x7f06301102a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f062800b0a0 tx=0x7f062800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.173+0000 7f0634a00700 1 -- 172.21.15.112:0/332886030 shutdown_connections 2024-05-08T05:58:58.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.173+0000 7f0634a00700 1 --2- 172.21.15.112:0/332886030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0630103d90 0x7f06301102a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.173+0000 7f0634a00700 1 --2- 172.21.15.112:0/332886030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0630109460 0x7f0630109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.173+0000 7f0634a00700 1 --2- 172.21.15.112:0/332886030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06301033d0 0x7f0630103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.173+0000 7f0634a00700 1 -- 172.21.15.112:0/332886030 >> 172.21.15.112:0/332886030 conn(0x7f06300fef20 msgr2=0x7f0630101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:58.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.173+0000 7f0634a00700 1 -- 172.21.15.112:0/332886030 shutdown_connections 2024-05-08T05:58:58.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.173+0000 7f0634a00700 1 -- 172.21.15.112:0/332886030 wait complete. 2024-05-08T05:58:58.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.175+0000 7f0634a00700 1 Processor -- start 2024-05-08T05:58:58.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.175+0000 7f0634a00700 1 -- start start 2024-05-08T05:58:58.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.176+0000 7f0634a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06301033d0 0x7f06300757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.176+0000 7f0634a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0630103d90 0x7f0630075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.176+0000 7f0634a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0630109460 0x7f0630076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.176+0000 7f0634a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06301135c0 con 0x7f06301033d0 2024-05-08T05:58:58.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.176+0000 7f0634a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0630113440 con 0x7f0630109460 2024-05-08T05:58:58.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.176+0000 7f0634a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0630113740 con 0x7f0630103d90 2024-05-08T05:58:58.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.176+0000 7f062ea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0630103d90 0x7f0630075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.176+0000 7f062ea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0630103d90 0x7f0630075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:56324/0 (socket says 172.21.15.112:56324) 2024-05-08T05:58:58.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.176+0000 7f062ea00700 1 -- 172.21.15.112:0/2950429908 learned_addr learned my addr 172.21.15.112:0/2950429908 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:58.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.176+0000 7f062f400700 1 --2- 172.21.15.112:0/2950429908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06301033d0 0x7f06300757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.176+0000 7f062fe00700 1 --2- 172.21.15.112:0/2950429908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0630109460 0x7f0630076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.177+0000 7f062ea00700 1 -- 172.21.15.112:0/2950429908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0630109460 msgr2=0x7f0630076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:58.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.177+0000 7f062ea00700 1 --2- 172.21.15.112:0/2950429908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0630109460 0x7f0630076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.177+0000 7f062ea00700 1 -- 172.21.15.112:0/2950429908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06301033d0 msgr2=0x7f06300757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:58.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.177+0000 7f062ea00700 1 --2- 172.21.15.112:0/2950429908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06301033d0 0x7f06300757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.177+0000 7f062ea00700 1 -- 172.21.15.112:0/2950429908 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06301a7fa0 con 0x7f0630103d90 2024-05-08T05:58:58.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.177+0000 7f062ea00700 1 --2- 172.21.15.112:0/2950429908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0630103d90 0x7f0630075ce0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f061c00e9a0 tx=0x7f061c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:58.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.177+0000 7f0627e00700 1 -- 172.21.15.112:0/2950429908 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c00ca10 con 0x7f0630103d90 2024-05-08T05:58:58.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.177+0000 7f0627e00700 1 -- 172.21.15.112:0/2950429908 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061c00cbb0 con 0x7f0630103d90 2024-05-08T05:58:58.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.178+0000 7f0627e00700 1 -- 172.21.15.112:0/2950429908 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c018680 con 0x7f0630103d90 2024-05-08T05:58:58.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.178+0000 7f0634a00700 1 -- 172.21.15.112:0/2950429908 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0630199810 con 0x7f0630103d90 2024-05-08T05:58:58.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.178+0000 7f0634a00700 1 -- 172.21.15.112:0/2950429908 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0630199cd0 con 0x7f0630103d90 2024-05-08T05:58:58.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.184+0000 7f06408e3b80 1 Processor -- start 2024-05-08T05:58:58.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.184+0000 7f06408e3b80 1 -- start start 2024-05-08T05:58:58.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.184+0000 7f0627e00700 1 -- 172.21.15.112:0/2950429908 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f061c010430 con 0x7f0630103d90 2024-05-08T05:58:58.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.184+0000 7f06408e3b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb2989e0 0x562eeb298dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.184+0000 7f06408e3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb299300 0x562eeb292fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.184+0000 7f06408e3b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb293600 0x562eeb293a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.184+0000 7f06408e3b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562eeb1e5a20 con 0x562eeb2989e0 2024-05-08T05:58:58.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.185+0000 7f06408e3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562eeb183760 con 0x562eeb299300 2024-05-08T05:58:58.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.185+0000 7f06408e3b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562eeb1e19e0 con 0x562eeb293600 2024-05-08T05:58:58.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.185+0000 7f062f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb299300 0x562eeb292fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.185+0000 7f062f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb299300 0x562eeb292fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:47548/0 (socket says 172.21.15.112:47548) 2024-05-08T05:58:58.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.185+0000 7f062f400700 1 -- 172.21.15.112:0/3837468643 learned_addr learned my addr 172.21.15.112:0/3837468643 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:58.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.185+0000 7f062ea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb293600 0x562eeb293a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.185+0000 7f062f400700 1 -- 172.21.15.112:0/3837468643 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb293600 msgr2=0x562eeb293a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:58.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.185+0000 7f062f400700 1 --2- 172.21.15.112:0/3837468643 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb293600 0x562eeb293a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.185+0000 7f062f400700 1 -- 172.21.15.112:0/3837468643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb2989e0 msgr2=0x562eeb298dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:58:58.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.186+0000 7f062fe00700 1 --2- 172.21.15.112:0/3837468643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb2989e0 0x562eeb298dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.186+0000 7f062f400700 1 --2- 172.21.15.112:0/3837468643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb2989e0 0x562eeb298dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.186+0000 7f062f400700 1 -- 172.21.15.112:0/3837468643 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eeb1328d0 con 0x562eeb299300 2024-05-08T05:58:58.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.186+0000 7f062fe00700 1 --2- 172.21.15.112:0/3837468643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb2989e0 0x562eeb298dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:58:58.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.186+0000 7f062f400700 1 --2- 172.21.15.112:0/3837468643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb299300 0x562eeb292fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0620009d00 tx=0x7f0620011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25cb878e1333831d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:58.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.186+0000 7f0627e00700 1 --2- 172.21.15.112:0/2950429908 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f06140b2900 0x7f06140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.186+0000 7f0624c00700 1 -- 172.21.15.112:0/3837468643 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620004230 con 0x562eeb299300 2024-05-08T05:58:58.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.186+0000 7f0624c00700 1 -- 172.21.15.112:0/3837468643 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06200043d0 con 0x562eeb299300 2024-05-08T05:58:58.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.186+0000 7f0624c00700 1 -- 172.21.15.112:0/3837468643 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620019920 con 0x562eeb299300 2024-05-08T05:58:58.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.186+0000 7f0627e00700 1 -- 172.21.15.112:0/2950429908 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f061c0e2570 con 0x7f0630103d90 2024-05-08T05:58:58.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.187+0000 7f06408e3b80 1 -- 172.21.15.112:0/3837468643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb299300 msgr2=0x562eeb292fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:58.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.187+0000 7f06408e3b80 1 --2- 172.21.15.112:0/3837468643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb299300 0x562eeb292fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0620009d00 tx=0x7f0620011720 comp rx=0 tx=0).stop 2024-05-08T05:58:58.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.187+0000 7f06408e3b80 1 -- 172.21.15.112:0/3837468643 shutdown_connections 2024-05-08T05:58:58.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.187+0000 7f06408e3b80 1 --2- 172.21.15.112:0/3837468643 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb293600 0x562eeb293a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.187+0000 7f06408e3b80 1 --2- 172.21.15.112:0/3837468643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb2989e0 0x562eeb298dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.188+0000 7f06408e3b80 1 --2- 172.21.15.112:0/3837468643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb299300 0x562eeb292fe0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.188+0000 7f06408e3b80 1 -- 172.21.15.112:0/3837468643 >> 172.21.15.112:0/3837468643 conn(0x562eeb1c1b60 msgr2=0x562eeb28d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:58.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.188+0000 7f06408e3b80 1 -- 172.21.15.112:0/3837468643 shutdown_connections 2024-05-08T05:58:58.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.188+0000 7f06408e3b80 1 -- 172.21.15.112:0/3837468643 wait complete. 2024-05-08T05:58:58.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.188+0000 7f06408e3b80 1 -- start start 2024-05-08T05:58:58.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.188+0000 7f06408e3b80 1 Processor -- start 2024-05-08T05:58:58.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.189+0000 7f06408e3b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb293600 0x562eeb29eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.189+0000 7f06408e3b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb2989e0 0x562eeb29c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.189+0000 7f06408e3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb299300 0x562eeb29c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.189+0000 7f06408e3b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562eeb29fdb0 con 0x562eeb293600 2024-05-08T05:58:58.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.189+0000 7f06408e3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562eeb14bb20 con 0x562eeb299300 2024-05-08T05:58:58.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.189+0000 7f06408e3b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562eeb29ff30 con 0x562eeb2989e0 2024-05-08T05:58:58.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.189+0000 7f06408e3b80 10 client.0 _ll_register_callbacks cb 0x562eeb18cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:58:58.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.190+0000 7f062fe00700 1 --2- 172.21.15.112:0/2950429908 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f06140b2900 0x7f06140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.190+0000 7f062ea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb2989e0 0x562eeb29c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.190+0000 7f062ea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb2989e0 0x562eeb29c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:56338/0 (socket says 172.21.15.112:56338) 2024-05-08T05:58:58.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.190+0000 7f062ea00700 1 -- 172.21.15.112:0/253649362 learned_addr learned my addr 172.21.15.112:0/253649362 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:58.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.190+0000 7f062f400700 1 --2- 172.21.15.112:0/253649362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb293600 0x562eeb29eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.190+0000 7f062ea00700 1 -- 172.21.15.112:0/253649362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb299300 msgr2=0x562eeb29c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:58.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.190+0000 7f062ea00700 1 --2- 172.21.15.112:0/253649362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb299300 0x562eeb29c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.190+0000 7f062ea00700 1 -- 172.21.15.112:0/253649362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb293600 msgr2=0x562eeb29eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:58.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.190+0000 7f062ea00700 1 --2- 172.21.15.112:0/253649362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb293600 0x562eeb29eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.190+0000 7f062ea00700 1 -- 172.21.15.112:0/253649362 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eeb1ab420 con 0x562eeb2989e0 2024-05-08T05:58:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.190+0000 7f062f400700 1 --2- 172.21.15.112:0/253649362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb293600 0x562eeb29eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:58:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.190+0000 7f062ea00700 1 --2- 172.21.15.112:0/253649362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb2989e0 0x562eeb29c2d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f061c0a9860 tx=0x7f061c0a5b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5bc0f580439e32a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.190+0000 7f0619600700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:58:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.191+0000 7f062fe00700 1 --2- 172.21.15.112:0/2950429908 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f06140b2900 0x7f06140b4dc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f062800b780 tx=0x7f0628009ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.191+0000 7f0619600700 1 -- 172.21.15.112:0/253649362 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c0a52f0 con 0x562eeb2989e0 2024-05-08T05:58:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.191+0000 7f06408e3b80 1 -- 172.21.15.112:0/253649362 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06200043d0 con 0x562eeb2989e0 2024-05-08T05:58:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.191+0000 7f0619600700 1 -- 172.21.15.112:0/253649362 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f061c0b2530 con 0x562eeb2989e0 2024-05-08T05:58:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.191+0000 7f0619600700 1 -- 172.21.15.112:0/253649362 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f061c0a9670 con 0x562eeb2989e0 2024-05-08T05:58:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.191+0000 7f06408e3b80 10 client.25564 fetch_fsmap learned FSMap version 29 2024-05-08T05:58:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.191+0000 7f06408e3b80 1 -- 172.21.15.112:0/253649362 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562eeb2aa530 con 0x562eeb2989e0 2024-05-08T05:58:58.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.192+0000 7f0619600700 1 -- 172.21.15.112:0/253649362 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06200043d0 con 0x562eeb2989e0 2024-05-08T05:58:58.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.192+0000 7f0619600700 1 -- 172.21.15.112:0/253649362 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f061c0a8dc0 con 0x562eeb2989e0 2024-05-08T05:58:58.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.192+0000 7f06408e3b80 10 client.25564 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:58:58.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.192+0000 7f06408e3b80 10 client.25564 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T05:58:58.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.192+0000 7f06408e3b80 20 client.25564 populate_metadata read hostname 'smithi112' 2024-05-08T05:58:58.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.192+0000 7f06408e3b80 1 --2- 172.21.15.112:0/253649362 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562eeb2ace90 0x562eeb2af330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.192+0000 7f06408e3b80 4 client.25564 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:58:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.192+0000 7f06408e3b80 1 -- 172.21.15.112:0/253649362 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eeb2600a0 con 0x562eeb2ace90 2024-05-08T05:58:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.192+0000 7f062f400700 1 --2- 172.21.15.112:0/253649362 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562eeb2ace90 0x562eeb2af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.193+0000 7f062f400700 1 --2- 172.21.15.112:0/253649362 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562eeb2ace90 0x562eeb2af330 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71c3ddb81d14fc20 server_cookie=777d9a38d03b824a in_seq=0 out_seq=0 2024-05-08T05:58:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.193+0000 7f0619600700 10 client.25564 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:58:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.194+0000 7f0619600700 1 -- 172.21.15.112:0/253649362 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562eeb2600a0 con 0x562eeb2ace90 2024-05-08T05:58:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.194+0000 7f0619600700 10 client.25564 handle_command_reply: tid=0 2024-05-08T05:58:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.194+0000 7f06408e3b80 1 client.25564 shutdown 2024-05-08T05:58:58.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.194+0000 7f06408e3b80 1 -- 172.21.15.112:0/253649362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb2989e0 msgr2=0x562eeb29c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:58.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.195+0000 7f06408e3b80 1 --2- 172.21.15.112:0/253649362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb2989e0 0x562eeb29c2d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f061c0a9860 tx=0x7f061c0a5b50 comp rx=0 tx=0).stop 2024-05-08T05:58:58.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.195+0000 7f06408e3b80 1 -- 172.21.15.112:0/253649362 shutdown_connections 2024-05-08T05:58:58.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.195+0000 7f06408e3b80 1 --2- 172.21.15.112:0/253649362 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562eeb2ace90 0x562eeb2af330 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.195+0000 7f06408e3b80 1 --2- 172.21.15.112:0/253649362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb299300 0x562eeb29c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.195+0000 7f0619600700 0 client.25564 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:58:58.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.195+0000 7f06408e3b80 1 --2- 172.21.15.112:0/253649362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb293600 0x562eeb29eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.195+0000 7f06408e3b80 1 --2- 172.21.15.112:0/253649362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb2989e0 0x562eeb29c2d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.195+0000 7f06408e3b80 1 -- 172.21.15.112:0/253649362 >> 172.21.15.112:0/253649362 conn(0x562eeb1c1b60 msgr2=0x562eeb288460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:58.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.195+0000 7f06408e3b80 1 -- 172.21.15.112:0/253649362 shutdown_connections 2024-05-08T05:58:58.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.195+0000 7f06408e3b80 1 -- 172.21.15.112:0/253649362 wait complete. 2024-05-08T05:58:58.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.195+0000 7f06408e3b80 20 client.25564 trim_cache size 0 max 16384 2024-05-08T05:58:58.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.214+0000 7f06408e3b80 1 Processor -- start 2024-05-08T05:58:58.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.214+0000 7f06408e3b80 1 -- start start 2024-05-08T05:58:58.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.214+0000 7f06408e3b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb293600 0x562eeb292fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.214+0000 7f06408e3b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb2989e0 0x562eeb29e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.214+0000 7f06408e3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb299300 0x562eeb299780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.214+0000 7f06408e3b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06301ae350 con 0x562eeb293600 2024-05-08T05:58:58.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.214+0000 7f06408e3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f06301abb50 con 0x562eeb299300 2024-05-08T05:58:58.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.214+0000 7f06408e3b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f063019d810 con 0x562eeb2989e0 2024-05-08T05:58:58.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.215+0000 7f062f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb293600 0x562eeb292fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.215+0000 7f062f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb293600 0x562eeb292fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40782/0 (socket says 172.21.15.112:40782) 2024-05-08T05:58:58.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.215+0000 7f062f400700 1 -- 172.21.15.112:0/148593236 learned_addr learned my addr 172.21.15.112:0/148593236 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:58.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.215+0000 7f062ea00700 1 --2- 172.21.15.112:0/148593236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb2989e0 0x562eeb29e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.215+0000 7f062f400700 1 -- 172.21.15.112:0/148593236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb2989e0 msgr2=0x562eeb29e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:58.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.215+0000 7f062f400700 1 --2- 172.21.15.112:0/148593236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb2989e0 0x562eeb29e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.215+0000 7f062f400700 1 -- 172.21.15.112:0/148593236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb299300 msgr2=0x562eeb299780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:58:58.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.215+0000 7f062f400700 1 --2- 172.21.15.112:0/148593236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb299300 0x562eeb299780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.215+0000 7f062f400700 1 -- 172.21.15.112:0/148593236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eeb2aa340 con 0x562eeb293600 2024-05-08T05:58:58.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.216+0000 7f062ea00700 1 --2- 172.21.15.112:0/148593236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb2989e0 0x562eeb29e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:58:58.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.216+0000 7f062f400700 1 --2- 172.21.15.112:0/148593236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb293600 0x562eeb292fe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0620009d00 tx=0x7f06200038d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1dfad5c2066361 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:58.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.216+0000 7f061be00700 1 -- 172.21.15.112:0/148593236 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620030070 con 0x562eeb293600 2024-05-08T05:58:58.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.216+0000 7f061be00700 1 -- 172.21.15.112:0/148593236 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0620005040 con 0x562eeb293600 2024-05-08T05:58:58.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.216+0000 7f061be00700 1 -- 172.21.15.112:0/148593236 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062002a550 con 0x562eeb293600 2024-05-08T05:58:58.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.217+0000 7f06408e3b80 1 -- 172.21.15.112:0/148593236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb293600 msgr2=0x562eeb292fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:58.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.217+0000 7f06408e3b80 1 --2- 172.21.15.112:0/148593236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb293600 0x562eeb292fe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0620009d00 tx=0x7f06200038d0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.218+0000 7f06408e3b80 1 -- 172.21.15.112:0/148593236 shutdown_connections 2024-05-08T05:58:58.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.218+0000 7f06408e3b80 1 --2- 172.21.15.112:0/148593236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb299300 0x562eeb299780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.218+0000 7f06408e3b80 1 --2- 172.21.15.112:0/148593236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb293600 0x562eeb292fe0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.218+0000 7f06408e3b80 1 --2- 172.21.15.112:0/148593236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb2989e0 0x562eeb29e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.218+0000 7f06408e3b80 1 -- 172.21.15.112:0/148593236 >> 172.21.15.112:0/148593236 conn(0x562eeb1c1b60 msgr2=0x562eeb29abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:58.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.218+0000 7f06408e3b80 1 -- 172.21.15.112:0/148593236 shutdown_connections 2024-05-08T05:58:58.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.218+0000 7f06408e3b80 1 -- 172.21.15.112:0/148593236 wait complete. 2024-05-08T05:58:58.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.219+0000 7f06408e3b80 1 -- start start 2024-05-08T05:58:58.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.219+0000 7f06408e3b80 1 Processor -- start 2024-05-08T05:58:58.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.219+0000 7f06408e3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb293600 0x562eeb2b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.219+0000 7f06408e3b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb2989e0 0x562eeb2b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.219+0000 7f06408e3b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb299300 0x562eeb2b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.220+0000 7f06408e3b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06301aa010 con 0x562eeb2989e0 2024-05-08T05:58:58.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.220+0000 7f06408e3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f06301ab480 con 0x562eeb293600 2024-05-08T05:58:58.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.220+0000 7f06408e3b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f06301aa6e0 con 0x562eeb299300 2024-05-08T05:58:58.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.220+0000 7f06408e3b80 10 client.0 _ll_register_callbacks cb 0x562eeb287ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:58:58.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.220+0000 7f062f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb293600 0x562eeb2b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.220+0000 7f062ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb2989e0 0x562eeb2b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.220+0000 7f062f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb293600 0x562eeb2b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:47560/0 (socket says 172.21.15.112:47560) 2024-05-08T05:58:58.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.220+0000 7f062f400700 1 -- 172.21.15.112:0/447057327 learned_addr learned my addr 172.21.15.112:0/447057327 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:58:58.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.220+0000 7f062fe00700 1 --2- 172.21.15.112:0/447057327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb299300 0x562eeb2b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.221+0000 7f062ea00700 1 -- 172.21.15.112:0/447057327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb299300 msgr2=0x562eeb2b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:58.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.221+0000 7f062ea00700 1 --2- 172.21.15.112:0/447057327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb299300 0x562eeb2b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.221+0000 7f062ea00700 1 -- 172.21.15.112:0/447057327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb293600 msgr2=0x562eeb2b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:58:58.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.221+0000 7f062ea00700 1 --2- 172.21.15.112:0/447057327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb293600 0x562eeb2b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.221+0000 7f062ea00700 1 -- 172.21.15.112:0/447057327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eeb2b26e0 con 0x562eeb2989e0 2024-05-08T05:58:58.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.221+0000 7f062f400700 1 --2- 172.21.15.112:0/447057327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb293600 0x562eeb2b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:58:58.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.221+0000 7f062ea00700 1 --2- 172.21.15.112:0/447057327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb2989e0 0x562eeb2b4d50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f061c0b0750 tx=0x7f061c0e42e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41be1037c33e6093 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:58.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.221+0000 7f0619600700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:58:58.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.221+0000 7f0619600700 1 -- 172.21.15.112:0/447057327 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c0a52f0 con 0x562eeb2989e0 2024-05-08T05:58:58.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.222+0000 7f0619600700 1 -- 172.21.15.112:0/447057327 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f061c0aab60 con 0x562eeb2989e0 2024-05-08T05:58:58.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.224+0000 7f06408e3b80 1 -- 172.21.15.112:0/447057327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0620005040 con 0x562eeb2989e0 2024-05-08T05:58:58.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.224+0000 7f0619600700 1 -- 172.21.15.112:0/447057327 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f061c0a90e0 con 0x562eeb2989e0 2024-05-08T05:58:58.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.224+0000 7f06408e3b80 10 client.15414 fetch_fsmap learned FSMap version 29 2024-05-08T05:58:58.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.224+0000 7f06408e3b80 1 -- 172.21.15.112:0/447057327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562eeb3170a0 con 0x562eeb2989e0 2024-05-08T05:58:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.225+0000 7f0619600700 1 -- 172.21.15.112:0/447057327 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0620005040 con 0x562eeb2989e0 2024-05-08T05:58:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.225+0000 7f0619600700 1 -- 172.21.15.112:0/447057327 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f061c0a8dc0 con 0x562eeb2989e0 2024-05-08T05:58:58.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.225+0000 7f06408e3b80 10 client.15414 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:58:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.225+0000 7f06408e3b80 10 client.15414 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T05:58:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.225+0000 7f06408e3b80 20 client.15414 populate_metadata read hostname 'smithi112' 2024-05-08T05:58:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.225+0000 7f06408e3b80 1 --2- 172.21.15.112:0/447057327 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562eeb2ace90 0x562eeb3197e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.225+0000 7f06408e3b80 4 client.15414 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:58:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.225+0000 7f06408e3b80 1 -- 172.21.15.112:0/447057327 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562eeb1f3400 con 0x562eeb2ace90 2024-05-08T05:58:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.226+0000 7f062f400700 1 --2- 172.21.15.112:0/447057327 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562eeb2ace90 0x562eeb3197e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.226+0000 7f062f400700 1 --2- 172.21.15.112:0/447057327 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562eeb2ace90 0x562eeb3197e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=246dbe9e30449c56 server_cookie=d7e9e10417c3851e in_seq=0 out_seq=0 2024-05-08T05:58:58.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.226+0000 7f0619600700 10 client.15414 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:58:58.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.227+0000 7f0619600700 1 -- 172.21.15.112:0/447057327 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562eeb1f3400 con 0x562eeb2ace90 2024-05-08T05:58:58.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.227+0000 7f0619600700 10 client.15414 handle_command_reply: tid=0 2024-05-08T05:58:58.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.227+0000 7f06408e3b80 1 client.15414 shutdown 2024-05-08T05:58:58.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.227+0000 7f06408e3b80 1 -- 172.21.15.112:0/447057327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb2989e0 msgr2=0x562eeb2b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:58.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.227+0000 7f06408e3b80 1 --2- 172.21.15.112:0/447057327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb2989e0 0x562eeb2b4d50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f061c0b0750 tx=0x7f061c0e42e0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.227+0000 7f06408e3b80 1 -- 172.21.15.112:0/447057327 shutdown_connections 2024-05-08T05:58:58.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.227+0000 7f06408e3b80 1 --2- 172.21.15.112:0/447057327 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562eeb2ace90 0x562eeb3197e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.227+0000 7f06408e3b80 1 --2- 172.21.15.112:0/447057327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562eeb299300 0x562eeb2b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.227+0000 7f06408e3b80 1 --2- 172.21.15.112:0/447057327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562eeb293600 0x562eeb2b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.227+0000 7f0619600700 0 client.15414 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:58:58.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.227+0000 7f06408e3b80 1 --2- 172.21.15.112:0/447057327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb2989e0 0x562eeb2b4d50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.227+0000 7f06408e3b80 1 -- 172.21.15.112:0/447057327 >> 172.21.15.112:0/447057327 conn(0x562eeb1c1b60 msgr2=0x562eeb29a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:58.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.229+0000 7f06408e3b80 1 -- 172.21.15.112:0/447057327 shutdown_connections 2024-05-08T05:58:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.229+0000 7f06408e3b80 1 -- 172.21.15.112:0/447057327 wait complete. 2024-05-08T05:58:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.230+0000 7f06408e3b80 20 client.15414 trim_cache size 0 max 16384 2024-05-08T05:58:58.231 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:58:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.230+0000 7f0634a00700 1 -- 172.21.15.112:0/2950429908 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f06140b2900 msgr2=0x7f06140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:58.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.230+0000 7f0634a00700 1 --2- 172.21.15.112:0/2950429908 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f06140b2900 0x7f06140b4dc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f062800b780 tx=0x7f0628009ce0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.230+0000 7f0634a00700 1 -- 172.21.15.112:0/2950429908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0630103d90 msgr2=0x7f0630075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.230+0000 7f0634a00700 1 --2- 172.21.15.112:0/2950429908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0630103d90 0x7f0630075ce0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f061c00e9a0 tx=0x7f061c00ecb0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.230+0000 7f0634a00700 1 -- 172.21.15.112:0/2950429908 shutdown_connections 2024-05-08T05:58:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.231+0000 7f0634a00700 1 --2- 172.21.15.112:0/2950429908 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f06140b2900 0x7f06140b4dc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.231+0000 7f0634a00700 1 --2- 172.21.15.112:0/2950429908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0630109460 0x7f0630076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.231+0000 7f0634a00700 1 --2- 172.21.15.112:0/2950429908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06301033d0 0x7f06300757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.231+0000 7f0634a00700 1 --2- 172.21.15.112:0/2950429908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0630103d90 0x7f0630075ce0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.231+0000 7f0634a00700 1 -- 172.21.15.112:0/2950429908 >> 172.21.15.112:0/2950429908 conn(0x7f06300fef20 msgr2=0x7f06300ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.231+0000 7f0634a00700 1 -- 172.21.15.112:0/2950429908 shutdown_connections 2024-05-08T05:58:58.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:58:58.231+0000 7f0634a00700 1 -- 172.21.15.112:0/2950429908 wait complete. 2024-05-08T05:58:59.108 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-08T05:58:59.584 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:58:59.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:58:59 smithi150 ceph-mon[34315]: pgmap v551: 97 pgs: 97 active+clean; 74 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 292 op/s 2024-05-08T05:58:59.735 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:58:59 smithi112 ceph-mon[30003]: pgmap v551: 97 pgs: 97 active+clean; 74 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 292 op/s 2024-05-08T05:58:59.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:58:59 smithi162 ceph-mon[34800]: pgmap v551: 97 pgs: 97 active+clean; 74 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 292 op/s 2024-05-08T05:59:01.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:01 smithi150 ceph-mon[34315]: pgmap v552: 97 pgs: 97 active+clean; 74 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 322 op/s 2024-05-08T05:59:01.754 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xffd97 2024-05-08T05:59:01.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:01 smithi112 ceph-mon[30003]: pgmap v552: 97 pgs: 97 active+clean; 74 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 322 op/s 2024-05-08T05:59:01.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:01 smithi162 ceph-mon[34800]: pgmap v552: 97 pgs: 97 active+clean; 74 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 322 op/s 2024-05-08T05:59:02.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.448+0000 7fcb9da00700 1 -- 172.21.15.112:0/3015718088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb98103fd0 msgr2=0x7fcb98108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:02.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.448+0000 7fcb9da00700 1 --2- 172.21.15.112:0/3015718088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb98103fd0 0x7fcb98108020 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009990 tx=0x7fcb84009ca0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.448+0000 7fcb9da00700 1 -- 172.21.15.112:0/3015718088 shutdown_connections 2024-05-08T05:59:02.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.448+0000 7fcb9da00700 1 --2- 172.21.15.112:0/3015718088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb98108700 0x7fcb981102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.448+0000 7fcb9da00700 1 --2- 172.21.15.112:0/3015718088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb98103620 0x7fcb98103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.448+0000 7fcb9da00700 1 --2- 172.21.15.112:0/3015718088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb98103fd0 0x7fcb98108020 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.448+0000 7fcb9da00700 1 -- 172.21.15.112:0/3015718088 >> 172.21.15.112:0/3015718088 conn(0x7fcb980feef0 msgr2=0x7fcb98101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:02.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.449+0000 7fcb9da00700 1 -- 172.21.15.112:0/3015718088 shutdown_connections 2024-05-08T05:59:02.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.449+0000 7fcb9da00700 1 -- 172.21.15.112:0/3015718088 wait complete. 2024-05-08T05:59:02.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.450+0000 7fcb9da00700 1 Processor -- start 2024-05-08T05:59:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.450+0000 7fcb9da00700 1 -- start start 2024-05-08T05:59:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.450+0000 7fcb9da00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb98103620 0x7fcb9819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.450+0000 7fcb9da00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb98103fd0 0x7fcb9819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.450+0000 7fcb9da00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb98108700 0x7fcb98199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.450+0000 7fcb9da00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb98113570 con 0x7fcb98108700 2024-05-08T05:59:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.450+0000 7fcb9da00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcb981133f0 con 0x7fcb98103620 2024-05-08T05:59:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.450+0000 7fcb9da00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fcb981136f0 con 0x7fcb98103fd0 2024-05-08T05:59:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.450+0000 7fcb97e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb98103620 0x7fcb9819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.450+0000 7fcb97e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb98103620 0x7fcb9819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:47580/0 (socket says 172.21.15.112:47580) 2024-05-08T05:59:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.450+0000 7fcb97400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb98103fd0 0x7fcb9819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.450+0000 7fcb97e00700 1 -- 172.21.15.112:0/951215895 learned_addr learned my addr 172.21.15.112:0/951215895 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.451+0000 7fcb9d000700 1 --2- 172.21.15.112:0/951215895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb98108700 0x7fcb98199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.451+0000 7fcb97e00700 1 -- 172.21.15.112:0/951215895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb98103fd0 msgr2=0x7fcb9819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:02.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.451+0000 7fcb97e00700 1 --2- 172.21.15.112:0/951215895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb98103fd0 0x7fcb9819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.451+0000 7fcb97e00700 1 -- 172.21.15.112:0/951215895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb98108700 msgr2=0x7fcb98199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:02.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.451+0000 7fcb97e00700 1 --2- 172.21.15.112:0/951215895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb98108700 0x7fcb98199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.451+0000 7fcb97e00700 1 -- 172.21.15.112:0/951215895 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9819a090 con 0x7fcb98103620 2024-05-08T05:59:02.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.451+0000 7fcb97e00700 1 --2- 172.21.15.112:0/951215895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb98103620 0x7fcb9819f720 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800e9a0 tx=0x7fcb8800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:02.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.452+0000 7fcb94c00700 1 -- 172.21.15.112:0/951215895 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8800ca10 con 0x7fcb98103620 2024-05-08T05:59:02.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.452+0000 7fcb9da00700 1 -- 172.21.15.112:0/951215895 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9819a380 con 0x7fcb98103620 2024-05-08T05:59:02.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.452+0000 7fcb94c00700 1 -- 172.21.15.112:0/951215895 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb8800cbb0 con 0x7fcb98103620 2024-05-08T05:59:02.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.452+0000 7fcb94c00700 1 -- 172.21.15.112:0/951215895 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88018680 con 0x7fcb98103620 2024-05-08T05:59:02.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.452+0000 7fcb9da00700 1 -- 172.21.15.112:0/951215895 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb980756d0 con 0x7fcb98103620 2024-05-08T05:59:02.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.457+0000 7fcb94c00700 1 -- 172.21.15.112:0/951215895 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fcb88010430 con 0x7fcb98103620 2024-05-08T05:59:02.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.459+0000 7fcba99b9b80 1 Processor -- start 2024-05-08T05:59:02.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.459+0000 7fcba99b9b80 1 -- start start 2024-05-08T05:59:02.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.459+0000 7fcb94c00700 1 --2- 172.21.15.112:0/951215895 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcb7c0b2880 0x7fcb7c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.459+0000 7fcba99b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a1349e0 0x55d76a134dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.459+0000 7fcba99b9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a135300 0x55d76a12efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcba99b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a12f600 0x55d76a12fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcba99b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d76a081a20 con 0x55d76a12f600 2024-05-08T05:59:02.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcba99b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d76a01f760 con 0x55d76a1349e0 2024-05-08T05:59:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcba99b9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d76a07d9e0 con 0x55d76a135300 2024-05-08T05:59:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcb94c00700 1 -- 172.21.15.112:0/951215895 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fcb880e2390 con 0x7fcb98103620 2024-05-08T05:59:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcb9d000700 1 --2- 172.21.15.112:0/951215895 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcb7c0b2880 0x7fcb7c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcb97e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a1349e0 0x55d76a134dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcb97e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a1349e0 0x55d76a134dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:47584/0 (socket says 172.21.15.112:47584) 2024-05-08T05:59:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcb97400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a135300 0x55d76a12efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcb97e00700 1 -- 172.21.15.112:0/3790826519 learned_addr learned my addr 172.21.15.112:0/3790826519 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcb9d000700 1 --2- 172.21.15.112:0/3790826519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a12f600 0x55d76a12fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcb97e00700 1 -- 172.21.15.112:0/3790826519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a135300 msgr2=0x55d76a12efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcb97e00700 1 --2- 172.21.15.112:0/3790826519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a135300 0x55d76a12efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcb97e00700 1 -- 172.21.15.112:0/3790826519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a12f600 msgr2=0x55d76a12fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcb97e00700 1 --2- 172.21.15.112:0/3790826519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a12f600 0x55d76a12fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.460+0000 7fcb97e00700 1 -- 172.21.15.112:0/3790826519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d769fce8d0 con 0x55d76a1349e0 2024-05-08T05:59:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.461+0000 7fcb97e00700 1 --2- 172.21.15.112:0/3790826519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a1349e0 0x55d76a134dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcb880a5a00 tx=0x7fcb880a9170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c3b35d095a8f708 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.461+0000 7fcb9d000700 1 --2- 172.21.15.112:0/951215895 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcb7c0b2880 0x7fcb7c0b4d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55d76a12ffc0 tx=0x7fcb90009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.461+0000 7fcb8d600700 1 -- 172.21.15.112:0/3790826519 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb880207e0 con 0x55d76a1349e0 2024-05-08T05:59:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.461+0000 7fcb8d600700 1 -- 172.21.15.112:0/3790826519 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb880b2120 con 0x55d76a1349e0 2024-05-08T05:59:02.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.461+0000 7fcb8d600700 1 -- 172.21.15.112:0/3790826519 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88017b30 con 0x55d76a1349e0 2024-05-08T05:59:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.462+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3790826519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a1349e0 msgr2=0x55d76a134dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.462+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/3790826519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a1349e0 0x55d76a134dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcb880a5a00 tx=0x7fcb880a9170 comp rx=0 tx=0).stop 2024-05-08T05:59:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.462+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3790826519 shutdown_connections 2024-05-08T05:59:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.462+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/3790826519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a12f600 0x55d76a12fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.462+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/3790826519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a1349e0 0x55d76a134dc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.462+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/3790826519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a135300 0x55d76a12efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.462+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3790826519 >> 172.21.15.112:0/3790826519 conn(0x55d76a05db60 msgr2=0x55d76a1299b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:02.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.462+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3790826519 shutdown_connections 2024-05-08T05:59:02.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.462+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3790826519 wait complete. 2024-05-08T05:59:02.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.462+0000 7fcba99b9b80 1 -- start start 2024-05-08T05:59:02.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.463+0000 7fcba99b9b80 1 Processor -- start 2024-05-08T05:59:02.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.463+0000 7fcba99b9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a12f600 0x55d76a13af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.463+0000 7fcba99b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a1349e0 0x55d76a1382d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.463+0000 7fcba99b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a135300 0x55d76a138810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.463+0000 7fcba99b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d76a13bd50 con 0x55d76a1349e0 2024-05-08T05:59:02.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.463+0000 7fcba99b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d769fe7b20 con 0x55d76a135300 2024-05-08T05:59:02.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.463+0000 7fcba99b9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d76a13bed0 con 0x55d76a12f600 2024-05-08T05:59:02.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.463+0000 7fcba99b9b80 10 client.0 _ll_register_callbacks cb 0x55d76a028c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:59:02.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.463+0000 7fcb97e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a12f600 0x55d76a13af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.464+0000 7fcb9d000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a135300 0x55d76a138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.464+0000 7fcb9d000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a135300 0x55d76a138810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:47598/0 (socket says 172.21.15.112:47598) 2024-05-08T05:59:02.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.464+0000 7fcb97e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a12f600 0x55d76a13af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:56406/0 (socket says 172.21.15.112:56406) 2024-05-08T05:59:02.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.464+0000 7fcb9d000700 1 -- 172.21.15.112:0/16439605 learned_addr learned my addr 172.21.15.112:0/16439605 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:02.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.464+0000 7fcb97400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a1349e0 0x55d76a1382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.464+0000 7fcb9d000700 1 -- 172.21.15.112:0/16439605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a12f600 msgr2=0x55d76a13af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.464+0000 7fcb9d000700 1 --2- 172.21.15.112:0/16439605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a12f600 0x55d76a13af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.464+0000 7fcb9d000700 1 -- 172.21.15.112:0/16439605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a1349e0 msgr2=0x55d76a1382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.464+0000 7fcb9d000700 1 --2- 172.21.15.112:0/16439605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a1349e0 0x55d76a1382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.464+0000 7fcb9d000700 1 -- 172.21.15.112:0/16439605 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d76a047420 con 0x55d76a135300 2024-05-08T05:59:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.464+0000 7fcb97e00700 1 --2- 172.21.15.112:0/16439605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a12f600 0x55d76a13af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:59:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.464+0000 7fcb9d000700 1 --2- 172.21.15.112:0/16439605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a135300 0x55d76a138810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb90018920 tx=0x7fcb90004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc17c0cd8689b207 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:02.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.464+0000 7fcb82000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:59:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.465+0000 7fcb82000700 1 -- 172.21.15.112:0/16439605 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9002a060 con 0x55d76a135300 2024-05-08T05:59:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.465+0000 7fcba99b9b80 1 -- 172.21.15.112:0/16439605 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d76a0fc0a0 con 0x55d76a135300 2024-05-08T05:59:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.465+0000 7fcb82000700 1 -- 172.21.15.112:0/16439605 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fcb90012040 con 0x55d76a135300 2024-05-08T05:59:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.465+0000 7fcb82000700 1 -- 172.21.15.112:0/16439605 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcb9001a680 con 0x55d76a135300 2024-05-08T05:59:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.465+0000 7fcba99b9b80 10 client.25541 fetch_fsmap learned FSMap version 29 2024-05-08T05:59:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.465+0000 7fcba99b9b80 1 -- 172.21.15.112:0/16439605 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d76a14ed50 con 0x55d76a135300 2024-05-08T05:59:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.465+0000 7fcb82000700 1 -- 172.21.15.112:0/16439605 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d76a0fc0a0 con 0x55d76a135300 2024-05-08T05:59:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.466+0000 7fcb82000700 1 -- 172.21.15.112:0/16439605 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fcb900259f0 con 0x55d76a135300 2024-05-08T05:59:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.466+0000 7fcba99b9b80 10 client.25541 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:59:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.466+0000 7fcba99b9b80 10 client.25541 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T05:59:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.466+0000 7fcba99b9b80 20 client.25541 populate_metadata read hostname 'smithi112' 2024-05-08T05:59:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.466+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/16439605 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d76a1512e0 0x55d76a1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.466+0000 7fcba99b9b80 4 client.25541 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:59:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.466+0000 7fcba99b9b80 1 -- 172.21.15.112:0/16439605 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d76a08f400 con 0x55d76a1512e0 2024-05-08T05:59:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.466+0000 7fcb97e00700 1 --2- 172.21.15.112:0/16439605 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d76a1512e0 0x55d76a1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.467+0000 7fcb82000700 10 client.25541 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T05:59:02.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.467+0000 7fcb97e00700 1 --2- 172.21.15.112:0/16439605 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d76a1512e0 0x55d76a1536d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8bc7a5db91bf1cc server_cookie=6bc9c290e6e427b9 in_seq=0 out_seq=0 2024-05-08T05:59:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.468+0000 7fcb82000700 1 -- 172.21.15.112:0/16439605 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d76a08f400 con 0x55d76a1512e0 2024-05-08T05:59:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.468+0000 7fcb82000700 10 client.25541 handle_command_reply: tid=0 2024-05-08T05:59:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.468+0000 7fcba99b9b80 1 client.25541 shutdown 2024-05-08T05:59:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.468+0000 7fcba99b9b80 1 -- 172.21.15.112:0/16439605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a135300 msgr2=0x55d76a138810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.468+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/16439605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a135300 0x55d76a138810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb90018920 tx=0x7fcb90004b40 comp rx=0 tx=0).stop 2024-05-08T05:59:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.468+0000 7fcba99b9b80 1 -- 172.21.15.112:0/16439605 shutdown_connections 2024-05-08T05:59:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.468+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/16439605 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d76a1512e0 0x55d76a1536d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.468+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/16439605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a135300 0x55d76a138810 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.468+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/16439605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a12f600 0x55d76a13af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.468+0000 7fcb82000700 0 client.25541 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T05:59:02.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.468+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/16439605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a1349e0 0x55d76a1382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.468+0000 7fcba99b9b80 1 -- 172.21.15.112:0/16439605 >> 172.21.15.112:0/16439605 conn(0x55d76a05db60 msgr2=0x55d76a124460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:02.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.469+0000 7fcba99b9b80 1 -- 172.21.15.112:0/16439605 shutdown_connections 2024-05-08T05:59:02.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.469+0000 7fcba99b9b80 1 -- 172.21.15.112:0/16439605 wait complete. 2024-05-08T05:59:02.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.469+0000 7fcba99b9b80 20 client.25541 trim_cache size 0 max 16384 2024-05-08T05:59:02.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.489+0000 7fcba99b9b80 1 Processor -- start 2024-05-08T05:59:02.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.489+0000 7fcba99b9b80 1 -- start start 2024-05-08T05:59:02.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.490+0000 7fcba99b9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a12f600 0x55d76a134dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.490+0000 7fcba99b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a1349e0 0x55d76a136bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.490+0000 7fcba99b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a135300 0x55d76a137100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.490+0000 7fcba99b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb981ae3b0 con 0x55d76a135300 2024-05-08T05:59:02.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.490+0000 7fcba99b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcb981abbb0 con 0x55d76a1349e0 2024-05-08T05:59:02.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.490+0000 7fcba99b9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fcb981a9a20 con 0x55d76a12f600 2024-05-08T05:59:02.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.490+0000 7fcb97e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a12f600 0x55d76a134dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.490+0000 7fcb97400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a1349e0 0x55d76a136bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.490+0000 7fcb97e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a12f600 0x55d76a134dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:56412/0 (socket says 172.21.15.112:56412) 2024-05-08T05:59:02.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.490+0000 7fcb97e00700 1 -- 172.21.15.112:0/3123106027 learned_addr learned my addr 172.21.15.112:0/3123106027 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:02.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.490+0000 7fcb9d000700 1 --2- 172.21.15.112:0/3123106027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a135300 0x55d76a137100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.490+0000 7fcb97400700 1 -- 172.21.15.112:0/3123106027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a12f600 msgr2=0x55d76a134dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:02.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.490+0000 7fcb97400700 1 --2- 172.21.15.112:0/3123106027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a12f600 0x55d76a134dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.490+0000 7fcb97400700 1 -- 172.21.15.112:0/3123106027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a135300 msgr2=0x55d76a137100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:02.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.490+0000 7fcb97400700 1 --2- 172.21.15.112:0/3123106027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a135300 0x55d76a137100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.491+0000 7fcb97400700 1 -- 172.21.15.112:0/3123106027 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d76a129760 con 0x55d76a1349e0 2024-05-08T05:59:02.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.491+0000 7fcb9d000700 1 --2- 172.21.15.112:0/3123106027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a135300 0x55d76a137100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:59:02.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.491+0000 7fcb97e00700 1 --2- 172.21.15.112:0/3123106027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a12f600 0x55d76a134dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:59:02.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.491+0000 7fcb97400700 1 --2- 172.21.15.112:0/3123106027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a1349e0 0x55d76a136bc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb840059d0 tx=0x7fcb84005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c31a1bcbc1c883f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:02.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.491+0000 7fcb8cc00700 1 -- 172.21.15.112:0/3123106027 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8402b070 con 0x55d76a1349e0 2024-05-08T05:59:02.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.491+0000 7fcb8cc00700 1 -- 172.21.15.112:0/3123106027 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb84005690 con 0x55d76a1349e0 2024-05-08T05:59:02.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.491+0000 7fcb8cc00700 1 -- 172.21.15.112:0/3123106027 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84003c40 con 0x55d76a1349e0 2024-05-08T05:59:02.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.492+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3123106027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a1349e0 msgr2=0x55d76a136bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:02.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.492+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/3123106027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a1349e0 0x55d76a136bc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb840059d0 tx=0x7fcb84005040 comp rx=0 tx=0).stop 2024-05-08T05:59:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.492+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3123106027 shutdown_connections 2024-05-08T05:59:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.492+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/3123106027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a135300 0x55d76a137100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.492+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/3123106027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a12f600 0x55d76a134dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.492+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/3123106027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a1349e0 0x55d76a136bc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.492+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3123106027 >> 172.21.15.112:0/3123106027 conn(0x55d76a05db60 msgr2=0x55d76a12b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.492+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3123106027 shutdown_connections 2024-05-08T05:59:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.492+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3123106027 wait complete. 2024-05-08T05:59:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.493+0000 7fcba99b9b80 1 -- start start 2024-05-08T05:59:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.493+0000 7fcba99b9b80 1 Processor -- start 2024-05-08T05:59:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.494+0000 7fcba99b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a12f600 0x55d76a149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.494+0000 7fcba99b9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a1349e0 0x55d76a14a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.494+0000 7fcba99b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a135300 0x55d76a157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.494+0000 7fcba99b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb981aa070 con 0x55d76a12f600 2024-05-08T05:59:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.494+0000 7fcba99b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcb981ab4e0 con 0x55d76a135300 2024-05-08T05:59:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.494+0000 7fcba99b9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fcb981aa740 con 0x55d76a1349e0 2024-05-08T05:59:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.494+0000 7fcba99b9b80 10 client.0 _ll_register_callbacks cb 0x55d76a028c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:59:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.494+0000 7fcb97e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a12f600 0x55d76a149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.494+0000 7fcb97e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a12f600 0x55d76a149bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40840/0 (socket says 172.21.15.112:40840) 2024-05-08T05:59:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.494+0000 7fcb97e00700 1 -- 172.21.15.112:0/3154495852 learned_addr learned my addr 172.21.15.112:0/3154495852 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.494+0000 7fcb9d000700 1 --2- 172.21.15.112:0/3154495852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a135300 0x55d76a157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.495+0000 7fcb97e00700 1 -- 172.21.15.112:0/3154495852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a1349e0 msgr2=0x55d76a14a110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:59:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.495+0000 7fcb97e00700 1 --2- 172.21.15.112:0/3154495852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a1349e0 0x55d76a14a110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.495+0000 7fcb97e00700 1 -- 172.21.15.112:0/3154495852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a135300 msgr2=0x55d76a157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.495+0000 7fcb97e00700 1 --2- 172.21.15.112:0/3154495852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a135300 0x55d76a157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.495+0000 7fcb97e00700 1 -- 172.21.15.112:0/3154495852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d76a1587e0 con 0x55d76a12f600 2024-05-08T05:59:02.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.495+0000 7fcb9d000700 1 --2- 172.21.15.112:0/3154495852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a135300 0x55d76a157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:59:02.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.495+0000 7fcb97e00700 1 --2- 172.21.15.112:0/3154495852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a12f600 0x55d76a149bd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcb880dfeb0 tx=0x7fcb88010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7297e4acd9eea08 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:02.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.495+0000 7fcb82000700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:59:02.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.496+0000 7fcb82000700 1 -- 172.21.15.112:0/3154495852 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8806d070 con 0x55d76a12f600 2024-05-08T05:59:02.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.496+0000 7fcb82000700 1 -- 172.21.15.112:0/3154495852 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fcb880a5f70 con 0x55d76a12f600 2024-05-08T05:59:02.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.496+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3154495852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb84005690 con 0x55d76a12f600 2024-05-08T05:59:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.496+0000 7fcb82000700 1 -- 172.21.15.112:0/3154495852 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcb880b27e0 con 0x55d76a12f600 2024-05-08T05:59:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.496+0000 7fcba99b9b80 10 client.15429 fetch_fsmap learned FSMap version 29 2024-05-08T05:59:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.496+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3154495852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d76a1bb610 con 0x55d76a12f600 2024-05-08T05:59:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.497+0000 7fcb82000700 1 -- 172.21.15.112:0/3154495852 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb88087080 con 0x55d76a12f600 2024-05-08T05:59:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.497+0000 7fcb82000700 1 -- 172.21.15.112:0/3154495852 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fcb880207a0 con 0x55d76a12f600 2024-05-08T05:59:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.497+0000 7fcba99b9b80 10 client.15429 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:59:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.497+0000 7fcba99b9b80 10 client.15429 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T05:59:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.497+0000 7fcba99b9b80 20 client.15429 populate_metadata read hostname 'smithi112' 2024-05-08T05:59:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.497+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/3154495852 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d76a1bb950 0x55d76a1bdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.497+0000 7fcba99b9b80 4 client.15429 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:59:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.497+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3154495852 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d76a0894a0 con 0x55d76a1bb950 2024-05-08T05:59:02.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.497+0000 7fcb97400700 1 --2- 172.21.15.112:0/3154495852 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d76a1bb950 0x55d76a1bdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.498+0000 7fcb97400700 1 --2- 172.21.15.112:0/3154495852 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d76a1bb950 0x55d76a1bdd60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10b550f89cd8e41f server_cookie=a5a386f38f5653fb in_seq=0 out_seq=0 2024-05-08T05:59:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.498+0000 7fcb82000700 10 client.15429 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T05:59:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.498+0000 7fcb82000700 1 -- 172.21.15.112:0/3154495852 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d76a0894a0 con 0x55d76a1bb950 2024-05-08T05:59:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.498+0000 7fcb82000700 10 client.15429 handle_command_reply: tid=0 2024-05-08T05:59:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.498+0000 7fcba99b9b80 1 client.15429 shutdown 2024-05-08T05:59:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.498+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3154495852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a12f600 msgr2=0x55d76a149bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.498+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/3154495852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a12f600 0x55d76a149bd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcb880dfeb0 tx=0x7fcb88010c10 comp rx=0 tx=0).stop 2024-05-08T05:59:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.498+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3154495852 shutdown_connections 2024-05-08T05:59:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.498+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/3154495852 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d76a1bb950 0x55d76a1bdd60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.498+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/3154495852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d76a135300 0x55d76a157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.498+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/3154495852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d76a12f600 0x55d76a149bd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.498+0000 7fcb82000700 0 client.15429 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T05:59:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.498+0000 7fcba99b9b80 1 --2- 172.21.15.112:0/3154495852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d76a1349e0 0x55d76a14a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.498+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3154495852 >> 172.21.15.112:0/3154495852 conn(0x55d76a05db60 msgr2=0x55d76a136b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.499+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3154495852 shutdown_connections 2024-05-08T05:59:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.499+0000 7fcba99b9b80 1 -- 172.21.15.112:0/3154495852 wait complete. 2024-05-08T05:59:02.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.499+0000 7fcba99b9b80 20 client.15429 trim_cache size 0 max 16384 2024-05-08T05:59:02.502 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:59:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.500+0000 7fcb9da00700 1 -- 172.21.15.112:0/951215895 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcb7c0b2880 msgr2=0x7fcb7c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.500+0000 7fcb9da00700 1 --2- 172.21.15.112:0/951215895 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcb7c0b2880 0x7fcb7c0b4d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55d76a12ffc0 tx=0x7fcb90009920 comp rx=0 tx=0).stop 2024-05-08T05:59:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.500+0000 7fcb9da00700 1 -- 172.21.15.112:0/951215895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb98103620 msgr2=0x7fcb9819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.500+0000 7fcb9da00700 1 --2- 172.21.15.112:0/951215895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb98103620 0x7fcb9819f720 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800e9a0 tx=0x7fcb8800ecb0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.500+0000 7fcb9da00700 1 -- 172.21.15.112:0/951215895 shutdown_connections 2024-05-08T05:59:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.500+0000 7fcb9da00700 1 --2- 172.21.15.112:0/951215895 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fcb7c0b2880 0x7fcb7c0b4d40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.500+0000 7fcb9da00700 1 --2- 172.21.15.112:0/951215895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb98108700 0x7fcb98199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.500+0000 7fcb9da00700 1 --2- 172.21.15.112:0/951215895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb98103620 0x7fcb9819f720 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.500+0000 7fcb9da00700 1 --2- 172.21.15.112:0/951215895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb98103fd0 0x7fcb9819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:02.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.500+0000 7fcb9da00700 1 -- 172.21.15.112:0/951215895 >> 172.21.15.112:0/951215895 conn(0x7fcb980feef0 msgr2=0x7fcb98100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:02.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.500+0000 7fcb9da00700 1 -- 172.21.15.112:0/951215895 shutdown_connections 2024-05-08T05:59:02.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:02.500+0000 7fcb9da00700 1 -- 172.21.15.112:0/951215895 wait complete. 2024-05-08T05:59:03.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:03 smithi112 ceph-mon[30003]: pgmap v553: 97 pgs: 97 active+clean; 75 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-05-08T05:59:03.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:03 smithi150 ceph-mon[34315]: pgmap v553: 97 pgs: 97 active+clean; 75 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-05-08T05:59:03.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:03 smithi162 ceph-mon[34800]: pgmap v553: 97 pgs: 97 active+clean; 75 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-05-08T05:59:03.831 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-08T05:59:04.314 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:59:05.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:05 smithi150 ceph-mon[34315]: pgmap v554: 97 pgs: 97 active+clean; 74 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 305 op/s 2024-05-08T05:59:05.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:05 smithi112 ceph-mon[30003]: pgmap v554: 97 pgs: 97 active+clean; 74 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 305 op/s 2024-05-08T05:59:05.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:05 smithi162 ceph-mon[34800]: pgmap v554: 97 pgs: 97 active+clean; 74 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 305 op/s 2024-05-08T05:59:06.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.891+0000 7fb308000700 1 -- 172.21.15.112:0/3819670261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb300104160 msgr2=0x7fb300108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:06.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.891+0000 7fb308000700 1 --2- 172.21.15.112:0/3819670261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb300104160 0x7fb300108150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009920 tx=0x7fb2ec009c30 comp rx=0 tx=0).stop 2024-05-08T05:59:06.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.891+0000 7fb308000700 1 -- 172.21.15.112:0/3819670261 shutdown_connections 2024-05-08T05:59:06.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.891+0000 7fb308000700 1 --2- 172.21.15.112:0/3819670261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb300108880 0x7fb300110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.891+0000 7fb308000700 1 --2- 172.21.15.112:0/3819670261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3001037b0 0x7fb300103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.891+0000 7fb308000700 1 --2- 172.21.15.112:0/3819670261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb300104160 0x7fb300108150 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.891+0000 7fb308000700 1 -- 172.21.15.112:0/3819670261 >> 172.21.15.112:0/3819670261 conn(0x7fb30007a530 msgr2=0x7fb30007a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:06.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.892+0000 7fb308000700 1 -- 172.21.15.112:0/3819670261 shutdown_connections 2024-05-08T05:59:06.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.892+0000 7fb308000700 1 -- 172.21.15.112:0/3819670261 wait complete. 2024-05-08T05:59:06.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.893+0000 7fb308000700 1 Processor -- start 2024-05-08T05:59:06.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.893+0000 7fb308000700 1 -- start start 2024-05-08T05:59:06.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb308000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3001037b0 0x7fb30019be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb308000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb300104160 0x7fb30019c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb308000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb300108880 0x7fb3001a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb308000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb300113730 con 0x7fb3001037b0 2024-05-08T05:59:06.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb308000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb3001135b0 con 0x7fb300104160 2024-05-08T05:59:06.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb308000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb3001138b0 con 0x7fb300108880 2024-05-08T05:59:06.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb304c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb300104160 0x7fb30019c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:06.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb304c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb300104160 0x7fb30019c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:36684/0 (socket says 172.21.15.112:36684) 2024-05-08T05:59:06.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb304c00700 1 -- 172.21.15.112:0/1997230909 learned_addr learned my addr 172.21.15.112:0/1997230909 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:06.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb305600700 1 --2- 172.21.15.112:0/1997230909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3001037b0 0x7fb30019be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:06.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb306000700 1 --2- 172.21.15.112:0/1997230909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb300108880 0x7fb3001a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:06.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb304c00700 1 -- 172.21.15.112:0/1997230909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb300108880 msgr2=0x7fb3001a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:06.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb304c00700 1 --2- 172.21.15.112:0/1997230909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb300108880 0x7fb3001a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb304c00700 1 -- 172.21.15.112:0/1997230909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3001037b0 msgr2=0x7fb30019be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:06.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb304c00700 1 --2- 172.21.15.112:0/1997230909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3001037b0 0x7fb30019be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.894+0000 7fb304c00700 1 -- 172.21.15.112:0/1997230909 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3001a0e30 con 0x7fb300104160 2024-05-08T05:59:06.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.895+0000 7fb304c00700 1 --2- 172.21.15.112:0/1997230909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb300104160 0x7fb30019c360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00c010 tx=0x7fb2ec00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:06.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.895+0000 7fb2fe000700 1 -- 172.21.15.112:0/1997230909 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec027070 con 0x7fb300104160 2024-05-08T05:59:06.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.895+0000 7fb308000700 1 -- 172.21.15.112:0/1997230909 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3001a10c0 con 0x7fb300104160 2024-05-08T05:59:06.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.895+0000 7fb2fe000700 1 -- 172.21.15.112:0/1997230909 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2ec005670 con 0x7fb300104160 2024-05-08T05:59:06.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.895+0000 7fb2fe000700 1 -- 172.21.15.112:0/1997230909 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec004a50 con 0x7fb300104160 2024-05-08T05:59:06.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.895+0000 7fb308000700 1 -- 172.21.15.112:0/1997230909 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3001a89a0 con 0x7fb300104160 2024-05-08T05:59:06.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.900+0000 7fb2fe000700 1 -- 172.21.15.112:0/1997230909 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb2ec02a070 con 0x7fb300104160 2024-05-08T05:59:06.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.901+0000 7fb3128f1b80 1 Processor -- start 2024-05-08T05:59:06.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.901+0000 7fb2fe000700 1 --2- 172.21.15.112:0/1997230909 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb2e40b2900 0x7fb2e40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.901+0000 7fb3128f1b80 1 -- start start 2024-05-08T05:59:06.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.901+0000 7fb2fe000700 1 -- 172.21.15.112:0/1997230909 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fb2ec0e2c70 con 0x7fb300104160 2024-05-08T05:59:06.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.901+0000 7fb3128f1b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e6bc0 0x55956a6e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.901+0000 7fb3128f1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a6e12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.901+0000 7fb3128f1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e18f0 0x55956a6e1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.901+0000 7fb3128f1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55956a633a20 con 0x55956a6e18f0 2024-05-08T05:59:06.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.901+0000 7fb3128f1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55956a5d1760 con 0x55956a6e74e0 2024-05-08T05:59:06.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.901+0000 7fb3128f1b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55956a62f9e0 con 0x55956a6e6bc0 2024-05-08T05:59:06.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.902+0000 7fb305600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e6bc0 0x55956a6e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:06.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.902+0000 7fb305600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e6bc0 0x55956a6e6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60314/0 (socket says 172.21.15.112:60314) 2024-05-08T05:59:06.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.902+0000 7fb305600700 1 -- 172.21.15.112:0/95830161 learned_addr learned my addr 172.21.15.112:0/95830161 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:06.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.902+0000 7fb304c00700 1 --2- 172.21.15.112:0/95830161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a6e12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:06.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.902+0000 7fb306000700 1 --2- 172.21.15.112:0/1997230909 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb2e40b2900 0x7fb2e40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:06.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.902+0000 7fb304c00700 1 -- 172.21.15.112:0/95830161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e6bc0 msgr2=0x55956a6e6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:06.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.902+0000 7fb304c00700 1 --2- 172.21.15.112:0/95830161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e6bc0 0x55956a6e6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.902+0000 7fb304c00700 1 -- 172.21.15.112:0/95830161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e18f0 msgr2=0x55956a6e1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:06.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.902+0000 7fb304c00700 1 --2- 172.21.15.112:0/95830161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e18f0 0x55956a6e1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.902+0000 7fb304c00700 1 -- 172.21.15.112:0/95830161 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956a5808d0 con 0x55956a6e74e0 2024-05-08T05:59:06.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.903+0000 7fb304c00700 1 --2- 172.21.15.112:0/95830161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a6e12d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0aa220 tx=0x7fb2ec009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=598f48783581fecc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:06.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.903+0000 7fb2f6a00700 1 -- 172.21.15.112:0/95830161 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0aa290 con 0x55956a6e74e0 2024-05-08T05:59:06.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.903+0000 7fb2f6a00700 1 -- 172.21.15.112:0/95830161 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2ec0ae640 con 0x55956a6e74e0 2024-05-08T05:59:06.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.903+0000 7fb2f6a00700 1 -- 172.21.15.112:0/95830161 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec004e50 con 0x55956a6e74e0 2024-05-08T05:59:06.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.904+0000 7fb3128f1b80 1 -- 172.21.15.112:0/95830161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 msgr2=0x55956a6e12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:06.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.904+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/95830161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a6e12d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0aa220 tx=0x7fb2ec009c30 comp rx=0 tx=0).stop 2024-05-08T05:59:06.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.904+0000 7fb306000700 1 --2- 172.21.15.112:0/1997230909 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb2e40b2900 0x7fb2e40b4dc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55956a6e22b0 tx=0x7fb2f80095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:06.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.904+0000 7fb3128f1b80 1 -- 172.21.15.112:0/95830161 shutdown_connections 2024-05-08T05:59:06.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.904+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/95830161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e18f0 0x55956a6e1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.904+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/95830161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e6bc0 0x55956a6e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.904+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/95830161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a6e12d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.904+0000 7fb3128f1b80 1 -- 172.21.15.112:0/95830161 >> 172.21.15.112:0/95830161 conn(0x55956a6596d0 msgr2=0x55956a6dbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:06.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.904+0000 7fb3128f1b80 1 -- 172.21.15.112:0/95830161 shutdown_connections 2024-05-08T05:59:06.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.904+0000 7fb3128f1b80 1 -- 172.21.15.112:0/95830161 wait complete. 2024-05-08T05:59:06.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.905+0000 7fb3128f1b80 1 -- start start 2024-05-08T05:59:06.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.906+0000 7fb3128f1b80 1 Processor -- start 2024-05-08T05:59:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.906+0000 7fb3128f1b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e18f0 0x55956a6ed280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.906+0000 7fb3128f1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e6bc0 0x55956a6ea620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.906+0000 7fb3128f1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a6eab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.906+0000 7fb3128f1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55956a6edce0 con 0x55956a6e6bc0 2024-05-08T05:59:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.906+0000 7fb3128f1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55956a599b20 con 0x55956a6e74e0 2024-05-08T05:59:06.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.906+0000 7fb3128f1b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55956a6ede60 con 0x55956a6e18f0 2024-05-08T05:59:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.906+0000 7fb3128f1b80 10 client.0 _ll_register_callbacks cb 0x55956a5dac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:59:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.906+0000 7fb305600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e18f0 0x55956a6ed280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.906+0000 7fb305600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e18f0 0x55956a6ed280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60316/0 (socket says 172.21.15.112:60316) 2024-05-08T05:59:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.906+0000 7fb304c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e6bc0 0x55956a6ea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.906+0000 7fb305600700 1 -- 172.21.15.112:0/4166397398 learned_addr learned my addr 172.21.15.112:0/4166397398 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.907+0000 7fb305600700 1 -- 172.21.15.112:0/4166397398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 msgr2=0x55956a6eab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.907+0000 7fb305600700 1 --2- 172.21.15.112:0/4166397398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a6eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.907+0000 7fb305600700 1 -- 172.21.15.112:0/4166397398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e6bc0 msgr2=0x55956a6ea620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.907+0000 7fb305600700 1 --2- 172.21.15.112:0/4166397398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e6bc0 0x55956a6ea620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.907+0000 7fb305600700 1 -- 172.21.15.112:0/4166397398 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55956a5f9420 con 0x55956a6e18f0 2024-05-08T05:59:06.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.907+0000 7fb305600700 1 --2- 172.21.15.112:0/4166397398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e18f0 0x55956a6ed280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb2f00127c0 tx=0x7fb2f0012ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4168e4661533812d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:06.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.907+0000 7fb2eb400700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:59:06.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.907+0000 7fb2eb400700 1 -- 172.21.15.112:0/4166397398 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0004050 con 0x55956a6e18f0 2024-05-08T05:59:06.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.907+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4166397398 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2ec0ae640 con 0x55956a6e18f0 2024-05-08T05:59:06.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.907+0000 7fb2eb400700 1 -- 172.21.15.112:0/4166397398 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fb2f00041f0 con 0x55956a6e18f0 2024-05-08T05:59:06.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.908+0000 7fb2eb400700 1 -- 172.21.15.112:0/4166397398 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2f0022030 con 0x55956a6e18f0 2024-05-08T05:59:06.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.908+0000 7fb3128f1b80 10 client.25594 fetch_fsmap learned FSMap version 29 2024-05-08T05:59:06.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.908+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4166397398 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55956a700dc0 con 0x55956a6e18f0 2024-05-08T05:59:06.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.908+0000 7fb2eb400700 1 -- 172.21.15.112:0/4166397398 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2ec0ae640 con 0x55956a6e18f0 2024-05-08T05:59:06.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.908+0000 7fb2eb400700 1 -- 172.21.15.112:0/4166397398 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fb2f000f570 con 0x55956a6e18f0 2024-05-08T05:59:06.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.908+0000 7fb3128f1b80 10 client.25594 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:59:06.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.908+0000 7fb3128f1b80 10 client.25594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:59:06.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.908+0000 7fb3128f1b80 20 client.25594 populate_metadata read hostname 'smithi112' 2024-05-08T05:59:06.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.908+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/4166397398 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55956a703350 0x55956a7057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.908+0000 7fb3128f1b80 4 client.25594 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:59:06.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.908+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4166397398 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55956a6ae140 con 0x55956a703350 2024-05-08T05:59:06.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.909+0000 7fb304c00700 1 --2- 172.21.15.112:0/4166397398 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55956a703350 0x55956a7057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:06.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.909+0000 7fb304c00700 1 --2- 172.21.15.112:0/4166397398 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55956a703350 0x55956a7057a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2824145910d81933 server_cookie=1b2042f3d9969402 in_seq=0 out_seq=0 2024-05-08T05:59:06.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.909+0000 7fb2eb400700 10 client.25594 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:06.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.910+0000 7fb2eb400700 1 -- 172.21.15.112:0/4166397398 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55956a6ae140 con 0x55956a703350 2024-05-08T05:59:06.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.910+0000 7fb2eb400700 10 client.25594 handle_command_reply: tid=0 2024-05-08T05:59:06.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.911+0000 7fb3128f1b80 1 client.25594 shutdown 2024-05-08T05:59:06.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.911+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4166397398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e18f0 msgr2=0x55956a6ed280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:06.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.911+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/4166397398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e18f0 0x55956a6ed280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb2f00127c0 tx=0x7fb2f0012ad0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.911+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4166397398 shutdown_connections 2024-05-08T05:59:06.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.911+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/4166397398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a6eab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.911+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/4166397398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e18f0 0x55956a6ed280 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb2f00127c0 tx=0x7fb2f0012ad0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.911+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/4166397398 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55956a703350 0x55956a7057a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.911+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/4166397398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e6bc0 0x55956a6ea620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.911+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4166397398 >> 172.21.15.112:0/4166397398 conn(0x55956a6596d0 msgr2=0x55956a6db190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:06.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.911+0000 7fb2eb400700 0 client.25594 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:06.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.911+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4166397398 shutdown_connections 2024-05-08T05:59:06.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.912+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4166397398 wait complete. 2024-05-08T05:59:06.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.912+0000 7fb3128f1b80 20 client.25594 trim_cache size 0 max 16384 2024-05-08T05:59:06.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.930+0000 7fb3128f1b80 1 Processor -- start 2024-05-08T05:59:06.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.930+0000 7fb3128f1b80 1 -- start start 2024-05-08T05:59:06.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.931+0000 7fb3128f1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e6bc0 0x55956a6e12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.931+0000 7fb3128f1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a6ff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.931+0000 7fb3128f1b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e8020 0x55956a6db910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.931+0000 7fb3128f1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb3001ae2b0 con 0x55956a6e6bc0 2024-05-08T05:59:06.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.931+0000 7fb3128f1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb3001abab0 con 0x55956a6e74e0 2024-05-08T05:59:06.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.931+0000 7fb3128f1b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb30010e2d0 con 0x55956a6e8020 2024-05-08T05:59:06.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.931+0000 7fb306000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e8020 0x55956a6db910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:06.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.931+0000 7fb304c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a6ff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:06.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.931+0000 7fb305600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e6bc0 0x55956a6e12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:06.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.931+0000 7fb304c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a6ff4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:36710/0 (socket says 172.21.15.112:36710) 2024-05-08T05:59:06.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.931+0000 7fb304c00700 1 -- 172.21.15.112:0/2352371601 learned_addr learned my addr 172.21.15.112:0/2352371601 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:06.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.933+0000 7fb306000700 1 -- 172.21.15.112:0/2352371601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 msgr2=0x55956a6ff4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:06.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.933+0000 7fb306000700 1 --2- 172.21.15.112:0/2352371601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a6ff4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.933+0000 7fb306000700 1 -- 172.21.15.112:0/2352371601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e6bc0 msgr2=0x55956a6e12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:06.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.933+0000 7fb306000700 1 --2- 172.21.15.112:0/2352371601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e6bc0 0x55956a6e12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.933+0000 7fb306000700 1 -- 172.21.15.112:0/2352371601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956a6eb560 con 0x55956a6e8020 2024-05-08T05:59:06.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.933+0000 7fb304c00700 1 --2- 172.21.15.112:0/2352371601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a6ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:59:06.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.933+0000 7fb306000700 1 --2- 172.21.15.112:0/2352371601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e8020 0x55956a6db910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80070b0 tx=0x7fb2f80093d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8106d899cd0d6d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:06.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.933+0000 7fb2f6000700 1 -- 172.21.15.112:0/2352371601 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f80159d0 con 0x55956a6e8020 2024-05-08T05:59:06.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.933+0000 7fb2f6000700 1 -- 172.21.15.112:0/2352371601 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2f8024070 con 0x55956a6e8020 2024-05-08T05:59:06.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.933+0000 7fb2f6000700 1 -- 172.21.15.112:0/2352371601 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f80143a0 con 0x55956a6e8020 2024-05-08T05:59:06.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.934+0000 7fb3128f1b80 1 -- 172.21.15.112:0/2352371601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e8020 msgr2=0x55956a6db910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:06.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.934+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/2352371601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e8020 0x55956a6db910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80070b0 tx=0x7fb2f80093d0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.934+0000 7fb3128f1b80 1 -- 172.21.15.112:0/2352371601 shutdown_connections 2024-05-08T05:59:06.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.935+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/2352371601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e8020 0x55956a6db910 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.935+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/2352371601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e6bc0 0x55956a6e12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.935+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/2352371601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a6ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.935+0000 7fb3128f1b80 1 -- 172.21.15.112:0/2352371601 >> 172.21.15.112:0/2352371601 conn(0x55956a6596d0 msgr2=0x55956a6e7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:06.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.935+0000 7fb3128f1b80 1 -- 172.21.15.112:0/2352371601 shutdown_connections 2024-05-08T05:59:06.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.935+0000 7fb3128f1b80 1 -- 172.21.15.112:0/2352371601 wait complete. 2024-05-08T05:59:06.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.935+0000 7fb3128f1b80 1 -- start start 2024-05-08T05:59:06.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.936+0000 7fb3128f1b80 1 Processor -- start 2024-05-08T05:59:06.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.936+0000 7fb3128f1b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e6bc0 0x55956a6fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.936+0000 7fb3128f1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a70a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.936+0000 7fb3128f1b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e8020 0x55956a70a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.936+0000 7fb3128f1b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb3001a9f70 con 0x55956a6e8020 2024-05-08T05:59:06.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.936+0000 7fb3128f1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb3001ab3e0 con 0x55956a6e74e0 2024-05-08T05:59:06.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.936+0000 7fb3128f1b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb3001aa640 con 0x55956a6e6bc0 2024-05-08T05:59:06.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.936+0000 7fb3128f1b80 10 client.0 _ll_register_callbacks cb 0x55956a5dac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:59:06.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.936+0000 7fb304c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a70a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:06.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.936+0000 7fb305600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e6bc0 0x55956a6fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:06.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.936+0000 7fb304c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a70a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:36718/0 (socket says 172.21.15.112:36718) 2024-05-08T05:59:06.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.936+0000 7fb304c00700 1 -- 172.21.15.112:0/4117964476 learned_addr learned my addr 172.21.15.112:0/4117964476 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:06.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.936+0000 7fb305600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e6bc0 0x55956a6fbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60334/0 (socket says 172.21.15.112:60334) 2024-05-08T05:59:06.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.937+0000 7fb304c00700 1 -- 172.21.15.112:0/4117964476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e6bc0 msgr2=0x55956a6fbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:06.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.937+0000 7fb304c00700 1 --2- 172.21.15.112:0/4117964476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e6bc0 0x55956a6fbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.937+0000 7fb304c00700 1 -- 172.21.15.112:0/4117964476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e8020 msgr2=0x55956a70a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.937+0000 7fb304c00700 1 --2- 172.21.15.112:0/4117964476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e8020 0x55956a70a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.937+0000 7fb304c00700 1 -- 172.21.15.112:0/4117964476 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55956a70afa0 con 0x55956a6e74e0 2024-05-08T05:59:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.937+0000 7fb304c00700 1 --2- 172.21.15.112:0/4117964476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a70a420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0aa220 tx=0x7fb2ec0a6200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=706a950d8916b9f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.937+0000 7fb2eb400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:59:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.937+0000 7fb2eb400700 1 -- 172.21.15.112:0/4117964476 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec004bf0 con 0x55956a6e74e0 2024-05-08T05:59:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.937+0000 7fb2eb400700 1 -- 172.21.15.112:0/4117964476 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fb2ec004d90 con 0x55956a6e74e0 2024-05-08T05:59:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.937+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4117964476 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2f8024070 con 0x55956a6e74e0 2024-05-08T05:59:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.937+0000 7fb2eb400700 1 -- 172.21.15.112:0/4117964476 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2ec0bf030 con 0x55956a6e74e0 2024-05-08T05:59:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.937+0000 7fb3128f1b80 10 client.25562 fetch_fsmap learned FSMap version 29 2024-05-08T05:59:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.937+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4117964476 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55956a76da50 con 0x55956a6e74e0 2024-05-08T05:59:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.938+0000 7fb2eb400700 1 -- 172.21.15.112:0/4117964476 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2f8024070 con 0x55956a6e74e0 2024-05-08T05:59:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.938+0000 7fb2eb400700 1 -- 172.21.15.112:0/4117964476 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fb2ec0b2bd0 con 0x55956a6e74e0 2024-05-08T05:59:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.938+0000 7fb3128f1b80 10 client.25562 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:59:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.938+0000 7fb3128f1b80 10 client.25562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:59:06.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.938+0000 7fb3128f1b80 20 client.25562 populate_metadata read hostname 'smithi112' 2024-05-08T05:59:06.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.938+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/4117964476 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55956a76dfc0 0x55956a7703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.938+0000 7fb3128f1b80 4 client.25562 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:59:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.938+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4117964476 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55956a641400 con 0x55956a76dfc0 2024-05-08T05:59:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.939+0000 7fb305600700 1 --2- 172.21.15.112:0/4117964476 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55956a76dfc0 0x55956a7703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.939+0000 7fb305600700 1 --2- 172.21.15.112:0/4117964476 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55956a76dfc0 0x55956a7703e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53573e6bbc6836f server_cookie=8c9ba634cb0d4472 in_seq=0 out_seq=0 2024-05-08T05:59:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.939+0000 7fb2eb400700 10 client.25562 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.940+0000 7fb2eb400700 1 -- 172.21.15.112:0/4117964476 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55956a641400 con 0x55956a76dfc0 2024-05-08T05:59:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.940+0000 7fb2eb400700 10 client.25562 handle_command_reply: tid=0 2024-05-08T05:59:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.940+0000 7fb3128f1b80 1 client.25562 shutdown 2024-05-08T05:59:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.941+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4117964476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 msgr2=0x55956a70a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:06.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.941+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/4117964476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a70a420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0aa220 tx=0x7fb2ec0a6200 comp rx=0 tx=0).stop 2024-05-08T05:59:06.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.943+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4117964476 shutdown_connections 2024-05-08T05:59:06.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.943+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/4117964476 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55956a76dfc0 0x55956a7703e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.943+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/4117964476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55956a6e8020 0x55956a70a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.943+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/4117964476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55956a6e6bc0 0x55956a6fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.943+0000 7fb3128f1b80 1 --2- 172.21.15.112:0/4117964476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55956a6e74e0 0x55956a70a420 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.943+0000 7fb2eb400700 0 client.25562 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:06.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.943+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4117964476 >> 172.21.15.112:0/4117964476 conn(0x55956a6596d0 msgr2=0x55956a6e5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:06.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.943+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4117964476 shutdown_connections 2024-05-08T05:59:06.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.943+0000 7fb3128f1b80 1 -- 172.21.15.112:0/4117964476 wait complete. 2024-05-08T05:59:06.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.943+0000 7fb3128f1b80 20 client.25562 trim_cache size 0 max 16384 2024-05-08T05:59:06.943 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:59:06.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.944+0000 7fb308000700 1 -- 172.21.15.112:0/1997230909 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb2e40b2900 msgr2=0x7fb2e40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:06.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.944+0000 7fb308000700 1 --2- 172.21.15.112:0/1997230909 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb2e40b2900 0x7fb2e40b4dc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55956a6e22b0 tx=0x7fb2f80095a0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.944+0000 7fb308000700 1 -- 172.21.15.112:0/1997230909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb300104160 msgr2=0x7fb30019c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:06.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.944+0000 7fb308000700 1 --2- 172.21.15.112:0/1997230909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb300104160 0x7fb30019c360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00c010 tx=0x7fb2ec00fa00 comp rx=0 tx=0).stop 2024-05-08T05:59:06.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.944+0000 7fb308000700 1 -- 172.21.15.112:0/1997230909 shutdown_connections 2024-05-08T05:59:06.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.944+0000 7fb308000700 1 --2- 172.21.15.112:0/1997230909 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb2e40b2900 0x7fb2e40b4dc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.944+0000 7fb308000700 1 --2- 172.21.15.112:0/1997230909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb300108880 0x7fb3001a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.944+0000 7fb308000700 1 --2- 172.21.15.112:0/1997230909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3001037b0 0x7fb30019be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.944+0000 7fb308000700 1 --2- 172.21.15.112:0/1997230909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb300104160 0x7fb30019c360 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:06.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.944+0000 7fb308000700 1 -- 172.21.15.112:0/1997230909 >> 172.21.15.112:0/1997230909 conn(0x7fb30007a530 msgr2=0x7fb300101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:06.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.944+0000 7fb308000700 1 -- 172.21.15.112:0/1997230909 shutdown_connections 2024-05-08T05:59:06.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:06.945+0000 7fb308000700 1 -- 172.21.15.112:0/1997230909 wait complete. 2024-05-08T05:59:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:07 smithi150 ceph-mon[34315]: pgmap v555: 97 pgs: 97 active+clean; 75 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 253 op/s 2024-05-08T05:59:07.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:07 smithi162 ceph-mon[34800]: pgmap v555: 97 pgs: 97 active+clean; 75 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 253 op/s 2024-05-08T05:59:07.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:07 smithi112 ceph-mon[30003]: pgmap v555: 97 pgs: 97 active+clean; 75 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 253 op/s 2024-05-08T05:59:08.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:08 smithi150 ceph-mon[34315]: pgmap v556: 97 pgs: 97 active+clean; 75 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 258 op/s 2024-05-08T05:59:08.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:08 smithi112 ceph-mon[30003]: pgmap v556: 97 pgs: 97 active+clean; 75 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 258 op/s 2024-05-08T05:59:08.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:08 smithi162 ceph-mon[34800]: pgmap v556: 97 pgs: 97 active+clean; 75 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 258 op/s 2024-05-08T05:59:08.993 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T05:59:08.994 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T05:59:09.505 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:59:10.931 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:10 smithi112 ceph-mon[30003]: pgmap v557: 97 pgs: 97 active+clean; 76 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 293 op/s 2024-05-08T05:59:11.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:10 smithi162 ceph-mon[34800]: pgmap v557: 97 pgs: 97 active+clean; 76 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 293 op/s 2024-05-08T05:59:11.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:10 smithi150 ceph-mon[34315]: pgmap v557: 97 pgs: 97 active+clean; 76 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 293 op/s 2024-05-08T05:59:12.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.032+0000 7fd2f4c00700 1 -- 172.21.15.112:0/914430686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f006ba40 msgr2=0x7fd2f006be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.032+0000 7fd2f4c00700 1 --2- 172.21.15.112:0/914430686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f006ba40 0x7fd2f006be20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00098f0 tx=0x7fd2e0009c00 comp rx=0 tx=0).stop 2024-05-08T05:59:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.032+0000 7fd2f4c00700 1 -- 172.21.15.112:0/914430686 shutdown_connections 2024-05-08T05:59:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.032+0000 7fd2f4c00700 1 --2- 172.21.15.112:0/914430686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2f010ff40 0x7fd2f0112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.032+0000 7fd2f4c00700 1 --2- 172.21.15.112:0/914430686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f006ba40 0x7fd2f006be20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.032+0000 7fd2f4c00700 1 --2- 172.21.15.112:0/914430686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2f006c360 0x7fd2f010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.032+0000 7fd2f4c00700 1 -- 172.21.15.112:0/914430686 >> 172.21.15.112:0/914430686 conn(0x7fd2f0079310 msgr2=0x7fd2f0079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.032+0000 7fd2f4c00700 1 -- 172.21.15.112:0/914430686 shutdown_connections 2024-05-08T05:59:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.032+0000 7fd2f4c00700 1 -- 172.21.15.112:0/914430686 wait complete. 2024-05-08T05:59:12.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.033+0000 7fd2f4c00700 1 Processor -- start 2024-05-08T05:59:12.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.033+0000 7fd2f4c00700 1 -- start start 2024-05-08T05:59:12.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.034+0000 7fd2f4c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2f006ba40 0x7fd2f019be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.034+0000 7fd2f4c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2f006c360 0x7fd2f019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.034+0000 7fd2f4c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f010ff40 0x7fd2f01a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.034+0000 7fd2f4c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01158a0 con 0x7fd2f006ba40 2024-05-08T05:59:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.034+0000 7fd2f4c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f0115720 con 0x7fd2f006c360 2024-05-08T05:59:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.034+0000 7fd2f4c00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f0115a20 con 0x7fd2f010ff40 2024-05-08T05:59:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.034+0000 7fd2ee800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f010ff40 0x7fd2f01a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.034+0000 7fd2ed400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2f006c360 0x7fd2f019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.034+0000 7fd2ee800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f010ff40 0x7fd2f01a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60350/0 (socket says 172.21.15.112:60350) 2024-05-08T05:59:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.034+0000 7fd2ee800700 1 -- 172.21.15.112:0/845305839 learned_addr learned my addr 172.21.15.112:0/845305839 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.034+0000 7fd2ede00700 1 --2- 172.21.15.112:0/845305839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2f006ba40 0x7fd2f019be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.035+0000 7fd2ee800700 1 -- 172.21.15.112:0/845305839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2f006c360 msgr2=0x7fd2f019c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.035+0000 7fd2ee800700 1 --2- 172.21.15.112:0/845305839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2f006c360 0x7fd2f019c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.035+0000 7fd2ee800700 1 -- 172.21.15.112:0/845305839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2f006ba40 msgr2=0x7fd2f019be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.035+0000 7fd2ee800700 1 --2- 172.21.15.112:0/845305839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2f006ba40 0x7fd2f019be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.035+0000 7fd2ee800700 1 -- 172.21.15.112:0/845305839 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f01a0e40 con 0x7fd2f010ff40 2024-05-08T05:59:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.035+0000 7fd2ede00700 1 --2- 172.21.15.112:0/845305839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2f006ba40 0x7fd2f019be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:59:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.035+0000 7fd2ee800700 1 --2- 172.21.15.112:0/845305839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f010ff40 0x7fd2f01a0700 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800c3d0 tx=0x7fd2e800c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.035+0000 7fd2e6a00700 1 -- 172.21.15.112:0/845305839 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80092d0 con 0x7fd2f010ff40 2024-05-08T05:59:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.035+0000 7fd2e6a00700 1 -- 172.21.15.112:0/845305839 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2e8009470 con 0x7fd2f010ff40 2024-05-08T05:59:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.035+0000 7fd2f4c00700 1 -- 172.21.15.112:0/845305839 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f01a1130 con 0x7fd2f010ff40 2024-05-08T05:59:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.036+0000 7fd2e6a00700 1 -- 172.21.15.112:0/845305839 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80048d0 con 0x7fd2f010ff40 2024-05-08T05:59:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.036+0000 7fd2f4c00700 1 -- 172.21.15.112:0/845305839 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f01a89b0 con 0x7fd2f010ff40 2024-05-08T05:59:12.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.040+0000 7fd2e6a00700 1 -- 172.21.15.112:0/845305839 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd2e8012070 con 0x7fd2f010ff40 2024-05-08T05:59:12.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.042+0000 7fd2ff4e9b80 1 Processor -- start 2024-05-08T05:59:12.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.042+0000 7fd2ff4e9b80 1 -- start start 2024-05-08T05:59:12.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.043+0000 7fd2ff4e9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56486666dc40 0x56486666e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.043+0000 7fd2e6a00700 1 --2- 172.21.15.112:0/845305839 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd2d40b2900 0x7fd2d40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.043+0000 7fd2ff4e9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666e560 0x564866668350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.043+0000 7fd2ff4e9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564866668970 0x564866668df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.043+0000 7fd2ff4e9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5648665baaa0 con 0x56486666dc40 2024-05-08T05:59:12.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.043+0000 7fd2ff4e9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5648665587e0 con 0x56486666e560 2024-05-08T05:59:12.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.043+0000 7fd2ff4e9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5648665b6a60 con 0x564866668970 2024-05-08T05:59:12.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.043+0000 7fd2e6a00700 1 -- 172.21.15.112:0/845305839 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd2e80e1bf0 con 0x7fd2f010ff40 2024-05-08T05:59:12.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.043+0000 7fd2ed400700 1 --2- 172.21.15.112:0/845305839 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd2d40b2900 0x7fd2d40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.043+0000 7fd2ed400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666e560 0x564866668350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.043+0000 7fd2ee800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564866668970 0x564866668df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.043+0000 7fd2ed400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666e560 0x564866668350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:36760/0 (socket says 172.21.15.112:36760) 2024-05-08T05:59:12.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.043+0000 7fd2ed400700 1 -- 172.21.15.112:0/2743711534 learned_addr learned my addr 172.21.15.112:0/2743711534 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:12.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.044+0000 7fd2ee800700 1 -- 172.21.15.112:0/2743711534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666e560 msgr2=0x564866668350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:12.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.044+0000 7fd2ee800700 1 --2- 172.21.15.112:0/2743711534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666e560 0x564866668350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.044+0000 7fd2ee800700 1 -- 172.21.15.112:0/2743711534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56486666dc40 msgr2=0x56486666e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:12.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.044+0000 7fd2ee800700 1 --2- 172.21.15.112:0/2743711534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56486666dc40 0x56486666e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.044+0000 7fd2ee800700 1 -- 172.21.15.112:0/2743711534 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564866507950 con 0x564866668970 2024-05-08T05:59:12.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.044+0000 7fd2ee800700 1 --2- 172.21.15.112:0/2743711534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564866668970 0x564866668df0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80df660 tx=0x7fd2e80a8bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0950c1f304d1676 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:12.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.044+0000 7fd2ed400700 1 --2- 172.21.15.112:0/845305839 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd2d40b2900 0x7fd2d40b4dc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc012980 tx=0x7fd2dc007b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:12.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.044+0000 7fd2db400700 1 -- 172.21.15.112:0/2743711534 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80a9880 con 0x564866668970 2024-05-08T05:59:12.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.044+0000 7fd2db400700 1 -- 172.21.15.112:0/2743711534 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2e80a9180 con 0x564866668970 2024-05-08T05:59:12.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.044+0000 7fd2db400700 1 -- 172.21.15.112:0/2743711534 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8092220 con 0x564866668970 2024-05-08T05:59:12.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.045+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/2743711534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564866668970 msgr2=0x564866668df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:12.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.045+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/2743711534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564866668970 0x564866668df0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80df660 tx=0x7fd2e80a8bd0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.045+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/2743711534 shutdown_connections 2024-05-08T05:59:12.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.045+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/2743711534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564866668970 0x564866668df0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.045+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/2743711534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56486666dc40 0x56486666e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.045+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/2743711534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666e560 0x564866668350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.046+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/2743711534 >> 172.21.15.112:0/2743711534 conn(0x5648665e0750 msgr2=0x564866662b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:12.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.046+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/2743711534 shutdown_connections 2024-05-08T05:59:12.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.046+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/2743711534 wait complete. 2024-05-08T05:59:12.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.046+0000 7fd2ff4e9b80 1 -- start start 2024-05-08T05:59:12.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.047+0000 7fd2ff4e9b80 1 Processor -- start 2024-05-08T05:59:12.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.047+0000 7fd2ff4e9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564866668970 0x564866674280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.047+0000 7fd2ff4e9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666dc40 0x564866671670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.047+0000 7fd2ff4e9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56486666e560 0x564866671bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.047+0000 7fd2ff4e9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564866674fa0 con 0x56486666e560 2024-05-08T05:59:12.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.047+0000 7fd2ff4e9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564866520ba0 con 0x56486666dc40 2024-05-08T05:59:12.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.047+0000 7fd2ff4e9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x564866675120 con 0x564866668970 2024-05-08T05:59:12.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.047+0000 7fd2ff4e9b80 10 client.0 _ll_register_callbacks cb 0x564866561cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:59:12.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.048+0000 7fd2ede00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564866668970 0x564866674280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.048+0000 7fd2ed400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666dc40 0x564866671670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.048+0000 7fd2ede00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564866668970 0x564866674280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60380/0 (socket says 172.21.15.112:60380) 2024-05-08T05:59:12.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.048+0000 7fd2ede00700 1 -- 172.21.15.112:0/1269238004 learned_addr learned my addr 172.21.15.112:0/1269238004 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:12.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.048+0000 7fd2ede00700 1 -- 172.21.15.112:0/1269238004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666dc40 msgr2=0x564866671670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:12.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.048+0000 7fd2ede00700 1 --2- 172.21.15.112:0/1269238004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666dc40 0x564866671670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.048+0000 7fd2ede00700 1 -- 172.21.15.112:0/1269238004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56486666e560 msgr2=0x564866671bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:59:12.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.048+0000 7fd2ede00700 1 --2- 172.21.15.112:0/1269238004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56486666e560 0x564866671bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.048+0000 7fd2ede00700 1 -- 172.21.15.112:0/1269238004 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648665804a0 con 0x564866668970 2024-05-08T05:59:12.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.048+0000 7fd2ede00700 1 --2- 172.21.15.112:0/1269238004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564866668970 0x564866674280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0005e90 tx=0x7fd2e000f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b796e8ce4ac3c0b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:12.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.048+0000 7fd2cfe00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:59:12.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.049+0000 7fd2cfe00700 1 -- 172.21.15.112:0/1269238004 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e00094d0 con 0x564866668970 2024-05-08T05:59:12.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.049+0000 7fd2cfe00700 1 -- 172.21.15.112:0/1269238004 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd2e0005040 con 0x564866668970 2024-05-08T05:59:12.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.049+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/1269238004 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2e80a9180 con 0x564866668970 2024-05-08T05:59:12.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.049+0000 7fd2cfe00700 1 -- 172.21.15.112:0/1269238004 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd2e0005240 con 0x564866668970 2024-05-08T05:59:12.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.049+0000 7fd2ff4e9b80 10 client.25612 fetch_fsmap learned FSMap version 29 2024-05-08T05:59:12.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.049+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/1269238004 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56486667f8a0 con 0x564866668970 2024-05-08T05:59:12.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.049+0000 7fd2cfe00700 1 -- 172.21.15.112:0/1269238004 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2e80a9180 con 0x564866668970 2024-05-08T05:59:12.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.049+0000 7fd2cfe00700 1 -- 172.21.15.112:0/1269238004 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fd2e000b560 con 0x564866668970 2024-05-08T05:59:12.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.050+0000 7fd2ff4e9b80 10 client.25612 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:59:12.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.050+0000 7fd2ff4e9b80 10 client.25612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:59:12.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.050+0000 7fd2ff4e9b80 20 client.25612 populate_metadata read hostname 'smithi112' 2024-05-08T05:59:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.050+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/1269238004 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x564866681f20 0x5648666843a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.050+0000 7fd2ff4e9b80 4 client.25612 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:59:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.050+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/1269238004 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648666351c0 con 0x564866681f20 2024-05-08T05:59:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.050+0000 7fd2ed400700 1 --2- 172.21.15.112:0/1269238004 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x564866681f20 0x5648666843a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.051+0000 7fd2ed400700 1 --2- 172.21.15.112:0/1269238004 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x564866681f20 0x5648666843a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80e5398c0eb7963a server_cookie=ffe40682274281d7 in_seq=0 out_seq=0 2024-05-08T05:59:12.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.051+0000 7fd2cfe00700 10 client.25612 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:12.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.052+0000 7fd2cfe00700 1 -- 172.21.15.112:0/1269238004 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5648666351c0 con 0x564866681f20 2024-05-08T05:59:12.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.053+0000 7fd2cfe00700 10 client.25612 handle_command_reply: tid=0 2024-05-08T05:59:12.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.053+0000 7fd2ff4e9b80 1 client.25612 shutdown 2024-05-08T05:59:12.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.053+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/1269238004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564866668970 msgr2=0x564866674280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:12.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.053+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/1269238004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564866668970 0x564866674280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0005e90 tx=0x7fd2e000f740 comp rx=0 tx=0).stop 2024-05-08T05:59:12.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.053+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/1269238004 shutdown_connections 2024-05-08T05:59:12.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.053+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/1269238004 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x564866681f20 0x5648666843a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.053+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/1269238004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56486666e560 0x564866671bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.053+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/1269238004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564866668970 0x564866674280 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.053+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/1269238004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666dc40 0x564866671670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.053+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/1269238004 >> 172.21.15.112:0/1269238004 conn(0x5648665e0750 msgr2=0x564866662210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:12.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.053+0000 7fd2cfe00700 0 client.25612 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:12.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.053+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/1269238004 shutdown_connections 2024-05-08T05:59:12.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.054+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/1269238004 wait complete. 2024-05-08T05:59:12.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.054+0000 7fd2ff4e9b80 20 client.25612 trim_cache size 0 max 16384 2024-05-08T05:59:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.078+0000 7fd2ff4e9b80 1 Processor -- start 2024-05-08T05:59:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.079+0000 7fd2ff4e9b80 1 -- start start 2024-05-08T05:59:12.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.079+0000 7fd2ff4e9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564866668970 0x564866668350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.079+0000 7fd2ff4e9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56486666dc40 0x56486667e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.079+0000 7fd2ff4e9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666e560 0x56486666d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.079+0000 7fd2ff4e9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01b26b0 con 0x564866668970 2024-05-08T05:59:12.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.079+0000 7fd2ff4e9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01afeb0 con 0x56486666e560 2024-05-08T05:59:12.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.079+0000 7fd2ff4e9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01add00 con 0x56486666dc40 2024-05-08T05:59:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.080+0000 7fd2ed400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56486666dc40 0x56486667e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.080+0000 7fd2ee800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666e560 0x56486666d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.080+0000 7fd2ede00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564866668970 0x564866668350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.080+0000 7fd2ed400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56486666dc40 0x56486667e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60390/0 (socket says 172.21.15.112:60390) 2024-05-08T05:59:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.080+0000 7fd2ed400700 1 -- 172.21.15.112:0/2414621552 learned_addr learned my addr 172.21.15.112:0/2414621552 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.080+0000 7fd2ee800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666e560 0x56486666d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:36770/0 (socket says 172.21.15.112:36770) 2024-05-08T05:59:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.080+0000 7fd2ed400700 1 -- 172.21.15.112:0/2414621552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666e560 msgr2=0x56486666d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.080+0000 7fd2ed400700 1 --2- 172.21.15.112:0/2414621552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666e560 0x56486666d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.080+0000 7fd2ed400700 1 -- 172.21.15.112:0/2414621552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564866668970 msgr2=0x564866668350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.080+0000 7fd2ed400700 1 --2- 172.21.15.112:0/2414621552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564866668970 0x564866668350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.080+0000 7fd2ed400700 1 -- 172.21.15.112:0/2414621552 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56486666c130 con 0x56486666dc40 2024-05-08T05:59:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.081+0000 7fd2ed400700 1 --2- 172.21.15.112:0/2414621552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56486666dc40 0x56486667e030 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc007fd0 tx=0x7fd2dc0073d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc534eaf115c6231 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.081+0000 7fd2daa00700 1 -- 172.21.15.112:0/2414621552 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc02e070 con 0x56486666dc40 2024-05-08T05:59:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.081+0000 7fd2daa00700 1 -- 172.21.15.112:0/2414621552 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2dc012e50 con 0x56486666dc40 2024-05-08T05:59:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.081+0000 7fd2daa00700 1 -- 172.21.15.112:0/2414621552 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc003850 con 0x56486666dc40 2024-05-08T05:59:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.082+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/2414621552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56486666dc40 msgr2=0x56486667e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.082+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/2414621552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56486666dc40 0x56486667e030 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc007fd0 tx=0x7fd2dc0073d0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.082+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/2414621552 shutdown_connections 2024-05-08T05:59:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.082+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/2414621552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56486666e560 0x56486666d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.082+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/2414621552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564866668970 0x564866668350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.082+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/2414621552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56486666dc40 0x56486667e030 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.082+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/2414621552 >> 172.21.15.112:0/2414621552 conn(0x5648665e0750 msgr2=0x56486666f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:12.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.083+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/2414621552 shutdown_connections 2024-05-08T05:59:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.083+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/2414621552 wait complete. 2024-05-08T05:59:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.083+0000 7fd2ff4e9b80 1 -- start start 2024-05-08T05:59:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.083+0000 7fd2ff4e9b80 1 Processor -- start 2024-05-08T05:59:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.084+0000 7fd2ff4e9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564866668970 0x564866689830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.084+0000 7fd2ff4e9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56486666dc40 0x564866689d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.084+0000 7fd2ff4e9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56486666e560 0x5648666eb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.084+0000 7fd2ff4e9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01ae370 con 0x56486666e560 2024-05-08T05:59:12.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.084+0000 7fd2ff4e9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01af7e0 con 0x564866668970 2024-05-08T05:59:12.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.084+0000 7fd2ed400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56486666dc40 0x564866689d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.084+0000 7fd2ed400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56486666dc40 0x564866689d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60406/0 (socket says 172.21.15.112:60406) 2024-05-08T05:59:12.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.084+0000 7fd2ed400700 1 -- 172.21.15.112:0/121596539 learned_addr learned my addr 172.21.15.112:0/121596539 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:12.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.085+0000 7fd2ede00700 1 --2- 172.21.15.112:0/121596539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564866668970 0x564866689830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.084+0000 7fd2ff4e9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01aea40 con 0x56486666dc40 2024-05-08T05:59:12.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.085+0000 7fd2ff4e9b80 10 client.0 _ll_register_callbacks cb 0x564866561cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:59:12.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.085+0000 7fd2ee800700 1 --2- 172.21.15.112:0/121596539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56486666e560 0x5648666eb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.085+0000 7fd2ede00700 1 -- 172.21.15.112:0/121596539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56486666dc40 msgr2=0x564866689d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:12.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.085+0000 7fd2ede00700 1 --2- 172.21.15.112:0/121596539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56486666dc40 0x564866689d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.085+0000 7fd2ede00700 1 -- 172.21.15.112:0/121596539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56486666e560 msgr2=0x5648666eb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:12.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.085+0000 7fd2ede00700 1 --2- 172.21.15.112:0/121596539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56486666e560 0x5648666eb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.085+0000 7fd2ede00700 1 -- 172.21.15.112:0/121596539 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56486668a660 con 0x564866668970 2024-05-08T05:59:12.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.085+0000 7fd2ee800700 1 --2- 172.21.15.112:0/121596539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56486666e560 0x5648666eb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:59:12.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.085+0000 7fd2ede00700 1 --2- 172.21.15.112:0/121596539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564866668970 0x564866689830 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009fd0 tx=0x7fd2e000b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add8ce15b223523a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:12.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.085+0000 7fd2cfe00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:59:12.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.086+0000 7fd2cfe00700 1 -- 172.21.15.112:0/121596539 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e00055f0 con 0x564866668970 2024-05-08T05:59:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.086+0000 7fd2cfe00700 1 -- 172.21.15.112:0/121596539 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd2e00046d0 con 0x564866668970 2024-05-08T05:59:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.086+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/121596539 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2dc012e50 con 0x564866668970 2024-05-08T05:59:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.086+0000 7fd2cfe00700 1 -- 172.21.15.112:0/121596539 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd2e003c050 con 0x564866668970 2024-05-08T05:59:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.086+0000 7fd2ff4e9b80 10 client.25580 fetch_fsmap learned FSMap version 29 2024-05-08T05:59:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.086+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/121596539 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648666ece90 con 0x564866668970 2024-05-08T05:59:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.087+0000 7fd2cfe00700 1 -- 172.21.15.112:0/121596539 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2dc012e50 con 0x564866668970 2024-05-08T05:59:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.087+0000 7fd2cfe00700 1 -- 172.21.15.112:0/121596539 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fd2e002c430 con 0x564866668970 2024-05-08T05:59:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.087+0000 7fd2ff4e9b80 10 client.25580 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:59:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.087+0000 7fd2ff4e9b80 10 client.25580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:59:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.087+0000 7fd2ff4e9b80 20 client.25580 populate_metadata read hostname 'smithi112' 2024-05-08T05:59:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.087+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/121596539 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5648666ed5a0 0x5648666ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.087+0000 7fd2ff4e9b80 4 client.25580 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T05:59:12.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.087+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/121596539 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5648665c8480 con 0x5648666ed5a0 2024-05-08T05:59:12.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.087+0000 7fd2ed400700 1 --2- 172.21.15.112:0/121596539 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5648666ed5a0 0x5648666ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:12.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.089+0000 7fd2ed400700 1 --2- 172.21.15.112:0/121596539 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5648666ed5a0 0x5648666ef9c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad995bcaa84fe74b server_cookie=c93f1fb6797ced2c in_seq=0 out_seq=0 2024-05-08T05:59:12.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.089+0000 7fd2cfe00700 10 client.25580 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.091+0000 7fd2cfe00700 1 -- 172.21.15.112:0/121596539 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5648665c8480 con 0x5648666ed5a0 2024-05-08T05:59:12.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.091+0000 7fd2cfe00700 10 client.25580 handle_command_reply: tid=0 2024-05-08T05:59:12.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.091+0000 7fd2ff4e9b80 1 client.25580 shutdown 2024-05-08T05:59:12.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.091+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/121596539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564866668970 msgr2=0x564866689830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:12.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.091+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/121596539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564866668970 0x564866689830 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009fd0 tx=0x7fd2e000b7f0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.092+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/121596539 shutdown_connections 2024-05-08T05:59:12.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.092+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/121596539 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5648666ed5a0 0x5648666ef9c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.092+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/121596539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56486666e560 0x5648666eb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.092+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/121596539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564866668970 0x564866689830 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.092+0000 7fd2ff4e9b80 1 --2- 172.21.15.112:0/121596539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56486666dc40 0x564866689d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.092+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/121596539 >> 172.21.15.112:0/121596539 conn(0x5648665e0750 msgr2=0x56486666d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:12.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.092+0000 7fd2cfe00700 0 client.25580 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:12.093 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:59:12.093 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-08T05:59:12.093 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "acdc742a-ba84-4862-96cf-9e9c423b782c", 2024-05-08T05:59:12.093 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-08T05:59:12.093 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:59:12.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.092+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/121596539 shutdown_connections 2024-05-08T05:59:12.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.092+0000 7fd2ff4e9b80 1 -- 172.21.15.112:0/121596539 wait complete. 2024-05-08T05:59:12.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.092+0000 7fd2ff4e9b80 20 client.25580 trim_cache size 0 max 16384 2024-05-08T05:59:12.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.093+0000 7fd2f4c00700 1 -- 172.21.15.112:0/845305839 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd2d40b2900 msgr2=0x7fd2d40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:12.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.093+0000 7fd2f4c00700 1 --2- 172.21.15.112:0/845305839 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd2d40b2900 0x7fd2d40b4dc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc012980 tx=0x7fd2dc007b80 comp rx=0 tx=0).stop 2024-05-08T05:59:12.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.093+0000 7fd2f4c00700 1 -- 172.21.15.112:0/845305839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f010ff40 msgr2=0x7fd2f01a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:12.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.093+0000 7fd2f4c00700 1 --2- 172.21.15.112:0/845305839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f010ff40 0x7fd2f01a0700 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800c3d0 tx=0x7fd2e800c6e0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.094+0000 7fd2f4c00700 1 -- 172.21.15.112:0/845305839 shutdown_connections 2024-05-08T05:59:12.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.094+0000 7fd2f4c00700 1 --2- 172.21.15.112:0/845305839 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd2d40b2900 0x7fd2d40b4dc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.094+0000 7fd2f4c00700 1 --2- 172.21.15.112:0/845305839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f010ff40 0x7fd2f01a0700 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.094+0000 7fd2f4c00700 1 --2- 172.21.15.112:0/845305839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2f006ba40 0x7fd2f019be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.094+0000 7fd2f4c00700 1 --2- 172.21.15.112:0/845305839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2f006c360 0x7fd2f019c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:12.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.094+0000 7fd2f4c00700 1 -- 172.21.15.112:0/845305839 >> 172.21.15.112:0/845305839 conn(0x7fd2f0079310 msgr2=0x7fd2f0111490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:12.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.094+0000 7fd2f4c00700 1 -- 172.21.15.112:0/845305839 shutdown_connections 2024-05-08T05:59:12.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:12.094+0000 7fd2f4c00700 1 -- 172.21.15.112:0/845305839 wait complete. 2024-05-08T05:59:13.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:12 smithi112 ceph-mon[30003]: pgmap v558: 97 pgs: 97 active+clean; 77 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-05-08T05:59:13.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:12 smithi112 ceph-mon[30003]: scrub queued for path: / 2024-05-08T05:59:13.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:12 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:59:13.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:12 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T05:59:13.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:12 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:59:13.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:12 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T05:59:13.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:12 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:59:13.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:12 smithi162 ceph-mon[34800]: pgmap v558: 97 pgs: 97 active+clean; 77 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-05-08T05:59:13.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:12 smithi162 ceph-mon[34800]: scrub queued for path: / 2024-05-08T05:59:13.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:12 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:59:13.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:12 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T05:59:13.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:12 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:59:13.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:12 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T05:59:13.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:12 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:59:13.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:12 smithi150 ceph-mon[34315]: pgmap v558: 97 pgs: 97 active+clean; 77 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-05-08T05:59:13.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:12 smithi150 ceph-mon[34315]: scrub queued for path: / 2024-05-08T05:59:13.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:12 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:59:13.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:12 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T05:59:13.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:12 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:59:13.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:12 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T05:59:13.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:12 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:59:13.188 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T05:59:13.657 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:59:14.024 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:13 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T05:59:14.024 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:13 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T05:59:14.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:13 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T05:59:14.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:13 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T05:59:14.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:13 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T05:59:14.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:13 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T05:59:14.736 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:14 smithi112 ceph-mon[30003]: pgmap v559: 97 pgs: 97 active+clean; 76 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 299 op/s 2024-05-08T05:59:15.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:14 smithi162 ceph-mon[34800]: pgmap v559: 97 pgs: 97 active+clean; 76 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 299 op/s 2024-05-08T05:59:15.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:14 smithi150 ceph-mon[34315]: pgmap v559: 97 pgs: 97 active+clean; 76 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 299 op/s 2024-05-08T05:59:16.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.133+0000 7f3f6d800700 1 -- 172.21.15.112:0/473169029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f68108850 msgr2=0x7f3f681103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:16.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.133+0000 7f3f6d800700 1 --2- 172.21.15.112:0/473169029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f68108850 0x7f3f681103d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000b0a0 tx=0x7f3f6000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.133+0000 7f3f6d800700 1 -- 172.21.15.112:0/473169029 shutdown_connections 2024-05-08T05:59:16.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.133+0000 7f3f6d800700 1 --2- 172.21.15.112:0/473169029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f68108850 0x7f3f681103d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.133+0000 7f3f6d800700 1 --2- 172.21.15.112:0/473169029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f68103750 0x7f3f68103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.133+0000 7f3f6d800700 1 --2- 172.21.15.112:0/473169029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f68104100 0x7f3f68108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.133+0000 7f3f6d800700 1 -- 172.21.15.112:0/473169029 >> 172.21.15.112:0/473169029 conn(0x7f3f6807a600 msgr2=0x7f3f6807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:16.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.135+0000 7f3f6d800700 1 -- 172.21.15.112:0/473169029 shutdown_connections 2024-05-08T05:59:16.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.135+0000 7f3f6d800700 1 -- 172.21.15.112:0/473169029 wait complete. 2024-05-08T05:59:16.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.136+0000 7f3f6d800700 1 Processor -- start 2024-05-08T05:59:16.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.136+0000 7f3f6d800700 1 -- start start 2024-05-08T05:59:16.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.136+0000 7f3f6d800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f68103750 0x7f3f681a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.136+0000 7f3f6d800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f68104100 0x7f3f681a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.136+0000 7f3f6d800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f68108850 0x7f3f6819bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.136+0000 7f3f6d800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68113670 con 0x7f3f68103750 2024-05-08T05:59:16.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.136+0000 7f3f6d800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681134f0 con 0x7f3f68104100 2024-05-08T05:59:16.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.136+0000 7f3f6d800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681137f0 con 0x7f3f68108850 2024-05-08T05:59:16.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.137+0000 7f3f66a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f68103750 0x7f3f681a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.137+0000 7f3f66000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f68104100 0x7f3f681a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.137+0000 7f3f66000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f68104100 0x7f3f681a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:57742/0 (socket says 172.21.15.112:57742) 2024-05-08T05:59:16.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.137+0000 7f3f66a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f68103750 0x7f3f681a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40408/0 (socket says 172.21.15.112:40408) 2024-05-08T05:59:16.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.137+0000 7f3f66000700 1 -- 172.21.15.112:0/1037995880 learned_addr learned my addr 172.21.15.112:0/1037995880 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:16.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.137+0000 7f3f67400700 1 --2- 172.21.15.112:0/1037995880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f68108850 0x7f3f6819bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.137+0000 7f3f66000700 1 -- 172.21.15.112:0/1037995880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f68108850 msgr2=0x7f3f6819bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:16.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.137+0000 7f3f66000700 1 --2- 172.21.15.112:0/1037995880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f68108850 0x7f3f6819bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.137+0000 7f3f66000700 1 -- 172.21.15.112:0/1037995880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f68103750 msgr2=0x7f3f681a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:16.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.137+0000 7f3f66000700 1 --2- 172.21.15.112:0/1037995880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f68103750 0x7f3f681a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.137+0000 7f3f66000700 1 -- 172.21.15.112:0/1037995880 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6819c210 con 0x7f3f68104100 2024-05-08T05:59:16.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.137+0000 7f3f67400700 1 --2- 172.21.15.112:0/1037995880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f68108850 0x7f3f6819bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:59:16.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.138+0000 7f3f66000700 1 --2- 172.21.15.112:0/1037995880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f68104100 0x7f3f681a1e30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400d950 tx=0x7f3f5400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:16.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.138+0000 7f3f5f400700 1 -- 172.21.15.112:0/1037995880 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54016070 con 0x7f3f68104100 2024-05-08T05:59:16.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.138+0000 7f3f5f400700 1 -- 172.21.15.112:0/1037995880 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f5400fd40 con 0x7f3f68104100 2024-05-08T05:59:16.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.138+0000 7f3f5f400700 1 -- 172.21.15.112:0/1037995880 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5400e3b0 con 0x7f3f68104100 2024-05-08T05:59:16.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.138+0000 7f3f6d800700 1 -- 172.21.15.112:0/1037995880 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f6819c500 con 0x7f3f68104100 2024-05-08T05:59:16.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.138+0000 7f3f6d800700 1 -- 172.21.15.112:0/1037995880 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f681a87c0 con 0x7f3f68104100 2024-05-08T05:59:16.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.144+0000 7f3f78281b80 1 Processor -- start 2024-05-08T05:59:16.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.145+0000 7f3f5f400700 1 -- 172.21.15.112:0/1037995880 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3f54019070 con 0x7f3f68104100 2024-05-08T05:59:16.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.145+0000 7f3f78281b80 1 -- start start 2024-05-08T05:59:16.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.145+0000 7f3f78281b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c39b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.145+0000 7f3f78281b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c39f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.145+0000 7f3f78281b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c39fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.145+0000 7f3f78281b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fa6c2eda20 con 0x55fa6c39b980 2024-05-08T05:59:16.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.145+0000 7f3f78281b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fa6c28b760 con 0x55fa6c39b060 2024-05-08T05:59:16.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.145+0000 7f3f78281b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55fa6c2e99e0 con 0x55fa6c39f9d0 2024-05-08T05:59:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.146+0000 7f3f66a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c39f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.146+0000 7f3f66000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c39fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.146+0000 7f3f66000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c39fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:52308/0 (socket says 172.21.15.112:52308) 2024-05-08T05:59:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.146+0000 7f3f66000700 1 -- 172.21.15.112:0/1137292509 learned_addr learned my addr 172.21.15.112:0/1137292509 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.146+0000 7f3f66a00700 1 -- 172.21.15.112:0/1137292509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 msgr2=0x55fa6c39fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.146+0000 7f3f66a00700 1 --2- 172.21.15.112:0/1137292509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c39fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.147+0000 7f3f66a00700 1 -- 172.21.15.112:0/1137292509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 msgr2=0x55fa6c39b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.147+0000 7f3f67400700 1 --2- 172.21.15.112:0/1137292509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c39b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.147+0000 7f3f66a00700 1 --2- 172.21.15.112:0/1137292509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c39b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.147+0000 7f3f66a00700 1 -- 172.21.15.112:0/1137292509 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa6c23a8d0 con 0x55fa6c39b980 2024-05-08T05:59:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.147+0000 7f3f67400700 1 --2- 172.21.15.112:0/1137292509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c39b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:59:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.147+0000 7f3f66a00700 1 --2- 172.21.15.112:0/1137292509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c39f490 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009c80 tx=0x7f3f58009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0bf6f35ad7c126 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.147+0000 7f3f53e00700 1 -- 172.21.15.112:0/1137292509 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58011cb0 con 0x55fa6c39b980 2024-05-08T05:59:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.148+0000 7f3f53e00700 1 -- 172.21.15.112:0/1137292509 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f58011e50 con 0x55fa6c39b980 2024-05-08T05:59:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.148+0000 7f3f53e00700 1 -- 172.21.15.112:0/1137292509 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58005280 con 0x55fa6c39b980 2024-05-08T05:59:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.148+0000 7f3f5f400700 1 --2- 172.21.15.112:0/1037995880 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3f4c0b2900 0x7f3f4c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.149+0000 7f3f5f400700 1 -- 172.21.15.112:0/1037995880 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f3f540e1d10 con 0x7f3f68104100 2024-05-08T05:59:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.149+0000 7f3f67400700 1 --2- 172.21.15.112:0/1037995880 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3f4c0b2900 0x7f3f4c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.149+0000 7f3f78281b80 1 -- 172.21.15.112:0/1137292509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 msgr2=0x55fa6c39f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.149+0000 7f3f78281b80 1 --2- 172.21.15.112:0/1137292509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c39f490 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009c80 tx=0x7f3f58009f90 comp rx=0 tx=0).stop 2024-05-08T05:59:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.149+0000 7f3f78281b80 1 -- 172.21.15.112:0/1137292509 shutdown_connections 2024-05-08T05:59:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.149+0000 7f3f78281b80 1 --2- 172.21.15.112:0/1137292509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c39fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.149+0000 7f3f78281b80 1 --2- 172.21.15.112:0/1137292509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c39b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.149+0000 7f3f78281b80 1 --2- 172.21.15.112:0/1137292509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c39f490 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.149+0000 7f3f78281b80 1 -- 172.21.15.112:0/1137292509 >> 172.21.15.112:0/1137292509 conn(0x55fa6c2c9b60 msgr2=0x55fa6c3957a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.150+0000 7f3f78281b80 1 -- 172.21.15.112:0/1137292509 shutdown_connections 2024-05-08T05:59:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.150+0000 7f3f67400700 1 --2- 172.21.15.112:0/1037995880 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3f4c0b2900 0x7f3f4c0b4dc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000b780 tx=0x7f3f60006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.150+0000 7f3f78281b80 1 -- 172.21.15.112:0/1137292509 wait complete. 2024-05-08T05:59:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.151+0000 7f3f78281b80 1 -- start start 2024-05-08T05:59:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.151+0000 7f3f78281b80 1 Processor -- start 2024-05-08T05:59:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.152+0000 7f3f78281b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c3a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.152+0000 7f3f78281b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c3a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.152+0000 7f3f78281b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c3a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.152+0000 7f3f78281b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fa6c3a7be0 con 0x55fa6c39b980 2024-05-08T05:59:16.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.152+0000 7f3f78281b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fa6c253b20 con 0x55fa6c39b060 2024-05-08T05:59:16.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.152+0000 7f3f78281b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55fa6c3a7d60 con 0x55fa6c39f9d0 2024-05-08T05:59:16.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.152+0000 7f3f66a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c3a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.152+0000 7f3f78281b80 10 client.0 _ll_register_callbacks cb 0x55fa6c294c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:59:16.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.152+0000 7f3f66a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c3a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:57760/0 (socket says 172.21.15.112:57760) 2024-05-08T05:59:16.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.152+0000 7f3f66a00700 1 -- 172.21.15.112:0/2489600973 learned_addr learned my addr 172.21.15.112:0/2489600973 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:16.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.153+0000 7f3f66000700 1 --2- 172.21.15.112:0/2489600973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c3a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.153+0000 7f3f67400700 1 --2- 172.21.15.112:0/2489600973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c3a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.154+0000 7f3f66000700 1 -- 172.21.15.112:0/2489600973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 msgr2=0x55fa6c3a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:16.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.154+0000 7f3f66000700 1 --2- 172.21.15.112:0/2489600973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c3a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.154+0000 7f3f66000700 1 -- 172.21.15.112:0/2489600973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 msgr2=0x55fa6c3a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:16.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.154+0000 7f3f66000700 1 --2- 172.21.15.112:0/2489600973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c3a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.154+0000 7f3f66000700 1 -- 172.21.15.112:0/2489600973 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa6c2b3420 con 0x55fa6c39b980 2024-05-08T05:59:16.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.154+0000 7f3f66a00700 1 --2- 172.21.15.112:0/2489600973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c3a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:59:16.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.154+0000 7f3f67400700 1 --2- 172.21.15.112:0/2489600973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c3a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:59:16.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.154+0000 7f3f66000700 1 --2- 172.21.15.112:0/2489600973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c3a41b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3f540e54f0 tx=0x7f3f54010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a378c14c0bdfc16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:16.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.154+0000 7f3f50c00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:59:16.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.155+0000 7f3f50c00700 1 -- 172.21.15.112:0/2489600973 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54020070 con 0x55fa6c39b980 2024-05-08T05:59:16.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.155+0000 7f3f50c00700 1 -- 172.21.15.112:0/2489600973 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f3f540a9500 con 0x55fa6c39b980 2024-05-08T05:59:16.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.155+0000 7f3f78281b80 1 -- 172.21.15.112:0/2489600973 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f58011e50 con 0x55fa6c39b980 2024-05-08T05:59:16.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.155+0000 7f3f50c00700 1 -- 172.21.15.112:0/2489600973 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3f540a8bf0 con 0x55fa6c39b980 2024-05-08T05:59:16.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.155+0000 7f3f78281b80 10 client.15459 fetch_fsmap learned FSMap version 29 2024-05-08T05:59:16.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.155+0000 7f3f78281b80 1 -- 172.21.15.112:0/2489600973 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa6c3b2430 con 0x55fa6c39b980 2024-05-08T05:59:16.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.156+0000 7f3f50c00700 1 -- 172.21.15.112:0/2489600973 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f58011e50 con 0x55fa6c39b980 2024-05-08T05:59:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.156+0000 7f3f50c00700 1 -- 172.21.15.112:0/2489600973 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f3f540e7030 con 0x55fa6c39b980 2024-05-08T05:59:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.156+0000 7f3f78281b80 10 client.15459 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:59:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.156+0000 7f3f78281b80 10 client.15459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:59:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.156+0000 7f3f78281b80 20 client.15459 populate_metadata read hostname 'smithi112' 2024-05-08T05:59:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.156+0000 7f3f78281b80 1 --2- 172.21.15.112:0/2489600973 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fa6c3b4a70 0x55fa6c3b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.156+0000 7f3f78281b80 4 client.15459 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:59:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.156+0000 7f3f78281b80 1 -- 172.21.15.112:0/2489600973 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa6c3680a0 con 0x55fa6c3b4a70 2024-05-08T05:59:16.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.157+0000 7f3f66a00700 1 --2- 172.21.15.112:0/2489600973 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fa6c3b4a70 0x55fa6c3b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.157+0000 7f3f66a00700 1 --2- 172.21.15.112:0/2489600973 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fa6c3b4a70 0x55fa6c3b6e60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=532e3e90b563a4d6 server_cookie=96ec7fb9a5bb801f in_seq=0 out_seq=0 2024-05-08T05:59:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.157+0000 7f3f50c00700 10 client.15459 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:16.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.158+0000 7f3f50c00700 1 -- 172.21.15.112:0/2489600973 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa6c3680a0 con 0x55fa6c3b4a70 2024-05-08T05:59:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.158+0000 7f3f50c00700 10 client.15459 handle_command_reply: tid=0 2024-05-08T05:59:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.158+0000 7f3f78281b80 1 client.15459 shutdown 2024-05-08T05:59:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.159+0000 7f3f78281b80 1 -- 172.21.15.112:0/2489600973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 msgr2=0x55fa6c3a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.159+0000 7f3f78281b80 1 --2- 172.21.15.112:0/2489600973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c3a41b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3f540e54f0 tx=0x7f3f54010a40 comp rx=0 tx=0).stop 2024-05-08T05:59:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.159+0000 7f3f78281b80 1 -- 172.21.15.112:0/2489600973 shutdown_connections 2024-05-08T05:59:16.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.159+0000 7f3f78281b80 1 --2- 172.21.15.112:0/2489600973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c3a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.159+0000 7f3f78281b80 1 --2- 172.21.15.112:0/2489600973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c3a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.159+0000 7f3f78281b80 1 --2- 172.21.15.112:0/2489600973 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fa6c3b4a70 0x55fa6c3b6e60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.159+0000 7f3f78281b80 1 --2- 172.21.15.112:0/2489600973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c3a41b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.159+0000 7f3f50c00700 0 client.15459 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:16.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.159+0000 7f3f78281b80 1 -- 172.21.15.112:0/2489600973 >> 172.21.15.112:0/2489600973 conn(0x55fa6c2c9b60 msgr2=0x55fa6c3902b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:16.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.159+0000 7f3f78281b80 1 -- 172.21.15.112:0/2489600973 shutdown_connections 2024-05-08T05:59:16.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.159+0000 7f3f78281b80 1 -- 172.21.15.112:0/2489600973 wait complete. 2024-05-08T05:59:16.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.160+0000 7f3f78281b80 20 client.15459 trim_cache size 0 max 16384 2024-05-08T05:59:16.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.187+0000 7f3f78281b80 1 Processor -- start 2024-05-08T05:59:16.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.187+0000 7f3f78281b80 1 -- start start 2024-05-08T05:59:16.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.187+0000 7f3f78281b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b060 0x55fa6c3b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.187+0000 7f3f78281b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b980 0x55fa6c39eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.187+0000 7f3f78281b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c3bc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.187+0000 7f3f78281b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681ae390 con 0x55fa6c39b060 2024-05-08T05:59:16.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.187+0000 7f3f78281b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681abb90 con 0x55fa6c39b980 2024-05-08T05:59:16.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.187+0000 7f3f78281b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6810e490 con 0x55fa6c39f9d0 2024-05-08T05:59:16.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.188+0000 7f3f66a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b060 0x55fa6c3b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.188+0000 7f3f66a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b060 0x55fa6c3b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40438/0 (socket says 172.21.15.112:40438) 2024-05-08T05:59:16.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.188+0000 7f3f66a00700 1 -- 172.21.15.112:0/158736090 learned_addr learned my addr 172.21.15.112:0/158736090 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:16.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.188+0000 7f3f66000700 1 --2- 172.21.15.112:0/158736090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b980 0x55fa6c39eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.188+0000 7f3f67400700 1 --2- 172.21.15.112:0/158736090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c3bc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.188+0000 7f3f66000700 1 -- 172.21.15.112:0/158736090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 msgr2=0x55fa6c3bc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:16.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.188+0000 7f3f66000700 1 --2- 172.21.15.112:0/158736090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c3bc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.188+0000 7f3f66000700 1 -- 172.21.15.112:0/158736090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b060 msgr2=0x55fa6c3b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:16.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.188+0000 7f3f66000700 1 --2- 172.21.15.112:0/158736090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b060 0x55fa6c3b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.188+0000 7f3f66000700 1 -- 172.21.15.112:0/158736090 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa6c39f800 con 0x55fa6c39b980 2024-05-08T05:59:16.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.189+0000 7f3f66000700 1 --2- 172.21.15.112:0/158736090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b980 0x55fa6c39eaa0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400ffa0 tx=0x7f3f540df800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f371f0cbad48c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:16.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.189+0000 7f3f53400700 1 -- 172.21.15.112:0/158736090 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f540c2070 con 0x55fa6c39b980 2024-05-08T05:59:16.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.189+0000 7f3f53400700 1 -- 172.21.15.112:0/158736090 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f540a99a0 con 0x55fa6c39b980 2024-05-08T05:59:16.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.189+0000 7f3f53400700 1 -- 172.21.15.112:0/158736090 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54020070 con 0x55fa6c39b980 2024-05-08T05:59:16.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.190+0000 7f3f78281b80 1 -- 172.21.15.112:0/158736090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b980 msgr2=0x55fa6c39eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:16.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.190+0000 7f3f78281b80 1 --2- 172.21.15.112:0/158736090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b980 0x55fa6c39eaa0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400ffa0 tx=0x7f3f540df800 comp rx=0 tx=0).stop 2024-05-08T05:59:16.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.190+0000 7f3f78281b80 1 -- 172.21.15.112:0/158736090 shutdown_connections 2024-05-08T05:59:16.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.190+0000 7f3f78281b80 1 --2- 172.21.15.112:0/158736090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c3bc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.190+0000 7f3f78281b80 1 --2- 172.21.15.112:0/158736090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b060 0x55fa6c3b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.190+0000 7f3f78281b80 1 --2- 172.21.15.112:0/158736090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b980 0x55fa6c39eaa0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.190+0000 7f3f78281b80 1 -- 172.21.15.112:0/158736090 >> 172.21.15.112:0/158736090 conn(0x55fa6c2c9b60 msgr2=0x55fa6c39e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:16.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.190+0000 7f3f78281b80 1 -- 172.21.15.112:0/158736090 shutdown_connections 2024-05-08T05:59:16.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.190+0000 7f3f78281b80 1 -- 172.21.15.112:0/158736090 wait complete. 2024-05-08T05:59:16.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.191+0000 7f3f78281b80 1 -- start start 2024-05-08T05:59:16.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.191+0000 7f3f78281b80 1 Processor -- start 2024-05-08T05:59:16.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.191+0000 7f3f78281b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c3aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.191+0000 7f3f78281b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c3ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.191+0000 7f3f78281b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c3ab700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.191+0000 7f3f78281b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681aa050 con 0x55fa6c39b980 2024-05-08T05:59:16.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.191+0000 7f3f78281b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681ab4c0 con 0x55fa6c39b060 2024-05-08T05:59:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.191+0000 7f3f78281b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681aa720 con 0x55fa6c39f9d0 2024-05-08T05:59:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.192+0000 7f3f78281b80 10 client.0 _ll_register_callbacks cb 0x55fa6c294c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:59:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.192+0000 7f3f66a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c3aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.192+0000 7f3f66000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c3ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.192+0000 7f3f66000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c3ab1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40454/0 (socket says 172.21.15.112:40454) 2024-05-08T05:59:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.192+0000 7f3f66000700 1 -- 172.21.15.112:0/1444879557 learned_addr learned my addr 172.21.15.112:0/1444879557 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.192+0000 7f3f66a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c3aac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:57782/0 (socket says 172.21.15.112:57782) 2024-05-08T05:59:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.192+0000 7f3f67400700 1 --2- 172.21.15.112:0/1444879557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c3ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.192+0000 7f3f66000700 1 -- 172.21.15.112:0/1444879557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 msgr2=0x55fa6c3ab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.192+0000 7f3f66000700 1 --2- 172.21.15.112:0/1444879557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c3ab700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.192+0000 7f3f66000700 1 -- 172.21.15.112:0/1444879557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 msgr2=0x55fa6c3aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:16.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.192+0000 7f3f66000700 1 --2- 172.21.15.112:0/1444879557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c3aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.192+0000 7f3f66000700 1 -- 172.21.15.112:0/1444879557 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa6c41e350 con 0x55fa6c39b980 2024-05-08T05:59:16.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.192+0000 7f3f67400700 1 --2- 172.21.15.112:0/1444879557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c3ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:59:16.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.193+0000 7f3f66000700 1 --2- 172.21.15.112:0/1444879557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c3ab1c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3f540a8fa0 tx=0x7f3f540a5210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2bd0c46481b68f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:16.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.193+0000 7f3f50c00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:59:16.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.193+0000 7f3f50c00700 1 -- 172.21.15.112:0/1444879557 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f540c2030 con 0x55fa6c39b980 2024-05-08T05:59:16.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.193+0000 7f3f50c00700 1 -- 172.21.15.112:0/1444879557 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f3f540a5890 con 0x55fa6c39b980 2024-05-08T05:59:16.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.193+0000 7f3f78281b80 1 -- 172.21.15.112:0/1444879557 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f540a99a0 con 0x55fa6c39b980 2024-05-08T05:59:16.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.194+0000 7f3f50c00700 1 -- 172.21.15.112:0/1444879557 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3f54020070 con 0x55fa6c39b980 2024-05-08T05:59:16.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.194+0000 7f3f78281b80 10 client.15465 fetch_fsmap learned FSMap version 29 2024-05-08T05:59:16.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.194+0000 7f3f78281b80 1 -- 172.21.15.112:0/1444879557 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa6c41f620 con 0x55fa6c39b980 2024-05-08T05:59:16.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.195+0000 7f3f50c00700 1 -- 172.21.15.112:0/1444879557 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f540a99a0 con 0x55fa6c39b980 2024-05-08T05:59:16.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.195+0000 7f3f50c00700 1 -- 172.21.15.112:0/1444879557 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f3f540e7070 con 0x55fa6c39b980 2024-05-08T05:59:16.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.195+0000 7f3f78281b80 10 client.15465 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:59:16.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.195+0000 7f3f78281b80 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:59:16.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.195+0000 7f3f78281b80 20 client.15465 populate_metadata read hostname 'smithi112' 2024-05-08T05:59:16.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.195+0000 7f3f78281b80 1 --2- 172.21.15.112:0/1444879557 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fa6c41fa20 0x55fa6c421e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:16.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.195+0000 7f3f78281b80 4 client.15465 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:59:16.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.195+0000 7f3f78281b80 1 -- 172.21.15.112:0/1444879557 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa6c2fb400 con 0x55fa6c41fa20 2024-05-08T05:59:16.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.197+0000 7f3f66a00700 1 --2- 172.21.15.112:0/1444879557 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fa6c41fa20 0x55fa6c421e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:16.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.197+0000 7f3f66a00700 1 --2- 172.21.15.112:0/1444879557 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fa6c41fa20 0x55fa6c421e40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c3655da75a505d8 server_cookie=a6cc0b57a0cbc613 in_seq=0 out_seq=0 2024-05-08T05:59:16.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.197+0000 7f3f50c00700 10 client.15465 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:16.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.198+0000 7f3f50c00700 1 -- 172.21.15.112:0/1444879557 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fa6c2fb400 con 0x55fa6c41fa20 2024-05-08T05:59:16.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.198+0000 7f3f50c00700 10 client.15465 handle_command_reply: tid=0 2024-05-08T05:59:16.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.198+0000 7f3f78281b80 1 client.15465 shutdown 2024-05-08T05:59:16.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.198+0000 7f3f78281b80 1 -- 172.21.15.112:0/1444879557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 msgr2=0x55fa6c3ab1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:16.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.198+0000 7f3f78281b80 1 --2- 172.21.15.112:0/1444879557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c3ab1c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3f540a8fa0 tx=0x7f3f540a5210 comp rx=0 tx=0).stop 2024-05-08T05:59:16.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.198+0000 7f3f78281b80 1 -- 172.21.15.112:0/1444879557 shutdown_connections 2024-05-08T05:59:16.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.198+0000 7f3f78281b80 1 --2- 172.21.15.112:0/1444879557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fa6c39f9d0 0x55fa6c3ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.198+0000 7f3f78281b80 1 --2- 172.21.15.112:0/1444879557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fa6c39b060 0x55fa6c3aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.198+0000 7f3f78281b80 1 --2- 172.21.15.112:0/1444879557 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55fa6c41fa20 0x55fa6c421e40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.198+0000 7f3f78281b80 1 --2- 172.21.15.112:0/1444879557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fa6c39b980 0x55fa6c3ab1c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.198+0000 7f3f78281b80 1 -- 172.21.15.112:0/1444879557 >> 172.21.15.112:0/1444879557 conn(0x55fa6c2c9b60 msgr2=0x55fa6c39ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:16.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.198+0000 7f3f50c00700 0 client.15465 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:16.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.199+0000 7f3f78281b80 1 -- 172.21.15.112:0/1444879557 shutdown_connections 2024-05-08T05:59:16.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.199+0000 7f3f78281b80 1 -- 172.21.15.112:0/1444879557 wait complete. 2024-05-08T05:59:16.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.199+0000 7f3f78281b80 20 client.15465 trim_cache size 0 max 16384 2024-05-08T05:59:16.201 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:59:16.201 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T05:59:16.201 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-05-08T05:59:16.201 INFO:teuthology.orchestra.run.smithi112.stdout: "acdc742a-ba84-4862-96cf-9e9c423b782c": { 2024-05-08T05:59:16.201 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-05-08T05:59:16.201 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "acdc742a-ba84-4862-96cf-9e9c423b782c", 2024-05-08T05:59:16.201 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-05-08T05:59:16.201 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T05:59:16.201 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T05:59:16.201 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:59:16.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.200+0000 7f3f6d800700 1 -- 172.21.15.112:0/1037995880 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3f4c0b2900 msgr2=0x7f3f4c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:16.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.200+0000 7f3f6d800700 1 --2- 172.21.15.112:0/1037995880 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3f4c0b2900 0x7f3f4c0b4dc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000b780 tx=0x7f3f60006210 comp rx=0 tx=0).stop 2024-05-08T05:59:16.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.200+0000 7f3f6d800700 1 -- 172.21.15.112:0/1037995880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f68104100 msgr2=0x7f3f681a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:16.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.200+0000 7f3f6d800700 1 --2- 172.21.15.112:0/1037995880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f68104100 0x7f3f681a1e30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400d950 tx=0x7f3f5400dc60 comp rx=0 tx=0).stop 2024-05-08T05:59:16.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.200+0000 7f3f6d800700 1 -- 172.21.15.112:0/1037995880 shutdown_connections 2024-05-08T05:59:16.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.200+0000 7f3f6d800700 1 --2- 172.21.15.112:0/1037995880 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3f4c0b2900 0x7f3f4c0b4dc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.200+0000 7f3f6d800700 1 --2- 172.21.15.112:0/1037995880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f68108850 0x7f3f6819bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.200+0000 7f3f6d800700 1 --2- 172.21.15.112:0/1037995880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f68103750 0x7f3f681a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.200+0000 7f3f6d800700 1 --2- 172.21.15.112:0/1037995880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f68104100 0x7f3f681a1e30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:16.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.200+0000 7f3f6d800700 1 -- 172.21.15.112:0/1037995880 >> 172.21.15.112:0/1037995880 conn(0x7f3f6807a600 msgr2=0x7f3f68101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:16.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.200+0000 7f3f6d800700 1 -- 172.21.15.112:0/1037995880 shutdown_connections 2024-05-08T05:59:16.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:16.201+0000 7f3f6d800700 1 -- 172.21.15.112:0/1037995880 wait complete. 2024-05-08T05:59:17.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:16 smithi112 ceph-mon[30003]: pgmap v560: 97 pgs: 97 active+clean; 77 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-05-08T05:59:17.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:16 smithi162 ceph-mon[34800]: pgmap v560: 97 pgs: 97 active+clean; 77 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-05-08T05:59:17.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:16 smithi150 ceph-mon[34315]: pgmap v560: 97 pgs: 97 active+clean; 77 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-05-08T05:59:17.197 INFO:tasks.cephfs.filesystem:scrub status for tag:acdc742a-ba84-4862-96cf-9e9c423b782c - {'path': '/', 'tag': 'acdc742a-ba84-4862-96cf-9e9c423b782c', 'options': 'recursive,force'} 2024-05-08T05:59:17.950 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xffdb8 2024-05-08T05:59:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:18 smithi112 ceph-mon[30003]: pgmap v561: 97 pgs: 97 active+clean; 77 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 240 op/s 2024-05-08T05:59:19.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:18 smithi162 ceph-mon[34800]: pgmap v561: 97 pgs: 97 active+clean; 77 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 240 op/s 2024-05-08T05:59:19.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:18 smithi150 ceph-mon[34315]: pgmap v561: 97 pgs: 97 active+clean; 77 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 240 op/s 2024-05-08T05:59:21.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:20 smithi112 ceph-mon[30003]: pgmap v562: 97 pgs: 97 active+clean; 77 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-05-08T05:59:21.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:20 smithi162 ceph-mon[34800]: pgmap v562: 97 pgs: 97 active+clean; 77 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-05-08T05:59:21.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:20 smithi150 ceph-mon[34315]: pgmap v562: 97 pgs: 97 active+clean; 77 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-05-08T05:59:23.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:22 smithi112 ceph-mon[30003]: pgmap v563: 97 pgs: 97 active+clean; 78 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-05-08T05:59:23.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:22 smithi162 ceph-mon[34800]: pgmap v563: 97 pgs: 97 active+clean; 78 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-05-08T05:59:23.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:22 smithi150 ceph-mon[34315]: pgmap v563: 97 pgs: 97 active+clean; 78 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-05-08T05:59:24.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:23 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T05:59:24.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:23 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T05:59:24.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:23 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T05:59:24.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:23 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T05:59:24.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:23 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T05:59:24.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:23 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T05:59:25.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:24 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T05:59:25.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:24 smithi112 ceph-mon[30003]: pgmap v564: 97 pgs: 97 active+clean; 78 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 270 op/s 2024-05-08T05:59:25.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:24 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T05:59:25.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:24 smithi162 ceph-mon[34800]: pgmap v564: 97 pgs: 97 active+clean; 78 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 270 op/s 2024-05-08T05:59:25.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:24 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T05:59:25.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:24 smithi150 ceph-mon[34315]: pgmap v564: 97 pgs: 97 active+clean; 78 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 270 op/s 2024-05-08T05:59:27.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:26 smithi112 ceph-mon[30003]: pgmap v565: 97 pgs: 97 active+clean; 79 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.1 MiB/s wr, 219 op/s 2024-05-08T05:59:27.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:26 smithi162 ceph-mon[34800]: pgmap v565: 97 pgs: 97 active+clean; 79 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.1 MiB/s wr, 219 op/s 2024-05-08T05:59:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:26 smithi150 ceph-mon[34315]: pgmap v565: 97 pgs: 97 active+clean; 79 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.1 MiB/s wr, 219 op/s 2024-05-08T05:59:29.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:28 smithi112 ceph-mon[30003]: pgmap v566: 97 pgs: 97 active+clean; 79 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.0 MiB/s wr, 187 op/s 2024-05-08T05:59:29.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:28 smithi162 ceph-mon[34800]: pgmap v566: 97 pgs: 97 active+clean; 79 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.0 MiB/s wr, 187 op/s 2024-05-08T05:59:29.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:28 smithi150 ceph-mon[34315]: pgmap v566: 97 pgs: 97 active+clean; 79 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.0 MiB/s wr, 187 op/s 2024-05-08T05:59:31.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:30 smithi112 ceph-mon[30003]: pgmap v567: 97 pgs: 97 active+clean; 79 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.4 MiB/s wr, 235 op/s 2024-05-08T05:59:31.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:30 smithi162 ceph-mon[34800]: pgmap v567: 97 pgs: 97 active+clean; 79 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.4 MiB/s wr, 235 op/s 2024-05-08T05:59:31.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:30 smithi150 ceph-mon[34315]: pgmap v567: 97 pgs: 97 active+clean; 79 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.4 MiB/s wr, 235 op/s 2024-05-08T05:59:32.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:31 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:32.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:31 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:32.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:31 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:33.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:32 smithi112 ceph-mon[30003]: pgmap v568: 97 pgs: 97 active+clean; 79 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 903 KiB/s wr, 170 op/s 2024-05-08T05:59:33.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:33.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:33.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:33.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:32 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:33.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:32 smithi112 ceph-mon[30003]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-05-08T05:59:33.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:32 smithi112 ceph-mon[30003]: Unable to set osd_memory_target on smithi150 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-08T05:59:33.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:32 smithi162 ceph-mon[34800]: pgmap v568: 97 pgs: 97 active+clean; 79 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 903 KiB/s wr, 170 op/s 2024-05-08T05:59:33.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:32 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:32 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:32 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:32 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:32 smithi162 ceph-mon[34800]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-05-08T05:59:33.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:32 smithi162 ceph-mon[34800]: Unable to set osd_memory_target on smithi150 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-08T05:59:33.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:32 smithi150 ceph-mon[34315]: pgmap v568: 97 pgs: 97 active+clean; 79 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 903 KiB/s wr, 170 op/s 2024-05-08T05:59:33.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:32 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:33.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:32 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:33.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:32 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:33.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:32 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:33.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:32 smithi150 ceph-mon[34315]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-05-08T05:59:33.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:32 smithi150 ceph-mon[34315]: Unable to set osd_memory_target on smithi150 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-08T05:59:34.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:33 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:34.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:33 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:34.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:33 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:34.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:33 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:34.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:33 smithi112 ceph-mon[30003]: Adjusting osd_memory_target on smithi112 to 176.4M 2024-05-08T05:59:34.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:33 smithi112 ceph-mon[30003]: Unable to set osd_memory_target on smithi112 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-05-08T05:59:34.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:33 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:34.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:33 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:34.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:33 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:59:34.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:33 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:34.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:33 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:34.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:33 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:34.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:33 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:34.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:33 smithi162 ceph-mon[34800]: Adjusting osd_memory_target on smithi112 to 176.4M 2024-05-08T05:59:34.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:33 smithi162 ceph-mon[34800]: Unable to set osd_memory_target on smithi112 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-05-08T05:59:34.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:33 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:34.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:33 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:34.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:33 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:59:34.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:33 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:34.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:33 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:34.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:33 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:34.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:33 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:34.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:33 smithi150 ceph-mon[34315]: Adjusting osd_memory_target on smithi112 to 176.4M 2024-05-08T05:59:34.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:33 smithi150 ceph-mon[34315]: Unable to set osd_memory_target on smithi112 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-05-08T05:59:34.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:33 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:34.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:33 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:34.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:33 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T05:59:35.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:34 smithi150 ceph-mon[34315]: pgmap v569: 97 pgs: 97 active+clean; 80 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.2 MiB/s wr, 216 op/s 2024-05-08T05:59:35.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:34 smithi112 ceph-mon[30003]: pgmap v569: 97 pgs: 97 active+clean; 80 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.2 MiB/s wr, 216 op/s 2024-05-08T05:59:35.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:34 smithi162 ceph-mon[34800]: pgmap v569: 97 pgs: 97 active+clean; 80 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.2 MiB/s wr, 216 op/s 2024-05-08T05:59:37.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:36 smithi150 ceph-mon[34315]: pgmap v570: 97 pgs: 97 active+clean; 80 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 939 KiB/s wr, 169 op/s 2024-05-08T05:59:37.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:36 smithi112 ceph-mon[30003]: pgmap v570: 97 pgs: 97 active+clean; 80 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 939 KiB/s wr, 169 op/s 2024-05-08T05:59:37.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:36 smithi162 ceph-mon[34800]: pgmap v570: 97 pgs: 97 active+clean; 80 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 939 KiB/s wr, 169 op/s 2024-05-08T05:59:39.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:38 smithi150 ceph-mon[34315]: pgmap v571: 97 pgs: 97 active+clean; 81 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 946 KiB/s wr, 169 op/s 2024-05-08T05:59:39.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:38 smithi112 ceph-mon[30003]: pgmap v571: 97 pgs: 97 active+clean; 81 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 946 KiB/s wr, 169 op/s 2024-05-08T05:59:39.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:38 smithi162 ceph-mon[34800]: pgmap v571: 97 pgs: 97 active+clean; 81 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 946 KiB/s wr, 169 op/s 2024-05-08T05:59:40.323 INFO:tasks.workunit.client.0.smithi112.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-05-08T05:59:40.323 INFO:tasks.workunit.client.0.smithi112.stdout:All 50000 operations completed A-OK! 2024-05-08T05:59:40.329 INFO:tasks.workunit.client.0.smithi112.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-05-08T05:59:40.333 INFO:tasks.workunit.client.0.smithi112.stdout:Seed set to 1 2024-05-08T05:59:40.488 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-05-08T05:59:40.488 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-05-08T05:59:40.492 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-05-08T05:59:40.492 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-05-08T05:59:40.492 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support clone range, disabling! 2024-05-08T05:59:40.497 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support dedupe range, disabling! 2024-05-08T05:59:40.502 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support exchange range, disabling! 2024-05-08T05:59:40.502 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero length punch hole 2024-05-08T05:59:40.502 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero length punch hole 2024-05-08T05:59:40.503 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero length punch hole 2024-05-08T05:59:40.503 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero size read 2024-05-08T05:59:40.503 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero length punch hole 2024-05-08T05:59:40.503 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero size read 2024-05-08T05:59:40.503 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero length punch hole 2024-05-08T05:59:40.503 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x4310e9 2024-05-08T05:59:40.519 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x954a45 2024-05-08T05:59:40.590 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x9bc582 2024-05-08T05:59:40.649 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x9eecbc 2024-05-08T05:59:41.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:40 smithi150 ceph-mon[34315]: pgmap v572: 97 pgs: 97 active+clean; 81 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 207 op/s 2024-05-08T05:59:41.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:40 smithi112 ceph-mon[30003]: pgmap v572: 97 pgs: 97 active+clean; 81 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 207 op/s 2024-05-08T05:59:41.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:40 smithi162 ceph-mon[34800]: pgmap v572: 97 pgs: 97 active+clean; 81 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 207 op/s 2024-05-08T05:59:43.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:42 smithi150 ceph-mon[34315]: pgmap v573: 97 pgs: 97 active+clean; 81 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 940 KiB/s wr, 158 op/s 2024-05-08T05:59:43.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:42 smithi112 ceph-mon[30003]: pgmap v573: 97 pgs: 97 active+clean; 81 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 940 KiB/s wr, 158 op/s 2024-05-08T05:59:43.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:42 smithi162 ceph-mon[34800]: pgmap v573: 97 pgs: 97 active+clean; 81 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 940 KiB/s wr, 158 op/s 2024-05-08T05:59:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:44 smithi150 ceph-mon[34315]: pgmap v574: 97 pgs: 97 active+clean; 81 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 1.1 MiB/s wr, 203 op/s 2024-05-08T05:59:45.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:44 smithi112 ceph-mon[30003]: pgmap v574: 97 pgs: 97 active+clean; 81 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 1.1 MiB/s wr, 203 op/s 2024-05-08T05:59:45.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:44 smithi162 ceph-mon[34800]: pgmap v574: 97 pgs: 97 active+clean; 81 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 1.1 MiB/s wr, 203 op/s 2024-05-08T05:59:45.823 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x9fdd84 2024-05-08T05:59:47.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:46 smithi150 ceph-mon[34315]: pgmap v575: 97 pgs: 97 active+clean; 86 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 192 op/s 2024-05-08T05:59:47.199 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T05:59:47.276 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:46 smithi112 ceph-mon[30003]: pgmap v575: 97 pgs: 97 active+clean; 86 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 192 op/s 2024-05-08T05:59:47.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:46 smithi162 ceph-mon[34800]: pgmap v575: 97 pgs: 97 active+clean; 86 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 192 op/s 2024-05-08T05:59:47.681 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:59:49.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:48 smithi112 ceph-mon[30003]: pgmap v576: 97 pgs: 97 active+clean; 87 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 229 op/s 2024-05-08T05:59:49.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:48 smithi162 ceph-mon[34800]: pgmap v576: 97 pgs: 97 active+clean; 87 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 229 op/s 2024-05-08T05:59:49.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:48 smithi150 ceph-mon[34315]: pgmap v576: 97 pgs: 97 active+clean; 87 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 229 op/s 2024-05-08T05:59:50.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.177+0000 7f6339e00700 1 -- 172.21.15.112:0/2040386343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633406b1b0 msgr2=0x7f633406b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:50.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.177+0000 7f6339e00700 1 --2- 172.21.15.112:0/2040386343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633406b1b0 0x7f633406b590 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6324009990 tx=0x7f6324009ca0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.178+0000 7f6339e00700 1 -- 172.21.15.112:0/2040386343 shutdown_connections 2024-05-08T05:59:50.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.178+0000 7f6339e00700 1 --2- 172.21.15.112:0/2040386343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f633410ff50 0x7f6334112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.178+0000 7f6339e00700 1 --2- 172.21.15.112:0/2040386343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633406b1b0 0x7f633406b590 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.178+0000 7f6339e00700 1 --2- 172.21.15.112:0/2040386343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633406bad0 0x7f633410fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.178+0000 7f6339e00700 1 -- 172.21.15.112:0/2040386343 >> 172.21.15.112:0/2040386343 conn(0x7f6334079350 msgr2=0x7f6334079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:50.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.178+0000 7f6339e00700 1 -- 172.21.15.112:0/2040386343 shutdown_connections 2024-05-08T05:59:50.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.178+0000 7f6339e00700 1 -- 172.21.15.112:0/2040386343 wait complete. 2024-05-08T05:59:50.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.179+0000 7f6339e00700 1 Processor -- start 2024-05-08T05:59:50.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.179+0000 7f6339e00700 1 -- start start 2024-05-08T05:59:50.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.180+0000 7f6339e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633406b1b0 0x7f6334104760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.180+0000 7f6339e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633406bad0 0x7f6334104ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.180+0000 7f6339e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f633410ff50 0x7f633410ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.180+0000 7f6339e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63341158b0 con 0x7f633410ff50 2024-05-08T05:59:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.180+0000 7f6339e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6334115730 con 0x7f633406b1b0 2024-05-08T05:59:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.180+0000 7f6339e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6334115a30 con 0x7f633406bad0 2024-05-08T05:59:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.180+0000 7f6332000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633406bad0 0x7f6334104ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.180+0000 7f6332a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633406b1b0 0x7f6334104760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.180+0000 7f6332000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633406bad0 0x7f6334104ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:35244/0 (socket says 172.21.15.112:35244) 2024-05-08T05:59:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.180+0000 7f6332a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633406b1b0 0x7f6334104760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:41664/0 (socket says 172.21.15.112:41664) 2024-05-08T05:59:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.180+0000 7f6332000700 1 -- 172.21.15.112:0/3326038487 learned_addr learned my addr 172.21.15.112:0/3326038487 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.180+0000 7f6332a00700 1 -- 172.21.15.112:0/3326038487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633406bad0 msgr2=0x7f6334104ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.180+0000 7f6332a00700 1 --2- 172.21.15.112:0/3326038487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633406bad0 0x7f6334104ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.180+0000 7f6332a00700 1 -- 172.21.15.112:0/3326038487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f633410ff50 msgr2=0x7f633410ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.180+0000 7f6332a00700 1 --2- 172.21.15.112:0/3326038487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f633410ff50 0x7f633410ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.181+0000 7f6332a00700 1 -- 172.21.15.112:0/3326038487 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633410f620 con 0x7f633406b1b0 2024-05-08T05:59:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.181+0000 7f6332000700 1 --2- 172.21.15.112:0/3326038487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633406bad0 0x7f6334104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:59:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.181+0000 7f6332a00700 1 --2- 172.21.15.112:0/3326038487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633406b1b0 0x7f6334104760 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f63240098e0 tx=0x7f63240057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.181+0000 7f6338a00700 1 -- 172.21.15.112:0/3326038487 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324027070 con 0x7f633406b1b0 2024-05-08T05:59:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.181+0000 7f6338a00700 1 -- 172.21.15.112:0/3326038487 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6324022470 con 0x7f633406b1b0 2024-05-08T05:59:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.181+0000 7f6338a00700 1 -- 172.21.15.112:0/3326038487 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324017650 con 0x7f633406b1b0 2024-05-08T05:59:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.182+0000 7f6339e00700 1 -- 172.21.15.112:0/3326038487 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633410f8b0 con 0x7f633406b1b0 2024-05-08T05:59:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.182+0000 7f6339e00700 1 -- 172.21.15.112:0/3326038487 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f633410fd40 con 0x7f633406b1b0 2024-05-08T05:59:50.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.186+0000 7f6338a00700 1 -- 172.21.15.112:0/3326038487 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6324009d10 con 0x7f633406b1b0 2024-05-08T05:59:50.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.188+0000 7f6338a00700 1 --2- 172.21.15.112:0/3326038487 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f63180b2900 0x7f63180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.188+0000 7f6338a00700 1 -- 172.21.15.112:0/3326038487 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f63240e2a00 con 0x7f633406b1b0 2024-05-08T05:59:50.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.189+0000 7f6332000700 1 --2- 172.21.15.112:0/3326038487 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f63180b2900 0x7f63180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.189+0000 7f63447c4b80 1 Processor -- start 2024-05-08T05:59:50.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.189+0000 7f63447c4b80 1 -- start start 2024-05-08T05:59:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.189+0000 7f63447c4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f2cbc0 0x55d142f2cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.189+0000 7f63447c4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 0x55d142f272d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.189+0000 7f6332000700 1 --2- 172.21.15.112:0/3326038487 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f63180b2900 0x7f63180b4dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f63341039d0 tx=0x7f6320009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.189+0000 7f63447c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f278f0 0x55d142f27d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.189+0000 7f63447c4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d142e79a20 con 0x55d142f2d4e0 2024-05-08T05:59:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.189+0000 7f63447c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d142e17760 con 0x55d142f278f0 2024-05-08T05:59:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.190+0000 7f63447c4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d142e759e0 con 0x55d142f2cbc0 2024-05-08T05:59:50.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.190+0000 7f6332a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f2cbc0 0x55d142f2cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.190+0000 7f6332000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 0x55d142f272d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.190+0000 7f6332a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f2cbc0 0x55d142f2cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:35254/0 (socket says 172.21.15.112:35254) 2024-05-08T05:59:50.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.190+0000 7f6332a00700 1 -- 172.21.15.112:0/735270671 learned_addr learned my addr 172.21.15.112:0/735270671 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:50.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.190+0000 7f6333400700 1 --2- 172.21.15.112:0/735270671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f278f0 0x55d142f27d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.190+0000 7f6332a00700 1 -- 172.21.15.112:0/735270671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f278f0 msgr2=0x55d142f27d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:50.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.190+0000 7f6332a00700 1 --2- 172.21.15.112:0/735270671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f278f0 0x55d142f27d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.190+0000 7f6332a00700 1 -- 172.21.15.112:0/735270671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 msgr2=0x55d142f272d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:50.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.190+0000 7f6332a00700 1 --2- 172.21.15.112:0/735270671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 0x55d142f272d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.190+0000 7f6332a00700 1 -- 172.21.15.112:0/735270671 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d142dc68d0 con 0x55d142f2cbc0 2024-05-08T05:59:50.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.191+0000 7f6332a00700 1 --2- 172.21.15.112:0/735270671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f2cbc0 0x55d142f2cfa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63240a60a0 tx=0x7f6324022cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abfad8e19380d348 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:50.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.191+0000 7f6328c00700 1 -- 172.21.15.112:0/735270671 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63240aa020 con 0x55d142f2cbc0 2024-05-08T05:59:50.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.191+0000 7f6328c00700 1 -- 172.21.15.112:0/735270671 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63240ae3d0 con 0x55d142f2cbc0 2024-05-08T05:59:50.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.191+0000 7f6328c00700 1 -- 172.21.15.112:0/735270671 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63240aa280 con 0x55d142f2cbc0 2024-05-08T05:59:50.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.192+0000 7f63447c4b80 1 -- 172.21.15.112:0/735270671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f2cbc0 msgr2=0x55d142f2cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:50.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.192+0000 7f63447c4b80 1 --2- 172.21.15.112:0/735270671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f2cbc0 0x55d142f2cfa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63240a60a0 tx=0x7f6324022cc0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.192+0000 7f63447c4b80 1 -- 172.21.15.112:0/735270671 shutdown_connections 2024-05-08T05:59:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.192+0000 7f63447c4b80 1 --2- 172.21.15.112:0/735270671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f278f0 0x55d142f27d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.192+0000 7f63447c4b80 1 --2- 172.21.15.112:0/735270671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f2cbc0 0x55d142f2cfa0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.192+0000 7f63447c4b80 1 --2- 172.21.15.112:0/735270671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 0x55d142f272d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.192+0000 7f63447c4b80 1 -- 172.21.15.112:0/735270671 >> 172.21.15.112:0/735270671 conn(0x55d142e9f6d0 msgr2=0x55d142f21b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.192+0000 7f63447c4b80 1 -- 172.21.15.112:0/735270671 shutdown_connections 2024-05-08T05:59:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.192+0000 7f63447c4b80 1 -- 172.21.15.112:0/735270671 wait complete. 2024-05-08T05:59:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.193+0000 7f63447c4b80 1 -- start start 2024-05-08T05:59:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.193+0000 7f63447c4b80 1 Processor -- start 2024-05-08T05:59:50.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.194+0000 7f63447c4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f278f0 0x55d142f33250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.194+0000 7f63447c4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2cbc0 0x55d142f305f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.194+0000 7f63447c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f2d4e0 0x55d142f30b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.194+0000 7f63447c4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d142f33ce0 con 0x55d142f2cbc0 2024-05-08T05:59:50.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.194+0000 7f63447c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d142ddfb20 con 0x55d142f2d4e0 2024-05-08T05:59:50.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.194+0000 7f63447c4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d142f33e60 con 0x55d142f278f0 2024-05-08T05:59:50.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.194+0000 7f63447c4b80 10 client.0 _ll_register_callbacks cb 0x55d142e20c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:59:50.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.194+0000 7f6332a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f278f0 0x55d142f33250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.194+0000 7f6332a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f278f0 0x55d142f33250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:35260/0 (socket says 172.21.15.112:35260) 2024-05-08T05:59:50.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.194+0000 7f6332a00700 1 -- 172.21.15.112:0/2304953246 learned_addr learned my addr 172.21.15.112:0/2304953246 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:50.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.195+0000 7f6332a00700 1 -- 172.21.15.112:0/2304953246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f2d4e0 msgr2=0x55d142f30b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:50.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.195+0000 7f6332a00700 1 --2- 172.21.15.112:0/2304953246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f2d4e0 0x55d142f30b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.195+0000 7f6332a00700 1 -- 172.21.15.112:0/2304953246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2cbc0 msgr2=0x55d142f305f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:50.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.195+0000 7f6332000700 1 --2- 172.21.15.112:0/2304953246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2cbc0 0x55d142f305f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.195+0000 7f6332a00700 1 --2- 172.21.15.112:0/2304953246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2cbc0 0x55d142f305f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.195+0000 7f6332a00700 1 -- 172.21.15.112:0/2304953246 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d142e3f420 con 0x55d142f278f0 2024-05-08T05:59:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.196+0000 7f6332a00700 1 --2- 172.21.15.112:0/2304953246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f278f0 0x55d142f33250 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6324005cd0 tx=0x7f63240096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=878f0b20620d2f7d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.196+0000 7f631d600700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:59:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.196+0000 7f631d600700 1 -- 172.21.15.112:0/2304953246 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63240aa480 con 0x55d142f278f0 2024-05-08T05:59:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.196+0000 7f631d600700 1 -- 172.21.15.112:0/2304953246 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f63240b2960 con 0x55d142f278f0 2024-05-08T05:59:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.196+0000 7f63447c4b80 1 -- 172.21.15.112:0/2304953246 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63240ae3d0 con 0x55d142f278f0 2024-05-08T05:59:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.196+0000 7f631d600700 1 -- 172.21.15.112:0/2304953246 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63240b2b60 con 0x55d142f278f0 2024-05-08T05:59:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.196+0000 7f63447c4b80 10 client.25645 fetch_fsmap learned FSMap version 29 2024-05-08T05:59:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.196+0000 7f63447c4b80 1 -- 172.21.15.112:0/2304953246 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d142f33ce0 con 0x55d142f278f0 2024-05-08T05:59:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.196+0000 7f631d600700 1 -- 172.21.15.112:0/2304953246 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63240ae3d0 con 0x55d142f278f0 2024-05-08T05:59:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.197+0000 7f631d600700 1 -- 172.21.15.112:0/2304953246 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f63240a5990 con 0x55d142f278f0 2024-05-08T05:59:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.197+0000 7f63447c4b80 10 client.25645 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:59:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.197+0000 7f63447c4b80 10 client.25645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:59:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.197+0000 7f63447c4b80 20 client.25645 populate_metadata read hostname 'smithi112' 2024-05-08T05:59:50.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.197+0000 7f63447c4b80 1 --2- 172.21.15.112:0/2304953246 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55d142f40820 0x55d142f42c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.197+0000 7f63447c4b80 4 client.25645 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:59:50.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.197+0000 7f63447c4b80 1 -- 172.21.15.112:0/2304953246 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d142ef4140 con 0x55d142f40820 2024-05-08T05:59:50.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.197+0000 7f6332000700 1 --2- 172.21.15.112:0/2304953246 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55d142f40820 0x55d142f42c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.198+0000 7f6332000700 1 --2- 172.21.15.112:0/2304953246 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55d142f40820 0x55d142f42c10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=350980133149b9d7 server_cookie=6707fc075e4820f3 in_seq=0 out_seq=0 2024-05-08T05:59:50.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.198+0000 7f631d600700 10 client.25645 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.200+0000 7f631d600700 1 -- 172.21.15.112:0/2304953246 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d142ef4140 con 0x55d142f40820 2024-05-08T05:59:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.201+0000 7f631d600700 10 client.25645 handle_command_reply: tid=0 2024-05-08T05:59:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.201+0000 7f63447c4b80 1 client.25645 shutdown 2024-05-08T05:59:50.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.201+0000 7f63447c4b80 1 -- 172.21.15.112:0/2304953246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f278f0 msgr2=0x55d142f33250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:50.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.201+0000 7f63447c4b80 1 --2- 172.21.15.112:0/2304953246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f278f0 0x55d142f33250 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6324005cd0 tx=0x7f63240096e0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.201+0000 7f63447c4b80 1 -- 172.21.15.112:0/2304953246 shutdown_connections 2024-05-08T05:59:50.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.201+0000 7f63447c4b80 1 --2- 172.21.15.112:0/2304953246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f2d4e0 0x55d142f30b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.201+0000 7f63447c4b80 1 --2- 172.21.15.112:0/2304953246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f278f0 0x55d142f33250 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.201+0000 7f63447c4b80 1 --2- 172.21.15.112:0/2304953246 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55d142f40820 0x55d142f42c10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.201+0000 7f63447c4b80 1 --2- 172.21.15.112:0/2304953246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2cbc0 0x55d142f305f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.201+0000 7f63447c4b80 1 -- 172.21.15.112:0/2304953246 >> 172.21.15.112:0/2304953246 conn(0x55d142e9f6d0 msgr2=0x55d142f21190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:50.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.201+0000 7f631d600700 0 client.25645 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:50.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.201+0000 7f63447c4b80 1 -- 172.21.15.112:0/2304953246 shutdown_connections 2024-05-08T05:59:50.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.202+0000 7f63447c4b80 1 -- 172.21.15.112:0/2304953246 wait complete. 2024-05-08T05:59:50.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.202+0000 7f63447c4b80 20 client.25645 trim_cache size 0 max 16384 2024-05-08T05:59:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.227+0000 7f63447c4b80 1 Processor -- start 2024-05-08T05:59:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.227+0000 7f63447c4b80 1 -- start start 2024-05-08T05:59:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.227+0000 7f63447c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f278f0 0x55d142f272d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.227+0000 7f63447c4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f2cbc0 0x55d142f2e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.227+0000 7f63447c4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 0x55d142f3cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.227+0000 7f63447c4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63341bf340 con 0x55d142f2d4e0 2024-05-08T05:59:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.227+0000 7f63447c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f63341bcb40 con 0x55d142f278f0 2024-05-08T05:59:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.227+0000 7f63447c4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f63341ba990 con 0x55d142f2cbc0 2024-05-08T05:59:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.227+0000 7f6332000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f2cbc0 0x55d142f2e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.227+0000 7f6332a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f278f0 0x55d142f272d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.227+0000 7f6332000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f2cbc0 0x55d142f2e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:35268/0 (socket says 172.21.15.112:35268) 2024-05-08T05:59:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.227+0000 7f6332000700 1 -- 172.21.15.112:0/1949828634 learned_addr learned my addr 172.21.15.112:0/1949828634 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.228+0000 7f6333400700 1 --2- 172.21.15.112:0/1949828634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 0x55d142f3cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.228+0000 7f6332000700 1 -- 172.21.15.112:0/1949828634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f278f0 msgr2=0x55d142f272d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.228+0000 7f6332000700 1 --2- 172.21.15.112:0/1949828634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f278f0 0x55d142f272d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.228+0000 7f6332000700 1 -- 172.21.15.112:0/1949828634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 msgr2=0x55d142f3cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.228+0000 7f6332000700 1 --2- 172.21.15.112:0/1949828634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 0x55d142f3cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.228+0000 7f6332000700 1 -- 172.21.15.112:0/1949828634 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d142f1cab0 con 0x55d142f2cbc0 2024-05-08T05:59:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.228+0000 7f6333400700 1 --2- 172.21.15.112:0/1949828634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 0x55d142f3cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:59:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.228+0000 7f6332a00700 1 --2- 172.21.15.112:0/1949828634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f278f0 0x55d142f272d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:59:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.228+0000 7f6332000700 1 --2- 172.21.15.112:0/1949828634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f2cbc0 0x55d142f2e1a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6320013010 tx=0x7f6320014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e459607bbe384f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.229+0000 7f631fe00700 1 -- 172.21.15.112:0/1949828634 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63200293c0 con 0x55d142f2cbc0 2024-05-08T05:59:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.229+0000 7f631fe00700 1 -- 172.21.15.112:0/1949828634 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f632002f070 con 0x55d142f2cbc0 2024-05-08T05:59:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.229+0000 7f631fe00700 1 -- 172.21.15.112:0/1949828634 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632001db20 con 0x55d142f2cbc0 2024-05-08T05:59:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.229+0000 7f63447c4b80 1 -- 172.21.15.112:0/1949828634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f2cbc0 msgr2=0x55d142f2e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.229+0000 7f63447c4b80 1 --2- 172.21.15.112:0/1949828634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f2cbc0 0x55d142f2e1a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6320013010 tx=0x7f6320014000 comp rx=0 tx=0).stop 2024-05-08T05:59:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.230+0000 7f63447c4b80 1 -- 172.21.15.112:0/1949828634 shutdown_connections 2024-05-08T05:59:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.230+0000 7f63447c4b80 1 --2- 172.21.15.112:0/1949828634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 0x55d142f3cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.230+0000 7f63447c4b80 1 --2- 172.21.15.112:0/1949828634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f278f0 0x55d142f272d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.230+0000 7f63447c4b80 1 --2- 172.21.15.112:0/1949828634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f2cbc0 0x55d142f2e1a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.230+0000 7f63447c4b80 1 -- 172.21.15.112:0/1949828634 >> 172.21.15.112:0/1949828634 conn(0x55d142e9f6d0 msgr2=0x55d142f2b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.230+0000 7f63447c4b80 1 -- 172.21.15.112:0/1949828634 shutdown_connections 2024-05-08T05:59:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.230+0000 7f63447c4b80 1 -- 172.21.15.112:0/1949828634 wait complete. 2024-05-08T05:59:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.230+0000 7f63447c4b80 1 -- start start 2024-05-08T05:59:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.230+0000 7f63447c4b80 1 Processor -- start 2024-05-08T05:59:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.231+0000 7f63447c4b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f278f0 0x55d142f39760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.231+0000 7f63447c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f2cbc0 0x55d142f47ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.231+0000 7f63447c4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 0x55d142f48220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.231+0000 7f63447c4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63341bb000 con 0x55d142f2d4e0 2024-05-08T05:59:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.231+0000 7f63447c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f63341bc470 con 0x55d142f2cbc0 2024-05-08T05:59:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.231+0000 7f63447c4b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f63341bb6d0 con 0x55d142f278f0 2024-05-08T05:59:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.231+0000 7f63447c4b80 10 client.0 _ll_register_callbacks cb 0x55d142e20c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:59:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.231+0000 7f6333400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 0x55d142f48220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.232+0000 7f6333400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 0x55d142f48220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47374/0 (socket says 172.21.15.112:47374) 2024-05-08T05:59:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.232+0000 7f6333400700 1 -- 172.21.15.112:0/4191577888 learned_addr learned my addr 172.21.15.112:0/4191577888 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.232+0000 7f6332000700 1 --2- 172.21.15.112:0/4191577888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f2cbc0 0x55d142f47ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.232+0000 7f6333400700 1 -- 172.21.15.112:0/4191577888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f278f0 msgr2=0x55d142f39760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:59:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.232+0000 7f6332a00700 1 --2- 172.21.15.112:0/4191577888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f278f0 0x55d142f39760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.232+0000 7f6333400700 1 --2- 172.21.15.112:0/4191577888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f278f0 0x55d142f39760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.232+0000 7f6333400700 1 -- 172.21.15.112:0/4191577888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f2cbc0 msgr2=0x55d142f47ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.232+0000 7f6333400700 1 --2- 172.21.15.112:0/4191577888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f2cbc0 0x55d142f47ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.232+0000 7f6333400700 1 -- 172.21.15.112:0/4191577888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d142faa080 con 0x55d142f2d4e0 2024-05-08T05:59:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.232+0000 7f6332000700 1 --2- 172.21.15.112:0/4191577888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f2cbc0 0x55d142f47ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:59:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.233+0000 7f6333400700 1 --2- 172.21.15.112:0/4191577888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 0x55d142f48220 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f632c007b20 tx=0x7f632c007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791850ca7260be7e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.234+0000 7f631d600700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T05:59:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.234+0000 7f631d600700 1 -- 172.21.15.112:0/4191577888 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c003c00 con 0x55d142f2d4e0 2024-05-08T05:59:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.234+0000 7f631d600700 1 -- 172.21.15.112:0/4191577888 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f632c003da0 con 0x55d142f2d4e0 2024-05-08T05:59:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.234+0000 7f63447c4b80 1 -- 172.21.15.112:0/4191577888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f632002f070 con 0x55d142f2d4e0 2024-05-08T05:59:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.234+0000 7f631d600700 1 -- 172.21.15.112:0/4191577888 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f632c012900 con 0x55d142f2d4e0 2024-05-08T05:59:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.234+0000 7f63447c4b80 10 client.15474 fetch_fsmap learned FSMap version 29 2024-05-08T05:59:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.234+0000 7f63447c4b80 1 -- 172.21.15.112:0/4191577888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d142fab130 con 0x55d142f2d4e0 2024-05-08T05:59:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.234+0000 7f631d600700 1 -- 172.21.15.112:0/4191577888 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f632002f070 con 0x55d142f2d4e0 2024-05-08T05:59:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.235+0000 7f631d600700 1 -- 172.21.15.112:0/4191577888 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f632c020030 con 0x55d142f2d4e0 2024-05-08T05:59:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.236+0000 7f63447c4b80 10 client.15474 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:59:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.236+0000 7f63447c4b80 10 client.15474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T05:59:50.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.236+0000 7f63447c4b80 20 client.15474 populate_metadata read hostname 'smithi112' 2024-05-08T05:59:50.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.236+0000 7f63447c4b80 1 --2- 172.21.15.112:0/4191577888 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55d142fab840 0x55d142fadc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:50.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.236+0000 7f63447c4b80 4 client.15474 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:59:50.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.236+0000 7f63447c4b80 1 -- 172.21.15.112:0/4191577888 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d142e87400 con 0x55d142fab840 2024-05-08T05:59:50.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.236+0000 7f6332a00700 1 --2- 172.21.15.112:0/4191577888 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55d142fab840 0x55d142fadc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:50.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.237+0000 7f6332a00700 1 --2- 172.21.15.112:0/4191577888 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55d142fab840 0x55d142fadc60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47c3e8f57ed847ad server_cookie=bbbf5b9b3133db39 in_seq=0 out_seq=0 2024-05-08T05:59:50.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.237+0000 7f631d600700 10 client.15474 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.237+0000 7f631d600700 1 -- 172.21.15.112:0/4191577888 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d142e87400 con 0x55d142fab840 2024-05-08T05:59:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.237+0000 7f631d600700 10 client.15474 handle_command_reply: tid=0 2024-05-08T05:59:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.237+0000 7f63447c4b80 1 client.15474 shutdown 2024-05-08T05:59:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.238+0000 7f63447c4b80 1 -- 172.21.15.112:0/4191577888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 msgr2=0x55d142f48220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.238+0000 7f63447c4b80 1 --2- 172.21.15.112:0/4191577888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 0x55d142f48220 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f632c007b20 tx=0x7f632c007e30 comp rx=0 tx=0).stop 2024-05-08T05:59:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.238+0000 7f63447c4b80 1 -- 172.21.15.112:0/4191577888 shutdown_connections 2024-05-08T05:59:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.238+0000 7f63447c4b80 1 --2- 172.21.15.112:0/4191577888 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55d142fab840 0x55d142fadc60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.238+0000 7f63447c4b80 1 --2- 172.21.15.112:0/4191577888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d142f2d4e0 0x55d142f48220 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.238+0000 7f63447c4b80 1 --2- 172.21.15.112:0/4191577888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d142f278f0 0x55d142f39760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.238+0000 7f63447c4b80 1 --2- 172.21.15.112:0/4191577888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d142f2cbc0 0x55d142f47ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.238+0000 7f63447c4b80 1 -- 172.21.15.112:0/4191577888 >> 172.21.15.112:0/4191577888 conn(0x55d142e9f6d0 msgr2=0x55d142f2e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.238+0000 7f631d600700 0 client.15474 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T05:59:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.238+0000 7f63447c4b80 1 -- 172.21.15.112:0/4191577888 shutdown_connections 2024-05-08T05:59:50.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.238+0000 7f63447c4b80 1 -- 172.21.15.112:0/4191577888 wait complete. 2024-05-08T05:59:50.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.238+0000 7f63447c4b80 20 client.15474 trim_cache size 0 max 16384 2024-05-08T05:59:50.239 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T05:59:50.239 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-08T05:59:50.239 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-08T05:59:50.239 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T05:59:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.239+0000 7f6339e00700 1 -- 172.21.15.112:0/3326038487 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f63180b2900 msgr2=0x7f63180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.239+0000 7f6339e00700 1 --2- 172.21.15.112:0/3326038487 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f63180b2900 0x7f63180b4dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f63341039d0 tx=0x7f6320009260 comp rx=0 tx=0).stop 2024-05-08T05:59:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.239+0000 7f6339e00700 1 -- 172.21.15.112:0/3326038487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633406b1b0 msgr2=0x7f6334104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.239+0000 7f6339e00700 1 --2- 172.21.15.112:0/3326038487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633406b1b0 0x7f6334104760 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f63240098e0 tx=0x7f63240057c0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.240+0000 7f6339e00700 1 -- 172.21.15.112:0/3326038487 shutdown_connections 2024-05-08T05:59:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.240+0000 7f6339e00700 1 --2- 172.21.15.112:0/3326038487 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f63180b2900 0x7f63180b4dc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.240+0000 7f6339e00700 1 --2- 172.21.15.112:0/3326038487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f633410ff50 0x7f633410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.240+0000 7f6339e00700 1 --2- 172.21.15.112:0/3326038487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633406b1b0 0x7f6334104760 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.240+0000 7f6339e00700 1 --2- 172.21.15.112:0/3326038487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633406bad0 0x7f6334104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.240+0000 7f6339e00700 1 -- 172.21.15.112:0/3326038487 >> 172.21.15.112:0/3326038487 conn(0x7f6334079350 msgr2=0x7f6334111460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.240+0000 7f6339e00700 1 -- 172.21.15.112:0/3326038487 shutdown_connections 2024-05-08T05:59:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:50.240+0000 7f6339e00700 1 -- 172.21.15.112:0/3326038487 wait complete. 2024-05-08T05:59:51.027 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:50 smithi112 ceph-mon[30003]: pgmap v577: 97 pgs: 97 active+clean; 87 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 305 op/s 2024-05-08T05:59:51.188 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:59:51.188 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:59:51.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:50 smithi162 ceph-mon[34800]: pgmap v577: 97 pgs: 97 active+clean; 87 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 305 op/s 2024-05-08T05:59:51.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:50 smithi150 ceph-mon[34315]: pgmap v577: 97 pgs: 97 active+clean; 87 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 305 op/s 2024-05-08T05:59:51.662 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:59:53.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:52 smithi112 ceph-mon[30003]: pgmap v578: 97 pgs: 97 active+clean; 87 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.4 MiB/s wr, 261 op/s 2024-05-08T05:59:53.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:52 smithi162 ceph-mon[34800]: pgmap v578: 97 pgs: 97 active+clean; 87 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.4 MiB/s wr, 261 op/s 2024-05-08T05:59:53.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:52 smithi150 ceph-mon[34315]: pgmap v578: 97 pgs: 97 active+clean; 87 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.4 MiB/s wr, 261 op/s 2024-05-08T05:59:54.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.176+0000 7feb0c200700 1 -- 172.21.15.112:0/2071469453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb041056e0 msgr2=0x7feb04105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:54.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.176+0000 7feb0c200700 1 --2- 172.21.15.112:0/2071469453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb041056e0 0x7feb04105b60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009990 tx=0x7feaf0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:59:54.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.176+0000 7feb0c200700 1 -- 172.21.15.112:0/2071469453 shutdown_connections 2024-05-08T05:59:54.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.176+0000 7feb0c200700 1 --2- 172.21.15.112:0/2071469453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb041060a0 0x7feb041125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:54.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.176+0000 7feb0c200700 1 --2- 172.21.15.112:0/2071469453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb0410b6e0 0x7feb0410bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:54.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.176+0000 7feb0c200700 1 --2- 172.21.15.112:0/2071469453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb041056e0 0x7feb04105b60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:54.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.176+0000 7feb0c200700 1 -- 172.21.15.112:0/2071469453 >> 172.21.15.112:0/2071469453 conn(0x7feb041011d0 msgr2=0x7feb041035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:54.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.176+0000 7feb0c200700 1 -- 172.21.15.112:0/2071469453 shutdown_connections 2024-05-08T05:59:54.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.176+0000 7feb0c200700 1 -- 172.21.15.112:0/2071469453 wait complete. 2024-05-08T05:59:54.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.177+0000 7feb0c200700 1 Processor -- start 2024-05-08T05:59:54.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.177+0000 7feb0c200700 1 -- start start 2024-05-08T05:59:54.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.178+0000 7feb0c200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb041056e0 0x7feb0419bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:54.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.178+0000 7feb0c200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb041060a0 0x7feb0419c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:54.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.178+0000 7feb0c200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb0410b6e0 0x7feb041a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:54.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.178+0000 7feb0c200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb04115920 con 0x7feb041060a0 2024-05-08T05:59:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.178+0000 7feb0c200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feb041157a0 con 0x7feb0410b6e0 2024-05-08T05:59:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.178+0000 7feb0c200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7feb04115aa0 con 0x7feb041056e0 2024-05-08T05:59:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.178+0000 7feb09800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb041056e0 0x7feb0419bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.178+0000 7feb09800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb041056e0 0x7feb0419bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50758/0 (socket says 172.21.15.112:50758) 2024-05-08T05:59:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.178+0000 7feb09800700 1 -- 172.21.15.112:0/4118019607 learned_addr learned my addr 172.21.15.112:0/4118019607 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.178+0000 7feb0a200700 1 --2- 172.21.15.112:0/4118019607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb0410b6e0 0x7feb041a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.178+0000 7feb08e00700 1 --2- 172.21.15.112:0/4118019607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb041060a0 0x7feb0419c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.179+0000 7feb09800700 1 -- 172.21.15.112:0/4118019607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb0410b6e0 msgr2=0x7feb041a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.179+0000 7feb09800700 1 --2- 172.21.15.112:0/4118019607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb0410b6e0 0x7feb041a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.179+0000 7feb09800700 1 -- 172.21.15.112:0/4118019607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb041060a0 msgr2=0x7feb0419c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.179+0000 7feb09800700 1 --2- 172.21.15.112:0/4118019607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb041060a0 0x7feb0419c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.179+0000 7feb09800700 1 -- 172.21.15.112:0/4118019607 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb041a0de0 con 0x7feb041056e0 2024-05-08T05:59:54.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.179+0000 7feb09800700 1 --2- 172.21.15.112:0/4118019607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb041056e0 0x7feb0419bdd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feaf800e9a0 tx=0x7feaf800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:54.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.179+0000 7feb02000700 1 -- 172.21.15.112:0/4118019607 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf800ca10 con 0x7feb041056e0 2024-05-08T05:59:54.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.179+0000 7feb02000700 1 -- 172.21.15.112:0/4118019607 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf800cbb0 con 0x7feb041056e0 2024-05-08T05:59:54.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.179+0000 7feb0c200700 1 -- 172.21.15.112:0/4118019607 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb041a10d0 con 0x7feb041056e0 2024-05-08T05:59:54.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.179+0000 7feb02000700 1 -- 172.21.15.112:0/4118019607 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8018680 con 0x7feb041056e0 2024-05-08T05:59:54.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.179+0000 7feb0c200700 1 -- 172.21.15.112:0/4118019607 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb041a8950 con 0x7feb041056e0 2024-05-08T05:59:54.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.181+0000 7feb0c200700 1 -- 172.21.15.112:0/4118019607 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb0406a190 con 0x7feb041056e0 2024-05-08T05:59:54.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.183+0000 7feb02000700 1 -- 172.21.15.112:0/4118019607 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7feaf8010430 con 0x7feb041056e0 2024-05-08T05:59:54.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.184+0000 7feb02000700 1 --2- 172.21.15.112:0/4118019607 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feaec0b2900 0x7feaec0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:54.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.184+0000 7feb02000700 1 -- 172.21.15.112:0/4118019607 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7feaf80e2320 con 0x7feb041056e0 2024-05-08T05:59:54.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.185+0000 7feb08e00700 1 --2- 172.21.15.112:0/4118019607 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feaec0b2900 0x7feaec0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:54.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.185+0000 7feb08e00700 1 --2- 172.21.15.112:0/4118019607 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feaec0b2900 0x7feaec0b4dc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feb0410ac70 tx=0x7feaf000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:54.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.189+0000 7feb02000700 1 -- 172.21.15.112:0/4118019607 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feaf80a9fb0 con 0x7feb041056e0 2024-05-08T05:59:54.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.518+0000 7feb0c200700 1 -- 172.21.15.112:0/4118019607 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb0402e510 con 0x7feb041056e0 2024-05-08T05:59:54.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.519+0000 7feb02000700 1 -- 172.21.15.112:0/4118019607 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11898 (secure 0 0 0) 0x7feaf80a98b0 con 0x7feb041056e0 2024-05-08T05:59:54.521 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T05:59:54.521 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T05:58:06.021913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24518,24511]},"id":1}]} 2024-05-08T05:59:54.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.523+0000 7feb0c200700 1 -- 172.21.15.112:0/4118019607 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feaec0b2900 msgr2=0x7feaec0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:54.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.523+0000 7feb0c200700 1 --2- 172.21.15.112:0/4118019607 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feaec0b2900 0x7feaec0b4dc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feb0410ac70 tx=0x7feaf000b260 comp rx=0 tx=0).stop 2024-05-08T05:59:54.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.523+0000 7feb0c200700 1 -- 172.21.15.112:0/4118019607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb041056e0 msgr2=0x7feb0419bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:54.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.523+0000 7feb0c200700 1 --2- 172.21.15.112:0/4118019607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb041056e0 0x7feb0419bdd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feaf800e9a0 tx=0x7feaf800ecb0 comp rx=0 tx=0).stop 2024-05-08T05:59:54.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.523+0000 7feb0c200700 1 -- 172.21.15.112:0/4118019607 shutdown_connections 2024-05-08T05:59:54.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.523+0000 7feb0c200700 1 --2- 172.21.15.112:0/4118019607 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feaec0b2900 0x7feaec0b4dc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:54.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.523+0000 7feb0c200700 1 --2- 172.21.15.112:0/4118019607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb0410b6e0 0x7feb041a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:54.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.523+0000 7feb0c200700 1 --2- 172.21.15.112:0/4118019607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb041056e0 0x7feb0419bdd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:54.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.523+0000 7feb0c200700 1 --2- 172.21.15.112:0/4118019607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb041060a0 0x7feb0419c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:54.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.523+0000 7feb0c200700 1 -- 172.21.15.112:0/4118019607 >> 172.21.15.112:0/4118019607 conn(0x7feb041011d0 msgr2=0x7feb04101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:54.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.524+0000 7feb0c200700 1 -- 172.21.15.112:0/4118019607 shutdown_connections 2024-05-08T05:59:54.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:54.524+0000 7feb0c200700 1 -- 172.21.15.112:0/4118019607 wait complete. 2024-05-08T05:59:54.527 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 29 2024-05-08T05:59:55.267 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:54 smithi112 ceph-mon[30003]: pgmap v579: 97 pgs: 97 active+clean; 88 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.5 MiB/s wr, 295 op/s 2024-05-08T05:59:55.267 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:54 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/4118019607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:59:55.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:54 smithi162 ceph-mon[34800]: pgmap v579: 97 pgs: 97 active+clean; 88 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.5 MiB/s wr, 295 op/s 2024-05-08T05:59:55.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:54 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/4118019607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:59:55.338 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-08T05:59:55.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:54 smithi150 ceph-mon[34315]: pgmap v579: 97 pgs: 97 active+clean; 88 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.5 MiB/s wr, 295 op/s 2024-05-08T05:59:55.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:54 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/4118019607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:59:55.810 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:59:56.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:55 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:59:56.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:55 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:59:56.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:55 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T05:59:57.285 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:56 smithi112 ceph-mon[30003]: pgmap v580: 97 pgs: 97 active+clean; 91 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.2 MiB/s wr, 423 op/s 2024-05-08T05:59:57.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:56 smithi162 ceph-mon[34800]: pgmap v580: 97 pgs: 97 active+clean; 91 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.2 MiB/s wr, 423 op/s 2024-05-08T05:59:57.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:56 smithi150 ceph-mon[34315]: pgmap v580: 97 pgs: 97 active+clean; 91 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.2 MiB/s wr, 423 op/s 2024-05-08T05:59:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.150+0000 7fea7ce00700 1 -- 172.21.15.112:0/3683003529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea78104100 msgr2=0x7fea78108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.150+0000 7fea7ce00700 1 --2- 172.21.15.112:0/3683003529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea78104100 0x7fea78108120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fea64009920 tx=0x7fea64009c30 comp rx=0 tx=0).stop 2024-05-08T05:59:58.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.150+0000 7fea7ce00700 1 -- 172.21.15.112:0/3683003529 shutdown_connections 2024-05-08T05:59:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.150+0000 7fea7ce00700 1 --2- 172.21.15.112:0/3683003529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea78108850 0x7fea781103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.150+0000 7fea7ce00700 1 --2- 172.21.15.112:0/3683003529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea78103750 0x7fea78103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.150+0000 7fea7ce00700 1 --2- 172.21.15.112:0/3683003529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea78104100 0x7fea78108120 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.150+0000 7fea7ce00700 1 -- 172.21.15.112:0/3683003529 >> 172.21.15.112:0/3683003529 conn(0x7fea7807a600 msgr2=0x7fea7807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.150+0000 7fea7ce00700 1 -- 172.21.15.112:0/3683003529 shutdown_connections 2024-05-08T05:59:58.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.150+0000 7fea7ce00700 1 -- 172.21.15.112:0/3683003529 wait complete. 2024-05-08T05:59:58.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.151+0000 7fea7ce00700 1 Processor -- start 2024-05-08T05:59:58.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.152+0000 7fea7ce00700 1 -- start start 2024-05-08T05:59:58.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.152+0000 7fea7ce00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea78103750 0x7fea7819be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.152+0000 7fea7ce00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea78104100 0x7fea7819c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.152+0000 7fea7ce00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea78108850 0x7fea781a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.152+0000 7fea7ce00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea78113660 con 0x7fea78103750 2024-05-08T05:59:58.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.152+0000 7fea7ce00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fea781134e0 con 0x7fea78104100 2024-05-08T05:59:58.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.152+0000 7fea7ce00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fea781137e0 con 0x7fea78108850 2024-05-08T05:59:58.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.152+0000 7fea75400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea78104100 0x7fea7819c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.152+0000 7fea75400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea78104100 0x7fea7819c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:52616/0 (socket says 172.21.15.112:52616) 2024-05-08T05:59:58.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.152+0000 7fea75400700 1 -- 172.21.15.112:0/1951361349 learned_addr learned my addr 172.21.15.112:0/1951361349 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:58.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.152+0000 7fea76800700 1 --2- 172.21.15.112:0/1951361349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea78108850 0x7fea781a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.153+0000 7fea75e00700 1 --2- 172.21.15.112:0/1951361349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea78103750 0x7fea7819be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.153+0000 7fea75400700 1 -- 172.21.15.112:0/1951361349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea78108850 msgr2=0x7fea781a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:58.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.153+0000 7fea75400700 1 --2- 172.21.15.112:0/1951361349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea78108850 0x7fea781a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.153+0000 7fea75400700 1 -- 172.21.15.112:0/1951361349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea78103750 msgr2=0x7fea7819be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:58.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.153+0000 7fea75400700 1 --2- 172.21.15.112:0/1951361349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea78103750 0x7fea7819be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.153+0000 7fea75400700 1 -- 172.21.15.112:0/1951361349 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea781a0df0 con 0x7fea78104100 2024-05-08T05:59:58.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.153+0000 7fea75400700 1 --2- 172.21.15.112:0/1951361349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea78104100 0x7fea7819c370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fea64000c00 tx=0x7fea6400f990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:58.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.153+0000 7fea6ea00700 1 -- 172.21.15.112:0/1951361349 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64027070 con 0x7fea78104100 2024-05-08T05:59:58.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.153+0000 7fea6ea00700 1 -- 172.21.15.112:0/1951361349 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea640055f0 con 0x7fea78104100 2024-05-08T05:59:58.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.153+0000 7fea6ea00700 1 -- 172.21.15.112:0/1951361349 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64004930 con 0x7fea78104100 2024-05-08T05:59:58.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.154+0000 7fea7ce00700 1 -- 172.21.15.112:0/1951361349 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea781a1080 con 0x7fea78104100 2024-05-08T05:59:58.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.154+0000 7fea7ce00700 1 -- 172.21.15.112:0/1951361349 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea781a14e0 con 0x7fea78104100 2024-05-08T05:59:58.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.158+0000 7fea6ea00700 1 -- 172.21.15.112:0/1951361349 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fea6400b660 con 0x7fea78104100 2024-05-08T05:59:58.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.159+0000 7fea6ea00700 1 --2- 172.21.15.112:0/1951361349 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fea5c0b29d0 0x7fea5c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.160+0000 7fea87721b80 1 Processor -- start 2024-05-08T05:59:58.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.160+0000 7fea6ea00700 1 -- 172.21.15.112:0/1951361349 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fea640e2b50 con 0x7fea78104100 2024-05-08T05:59:58.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.160+0000 7fea87721b80 1 -- start start 2024-05-08T05:59:58.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.160+0000 7fea75e00700 1 --2- 172.21.15.112:0/1951361349 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fea5c0b29d0 0x7fea5c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.160+0000 7fea87721b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb723440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.160+0000 7fea87721b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb727490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.160+0000 7fea87721b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb727e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.160+0000 7fea87721b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562edb675a20 con 0x562edb723980 2024-05-08T05:59:58.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.160+0000 7fea87721b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562edb613760 con 0x562edb7279d0 2024-05-08T05:59:58.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.160+0000 7fea87721b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562edb6719e0 con 0x562edb723060 2024-05-08T05:59:58.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.161+0000 7fea75e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb723440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.161+0000 7fea75e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb723440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50784/0 (socket says 172.21.15.112:50784) 2024-05-08T05:59:58.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.161+0000 7fea75e00700 1 -- 172.21.15.112:0/3143156761 learned_addr learned my addr 172.21.15.112:0/3143156761 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:58.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.161+0000 7fea76800700 1 --2- 172.21.15.112:0/3143156761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb727e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.161+0000 7fea75400700 1 --2- 172.21.15.112:0/3143156761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb727490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.161+0000 7fea75e00700 1 --2- 172.21.15.112:0/1951361349 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fea5c0b29d0 0x7fea5c0b4e90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fea68009c40 tx=0x7fea68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:58.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.161+0000 7fea75e00700 1 -- 172.21.15.112:0/3143156761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 msgr2=0x562edb727e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:58.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.161+0000 7fea75e00700 1 --2- 172.21.15.112:0/3143156761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb727e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.161+0000 7fea75e00700 1 -- 172.21.15.112:0/3143156761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 msgr2=0x562edb727490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:58.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.161+0000 7fea75e00700 1 --2- 172.21.15.112:0/3143156761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb727490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.161+0000 7fea75e00700 1 -- 172.21.15.112:0/3143156761 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562edb5c28d0 con 0x562edb723060 2024-05-08T05:59:58.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.161+0000 7fea75400700 1 --2- 172.21.15.112:0/3143156761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb727490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:59:58.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.162+0000 7fea75e00700 1 --2- 172.21.15.112:0/3143156761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb723440 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fea68011e50 tx=0x7fea6801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7479355bb15a055 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:58.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.162+0000 7fea63400700 1 -- 172.21.15.112:0/3143156761 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68018040 con 0x562edb723060 2024-05-08T05:59:58.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.162+0000 7fea63400700 1 -- 172.21.15.112:0/3143156761 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea68011e80 con 0x562edb723060 2024-05-08T05:59:58.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.162+0000 7fea63400700 1 -- 172.21.15.112:0/3143156761 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6801e630 con 0x562edb723060 2024-05-08T05:59:58.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.163+0000 7fea87721b80 1 -- 172.21.15.112:0/3143156761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 msgr2=0x562edb723440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:58.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.163+0000 7fea87721b80 1 --2- 172.21.15.112:0/3143156761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb723440 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fea68011e50 tx=0x7fea6801c000 comp rx=0 tx=0).stop 2024-05-08T05:59:58.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.163+0000 7fea87721b80 1 -- 172.21.15.112:0/3143156761 shutdown_connections 2024-05-08T05:59:58.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.163+0000 7fea87721b80 1 --2- 172.21.15.112:0/3143156761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb727e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.163+0000 7fea87721b80 1 --2- 172.21.15.112:0/3143156761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb723440 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.163+0000 7fea87721b80 1 --2- 172.21.15.112:0/3143156761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb727490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.163+0000 7fea87721b80 1 -- 172.21.15.112:0/3143156761 >> 172.21.15.112:0/3143156761 conn(0x562edb651b60 msgr2=0x562edb71d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:58.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.163+0000 7fea87721b80 1 -- 172.21.15.112:0/3143156761 shutdown_connections 2024-05-08T05:59:58.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.163+0000 7fea87721b80 1 -- 172.21.15.112:0/3143156761 wait complete. 2024-05-08T05:59:58.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.164+0000 7fea87721b80 1 -- start start 2024-05-08T05:59:58.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.164+0000 7fea87721b80 1 Processor -- start 2024-05-08T05:59:58.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.165+0000 7fea87721b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb72ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.165+0000 7fea87721b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb72c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.165+0000 7fea87721b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb72c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.165+0000 7fea87721b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562edb72fbe0 con 0x562edb723980 2024-05-08T05:59:58.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.165+0000 7fea87721b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562edb5dbb20 con 0x562edb7279d0 2024-05-08T05:59:58.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.165+0000 7fea87721b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562edb72fd60 con 0x562edb723060 2024-05-08T05:59:58.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.165+0000 7fea87721b80 10 client.0 _ll_register_callbacks cb 0x562edb61cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:59:58.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.165+0000 7fea75e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb72ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.165+0000 7fea75e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb72ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50794/0 (socket says 172.21.15.112:50794) 2024-05-08T05:59:58.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.165+0000 7fea75e00700 1 -- 172.21.15.112:0/3397278702 learned_addr learned my addr 172.21.15.112:0/3397278702 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.166+0000 7fea76800700 1 --2- 172.21.15.112:0/3397278702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb72c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.166+0000 7fea75e00700 1 -- 172.21.15.112:0/3397278702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 msgr2=0x562edb72c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.166+0000 7fea75e00700 1 --2- 172.21.15.112:0/3397278702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb72c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.166+0000 7fea75e00700 1 -- 172.21.15.112:0/3397278702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 msgr2=0x562edb72c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.166+0000 7fea75e00700 1 --2- 172.21.15.112:0/3397278702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb72c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.166+0000 7fea75e00700 1 -- 172.21.15.112:0/3397278702 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562edb63b420 con 0x562edb723060 2024-05-08T05:59:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.167+0000 7fea75e00700 1 --2- 172.21.15.112:0/3397278702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb72ee20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fea68012fd0 tx=0x7fea68005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0fe6330f64dd3f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.167+0000 7fea57e00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T05:59:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.167+0000 7fea57e00700 1 -- 172.21.15.112:0/3397278702 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68018070 con 0x562edb723060 2024-05-08T05:59:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.167+0000 7fea57e00700 1 -- 172.21.15.112:0/3397278702 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fea680048f0 con 0x562edb723060 2024-05-08T05:59:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.167+0000 7fea87721b80 1 -- 172.21.15.112:0/3397278702 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562edb6f00a0 con 0x562edb723060 2024-05-08T05:59:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.168+0000 7fea57e00700 1 -- 172.21.15.112:0/3397278702 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fea68004b30 con 0x562edb723060 2024-05-08T05:59:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.168+0000 7fea87721b80 10 client.25666 fetch_fsmap learned FSMap version 30 2024-05-08T05:59:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.168+0000 7fea87721b80 1 -- 172.21.15.112:0/3397278702 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562edb73a420 con 0x562edb723060 2024-05-08T05:59:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.168+0000 7fea57e00700 1 -- 172.21.15.112:0/3397278702 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562edb6f00a0 con 0x562edb723060 2024-05-08T05:59:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.169+0000 7fea57e00700 1 -- 172.21.15.112:0/3397278702 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fea680304b0 con 0x562edb723060 2024-05-08T05:59:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.169+0000 7fea87721b80 10 client.25666 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:59:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.169+0000 7fea87721b80 10 client.25666 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T05:59:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.169+0000 7fea87721b80 20 client.25666 populate_metadata read hostname 'smithi112' 2024-05-08T05:59:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.169+0000 7fea87721b80 1 --2- 172.21.15.112:0/3397278702 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562edb73c9a0 0x562edb73ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.169+0000 7fea87721b80 4 client.25666 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:59:58.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.169+0000 7fea87721b80 1 -- 172.21.15.112:0/3397278702 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562edb683400 con 0x562edb73c9a0 2024-05-08T05:59:58.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.169+0000 7fea75400700 1 --2- 172.21.15.112:0/3397278702 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562edb73c9a0 0x562edb73ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.170+0000 7fea75400700 1 --2- 172.21.15.112:0/3397278702 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562edb73c9a0 0x562edb73ed90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79f1b25bee2467ec server_cookie=9060958b28747fd in_seq=0 out_seq=0 2024-05-08T05:59:58.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.170+0000 7fea57e00700 10 client.25666 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:59:58.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.171+0000 7fea57e00700 1 -- 172.21.15.112:0/3397278702 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562edb683400 con 0x562edb73c9a0 2024-05-08T05:59:58.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.171+0000 7fea57e00700 10 client.25666 handle_command_reply: tid=0 2024-05-08T05:59:58.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.171+0000 7fea87721b80 1 client.25666 shutdown 2024-05-08T05:59:58.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.172+0000 7fea87721b80 1 -- 172.21.15.112:0/3397278702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 msgr2=0x562edb72ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:58.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.172+0000 7fea87721b80 1 --2- 172.21.15.112:0/3397278702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb72ee20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fea68012fd0 tx=0x7fea68005270 comp rx=0 tx=0).stop 2024-05-08T05:59:58.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.172+0000 7fea87721b80 1 -- 172.21.15.112:0/3397278702 shutdown_connections 2024-05-08T05:59:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.172+0000 7fea87721b80 1 --2- 172.21.15.112:0/3397278702 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562edb73c9a0 0x562edb73ed90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.172+0000 7fea57e00700 0 client.25666 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:59:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.172+0000 7fea87721b80 1 --2- 172.21.15.112:0/3397278702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb72c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.172+0000 7fea87721b80 1 --2- 172.21.15.112:0/3397278702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb72ee20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.172+0000 7fea87721b80 1 --2- 172.21.15.112:0/3397278702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb72c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.172+0000 7fea87721b80 1 -- 172.21.15.112:0/3397278702 >> 172.21.15.112:0/3397278702 conn(0x562edb651b60 msgr2=0x562edb7182b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.174+0000 7fea87721b80 1 -- 172.21.15.112:0/3397278702 shutdown_connections 2024-05-08T05:59:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.174+0000 7fea87721b80 1 -- 172.21.15.112:0/3397278702 wait complete. 2024-05-08T05:59:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.174+0000 7fea87721b80 20 client.25666 trim_cache size 0 max 16384 2024-05-08T05:59:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.192+0000 7fea87721b80 1 Processor -- start 2024-05-08T05:59:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.192+0000 7fea87721b80 1 -- start start 2024-05-08T05:59:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.193+0000 7fea87721b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb72b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.193+0000 7fea87721b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb744330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.193+0000 7fea87721b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb744870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.193+0000 7fea87721b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea781b25a0 con 0x562edb723980 2024-05-08T05:59:58.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.193+0000 7fea87721b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fea781afdd0 con 0x562edb7279d0 2024-05-08T05:59:58.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.193+0000 7fea87721b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fea781adc20 con 0x562edb723060 2024-05-08T05:59:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.193+0000 7fea75e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb72b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.193+0000 7fea75e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb72b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50810/0 (socket says 172.21.15.112:50810) 2024-05-08T05:59:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.193+0000 7fea75e00700 1 -- 172.21.15.112:0/3848787912 learned_addr learned my addr 172.21.15.112:0/3848787912 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.193+0000 7fea76800700 1 --2- 172.21.15.112:0/3848787912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb744870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.193+0000 7fea75400700 1 --2- 172.21.15.112:0/3848787912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb744330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.194+0000 7fea75e00700 1 -- 172.21.15.112:0/3848787912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 msgr2=0x562edb744870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.194+0000 7fea75e00700 1 --2- 172.21.15.112:0/3848787912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb744870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.194+0000 7fea75e00700 1 -- 172.21.15.112:0/3848787912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 msgr2=0x562edb744330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.194+0000 7fea75e00700 1 --2- 172.21.15.112:0/3848787912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb744330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.194+0000 7fea75e00700 1 -- 172.21.15.112:0/3848787912 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562edb71e610 con 0x562edb723060 2024-05-08T05:59:58.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.194+0000 7fea75400700 1 --2- 172.21.15.112:0/3848787912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb744330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:59:58.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.194+0000 7fea75e00700 1 --2- 172.21.15.112:0/3848787912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb72b3d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fea68012ac0 tx=0x7fea68012af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8c311651a4d7716 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:58.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.194+0000 7fea62a00700 1 -- 172.21.15.112:0/3848787912 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68018040 con 0x562edb723060 2024-05-08T05:59:58.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.194+0000 7fea62a00700 1 -- 172.21.15.112:0/3848787912 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea68004cd0 con 0x562edb723060 2024-05-08T05:59:58.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.194+0000 7fea62a00700 1 -- 172.21.15.112:0/3848787912 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6801f440 con 0x562edb723060 2024-05-08T05:59:58.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.195+0000 7fea87721b80 1 -- 172.21.15.112:0/3848787912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 msgr2=0x562edb72b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:58.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.195+0000 7fea87721b80 1 --2- 172.21.15.112:0/3848787912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb72b3d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fea68012ac0 tx=0x7fea68012af0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.195+0000 7fea87721b80 1 -- 172.21.15.112:0/3848787912 shutdown_connections 2024-05-08T05:59:58.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.195+0000 7fea87721b80 1 --2- 172.21.15.112:0/3848787912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb744870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.195+0000 7fea87721b80 1 --2- 172.21.15.112:0/3848787912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb72b3d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.195+0000 7fea87721b80 1 --2- 172.21.15.112:0/3848787912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb744330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.195+0000 7fea87721b80 1 -- 172.21.15.112:0/3848787912 >> 172.21.15.112:0/3848787912 conn(0x562edb651b60 msgr2=0x562edb71d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:58.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.195+0000 7fea87721b80 1 -- 172.21.15.112:0/3848787912 shutdown_connections 2024-05-08T05:59:58.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.195+0000 7fea87721b80 1 -- 172.21.15.112:0/3848787912 wait complete. 2024-05-08T05:59:58.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.196+0000 7fea87721b80 1 -- start start 2024-05-08T05:59:58.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.196+0000 7fea87721b80 1 Processor -- start 2024-05-08T05:59:58.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea87721b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb735590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea87721b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb732980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea87721b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb732ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea87721b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea781ae290 con 0x562edb723980 2024-05-08T05:59:58.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea87721b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fea781af700 con 0x562edb7279d0 2024-05-08T05:59:58.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea87721b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fea781ae960 con 0x562edb723060 2024-05-08T05:59:58.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea87721b80 10 client.0 _ll_register_callbacks cb 0x562edb61cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:59:58.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea75400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb732980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea76800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb732ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea75e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb735590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea76800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb732ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:52656/0 (socket says 172.21.15.112:52656) 2024-05-08T05:59:58.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea75400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb732980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53938/0 (socket says 172.21.15.112:53938) 2024-05-08T05:59:58.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea76800700 1 -- 172.21.15.112:0/4237048297 learned_addr learned my addr 172.21.15.112:0/4237048297 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T05:59:58.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea75e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb735590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50826/0 (socket says 172.21.15.112:50826) 2024-05-08T05:59:58.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea76800700 1 -- 172.21.15.112:0/4237048297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 msgr2=0x562edb735590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:58.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea76800700 1 --2- 172.21.15.112:0/4237048297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb735590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea76800700 1 -- 172.21.15.112:0/4237048297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 msgr2=0x562edb732980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:59:58.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea76800700 1 --2- 172.21.15.112:0/4237048297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb732980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.197+0000 7fea76800700 1 -- 172.21.15.112:0/4237048297 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562edb71a3f0 con 0x562edb7279d0 2024-05-08T05:59:58.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.198+0000 7fea75e00700 1 --2- 172.21.15.112:0/4237048297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb735590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:59:58.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.198+0000 7fea76800700 1 --2- 172.21.15.112:0/4237048297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb732ee0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fea70007820 tx=0x7fea70007b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e43698f66811d555 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:58.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.198+0000 7fea57e00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T05:59:58.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.198+0000 7fea57e00700 1 -- 172.21.15.112:0/4237048297 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea700045d0 con 0x562edb7279d0 2024-05-08T05:59:58.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.198+0000 7fea57e00700 1 -- 172.21.15.112:0/4237048297 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fea70004770 con 0x562edb7279d0 2024-05-08T05:59:58.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.198+0000 7fea87721b80 1 -- 172.21.15.112:0/4237048297 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea68004cd0 con 0x562edb7279d0 2024-05-08T05:59:58.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.198+0000 7fea57e00700 1 -- 172.21.15.112:0/4237048297 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fea70021480 con 0x562edb7279d0 2024-05-08T05:59:58.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.198+0000 7fea87721b80 10 client.25634 fetch_fsmap learned FSMap version 30 2024-05-08T05:59:58.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.198+0000 7fea87721b80 1 -- 172.21.15.112:0/4237048297 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562edb7a6d80 con 0x562edb7279d0 2024-05-08T05:59:58.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.199+0000 7fea57e00700 1 -- 172.21.15.112:0/4237048297 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea68004cd0 con 0x562edb7279d0 2024-05-08T05:59:58.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.199+0000 7fea57e00700 1 -- 172.21.15.112:0/4237048297 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fea7002a070 con 0x562edb7279d0 2024-05-08T05:59:58.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.199+0000 7fea87721b80 10 client.25634 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:59:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.199+0000 7fea87721b80 10 client.25634 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T05:59:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.199+0000 7fea87721b80 20 client.25634 populate_metadata read hostname 'smithi112' 2024-05-08T05:59:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.199+0000 7fea87721b80 1 --2- 172.21.15.112:0/4237048297 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562edb73c9a0 0x562edb7a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.199+0000 7fea87721b80 4 client.25634 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T05:59:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.199+0000 7fea87721b80 1 -- 172.21.15.112:0/4237048297 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562edb67d4a0 con 0x562edb73c9a0 2024-05-08T05:59:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.200+0000 7fea75e00700 1 --2- 172.21.15.112:0/4237048297 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562edb73c9a0 0x562edb7a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.200+0000 7fea57e00700 10 client.25634 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T05:59:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.200+0000 7fea75e00700 1 --2- 172.21.15.112:0/4237048297 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562edb73c9a0 0x562edb7a94b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d23b82f4b93aa1f5 server_cookie=58971465eb1d78f4 in_seq=0 out_seq=0 2024-05-08T05:59:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.200+0000 7fea57e00700 1 -- 172.21.15.112:0/4237048297 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562edb67d4a0 con 0x562edb73c9a0 2024-05-08T05:59:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.200+0000 7fea57e00700 10 client.25634 handle_command_reply: tid=0 2024-05-08T05:59:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.200+0000 7fea87721b80 1 client.25634 shutdown 2024-05-08T05:59:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.200+0000 7fea87721b80 1 -- 172.21.15.112:0/4237048297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 msgr2=0x562edb732ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.200+0000 7fea87721b80 1 --2- 172.21.15.112:0/4237048297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb732ee0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fea70007820 tx=0x7fea70007b30 comp rx=0 tx=0).stop 2024-05-08T05:59:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.201+0000 7fea87721b80 1 -- 172.21.15.112:0/4237048297 shutdown_connections 2024-05-08T05:59:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.201+0000 7fea87721b80 1 --2- 172.21.15.112:0/4237048297 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x562edb73c9a0 0x562edb7a94b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.201+0000 7fea87721b80 1 --2- 172.21.15.112:0/4237048297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562edb7279d0 0x562edb732ee0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.201+0000 7fea87721b80 1 --2- 172.21.15.112:0/4237048297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562edb723060 0x562edb735590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.201+0000 7fea87721b80 1 --2- 172.21.15.112:0/4237048297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562edb723980 0x562edb732980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.201+0000 7fea57e00700 0 client.25634 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T05:59:58.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.201+0000 7fea87721b80 1 -- 172.21.15.112:0/4237048297 >> 172.21.15.112:0/4237048297 conn(0x562edb651b60 msgr2=0x562edb717a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:58.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.201+0000 7fea87721b80 1 -- 172.21.15.112:0/4237048297 shutdown_connections 2024-05-08T05:59:58.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.201+0000 7fea87721b80 1 -- 172.21.15.112:0/4237048297 wait complete. 2024-05-08T05:59:58.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.201+0000 7fea87721b80 20 client.25634 trim_cache size 0 max 16384 2024-05-08T05:59:58.204 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T05:59:58.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.202+0000 7fea7ce00700 1 -- 172.21.15.112:0/1951361349 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fea5c0b29d0 msgr2=0x7fea5c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:58.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.202+0000 7fea7ce00700 1 --2- 172.21.15.112:0/1951361349 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fea5c0b29d0 0x7fea5c0b4e90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fea68009c40 tx=0x7fea68009260 comp rx=0 tx=0).stop 2024-05-08T05:59:58.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.202+0000 7fea7ce00700 1 -- 172.21.15.112:0/1951361349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea78104100 msgr2=0x7fea7819c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:58.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.202+0000 7fea7ce00700 1 --2- 172.21.15.112:0/1951361349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea78104100 0x7fea7819c370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fea64000c00 tx=0x7fea6400f990 comp rx=0 tx=0).stop 2024-05-08T05:59:58.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.202+0000 7fea7ce00700 1 -- 172.21.15.112:0/1951361349 shutdown_connections 2024-05-08T05:59:58.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.202+0000 7fea7ce00700 1 --2- 172.21.15.112:0/1951361349 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fea5c0b29d0 0x7fea5c0b4e90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.202+0000 7fea7ce00700 1 --2- 172.21.15.112:0/1951361349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea78108850 0x7fea781a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.202+0000 7fea7ce00700 1 --2- 172.21.15.112:0/1951361349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea78103750 0x7fea7819be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.202+0000 7fea7ce00700 1 --2- 172.21.15.112:0/1951361349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea78104100 0x7fea7819c370 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:58.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.202+0000 7fea7ce00700 1 -- 172.21.15.112:0/1951361349 >> 172.21.15.112:0/1951361349 conn(0x7fea7807a600 msgr2=0x7fea78101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:58.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.202+0000 7fea7ce00700 1 -- 172.21.15.112:0/1951361349 shutdown_connections 2024-05-08T05:59:58.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T05:59:58.202+0000 7fea7ce00700 1 -- 172.21.15.112:0/1951361349 wait complete. 2024-05-08T05:59:59.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 05:59:58 smithi112 ceph-mon[30003]: pgmap v581: 97 pgs: 97 active+clean; 91 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 357 op/s 2024-05-08T05:59:59.218 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-08T05:59:59.234 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 05:59:58 smithi150 ceph-mon[34315]: pgmap v581: 97 pgs: 97 active+clean; 91 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 357 op/s 2024-05-08T05:59:59.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 05:59:58 smithi162 ceph-mon[34800]: pgmap v581: 97 pgs: 97 active+clean; 91 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 357 op/s 2024-05-08T05:59:59.683 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:00:01.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:00 smithi112 ceph-mon[30003]: pgmap v582: 97 pgs: 97 active+clean; 92 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.4 MiB/s wr, 437 op/s 2024-05-08T06:00:01.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:00 smithi112 ceph-mon[30003]: overall HEALTH_OK 2024-05-08T06:00:01.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:00 smithi162 ceph-mon[34800]: pgmap v582: 97 pgs: 97 active+clean; 92 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.4 MiB/s wr, 437 op/s 2024-05-08T06:00:01.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:00 smithi162 ceph-mon[34800]: overall HEALTH_OK 2024-05-08T06:00:01.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:00 smithi150 ceph-mon[34315]: pgmap v582: 97 pgs: 97 active+clean; 92 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.4 MiB/s wr, 437 op/s 2024-05-08T06:00:01.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:00 smithi150 ceph-mon[34315]: overall HEALTH_OK 2024-05-08T06:00:03.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.088+0000 7ff5d5a00700 1 -- 172.21.15.112:0/2844253345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d0075840 msgr2=0x7ff5d0075c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:03.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.088+0000 7ff5d5a00700 1 --2- 172.21.15.112:0/2844253345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d0075840 0x7ff5d0075c20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009990 tx=0x7ff5c0009ca0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.089+0000 7ff5d5a00700 1 -- 172.21.15.112:0/2844253345 shutdown_connections 2024-05-08T06:00:03.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.089+0000 7ff5d5a00700 1 --2- 172.21.15.112:0/2844253345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5d01143e0 0x7ff5d01167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.089+0000 7ff5d5a00700 1 --2- 172.21.15.112:0/2844253345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d0075840 0x7ff5d0075c20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.089+0000 7ff5d5a00700 1 --2- 172.21.15.112:0/2844253345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0076160 0x7ff5d0113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.089+0000 7ff5d5a00700 1 -- 172.21.15.112:0/2844253345 >> 172.21.15.112:0/2844253345 conn(0x7ff5d00fef20 msgr2=0x7ff5d0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:03.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.089+0000 7ff5d5a00700 1 -- 172.21.15.112:0/2844253345 shutdown_connections 2024-05-08T06:00:03.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.089+0000 7ff5d5a00700 1 -- 172.21.15.112:0/2844253345 wait complete. 2024-05-08T06:00:03.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.090+0000 7ff5d5a00700 1 Processor -- start 2024-05-08T06:00:03.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.090+0000 7ff5d5a00700 1 -- start start 2024-05-08T06:00:03.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.090+0000 7ff5d5a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5d0075840 0x7ff5d01a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.090+0000 7ff5d5a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0076160 0x7ff5d01a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.090+0000 7ff5d5a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d01143e0 0x7ff5d01a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.090+0000 7ff5d5a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0119c10 con 0x7ff5d0076160 2024-05-08T06:00:03.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.090+0000 7ff5d5a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0119a90 con 0x7ff5d01143e0 2024-05-08T06:00:03.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.090+0000 7ff5d5a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0119d90 con 0x7ff5d0075840 2024-05-08T06:00:03.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.091+0000 7ff5d5000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d01143e0 0x7ff5d01a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.091+0000 7ff5cfe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5d0075840 0x7ff5d01a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.091+0000 7ff5d5000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d01143e0 0x7ff5d01a49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:52678/0 (socket says 172.21.15.112:52678) 2024-05-08T06:00:03.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.091+0000 7ff5d5000700 1 -- 172.21.15.112:0/1233839935 learned_addr learned my addr 172.21.15.112:0/1233839935 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:03.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.091+0000 7ff5cfe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5d0075840 0x7ff5d01a0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50834/0 (socket says 172.21.15.112:50834) 2024-05-08T06:00:03.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.091+0000 7ff5cf400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0076160 0x7ff5d01a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.091+0000 7ff5d5000700 1 -- 172.21.15.112:0/1233839935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5d0075840 msgr2=0x7ff5d01a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:03.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.091+0000 7ff5d5000700 1 --2- 172.21.15.112:0/1233839935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5d0075840 0x7ff5d01a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.091+0000 7ff5d5000700 1 -- 172.21.15.112:0/1233839935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0076160 msgr2=0x7ff5d01a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:03.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.091+0000 7ff5d5000700 1 --2- 172.21.15.112:0/1233839935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0076160 0x7ff5d01a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.091+0000 7ff5d5000700 1 -- 172.21.15.112:0/1233839935 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d01a5110 con 0x7ff5d01143e0 2024-05-08T06:00:03.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.092+0000 7ff5d5000700 1 --2- 172.21.15.112:0/1233839935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d01143e0 0x7ff5d01a49d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800ef90 tx=0x7ff5c800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:03.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.092+0000 7ff5ccc00700 1 -- 172.21.15.112:0/1233839935 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80092e0 con 0x7ff5d01143e0 2024-05-08T06:00:03.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.092+0000 7ff5d5a00700 1 -- 172.21.15.112:0/1233839935 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d01a53a0 con 0x7ff5d01143e0 2024-05-08T06:00:03.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.092+0000 7ff5ccc00700 1 -- 172.21.15.112:0/1233839935 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c8004bb0 con 0x7ff5d01143e0 2024-05-08T06:00:03.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.092+0000 7ff5ccc00700 1 -- 172.21.15.112:0/1233839935 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800ce60 con 0x7ff5d01143e0 2024-05-08T06:00:03.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.092+0000 7ff5d5a00700 1 -- 172.21.15.112:0/1233839935 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d01acc80 con 0x7ff5d01143e0 2024-05-08T06:00:03.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.097+0000 7ff5ccc00700 1 -- 172.21.15.112:0/1233839935 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff5c801b030 con 0x7ff5d01143e0 2024-05-08T06:00:03.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.098+0000 7ff5e1942b80 1 Processor -- start 2024-05-08T06:00:03.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.098+0000 7ff5e1942b80 1 -- start start 2024-05-08T06:00:03.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.098+0000 7ff5e1942b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e5da0 0x55d6858e6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.098+0000 7ff5e1942b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e66c0 0x55d6858e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.098+0000 7ff5e1942b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e0a50 0x55d6858e0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.098+0000 7ff5e1942b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d685832a20 con 0x55d6858e66c0 2024-05-08T06:00:03.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.098+0000 7ff5e1942b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d6857d0760 con 0x55d6858e5da0 2024-05-08T06:00:03.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.098+0000 7ff5e1942b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d68582e9e0 con 0x55d6858e0a50 2024-05-08T06:00:03.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.099+0000 7ff5cf400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e5da0 0x55d6858e6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.099+0000 7ff5cf400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e5da0 0x55d6858e6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:52686/0 (socket says 172.21.15.112:52686) 2024-05-08T06:00:03.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.099+0000 7ff5cf400700 1 -- 172.21.15.112:0/1877894293 learned_addr learned my addr 172.21.15.112:0/1877894293 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:03.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.099+0000 7ff5cf400700 1 --2- 172.21.15.112:0/1877894293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e0a50 0x55d6858e0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.099+0000 7ff5ccc00700 1 --2- 172.21.15.112:0/1233839935 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff5b40b2900 0x7ff5b40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.099+0000 7ff5cfe00700 1 --2- 172.21.15.112:0/1877894293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e66c0 0x55d6858e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.099+0000 7ff5ccc00700 1 -- 172.21.15.112:0/1233839935 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7ff5c80e1b80 con 0x7ff5d01143e0 2024-05-08T06:00:03.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.099+0000 7ff5cf400700 1 -- 172.21.15.112:0/1877894293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e0a50 msgr2=0x55d6858e0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:03.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.099+0000 7ff5cf400700 1 --2- 172.21.15.112:0/1877894293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e0a50 0x55d6858e0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.099+0000 7ff5cf400700 1 -- 172.21.15.112:0/1877894293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e66c0 msgr2=0x55d6858e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.100+0000 7ff5cf400700 1 --2- 172.21.15.112:0/1877894293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e66c0 0x55d6858e0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.100+0000 7ff5cf400700 1 -- 172.21.15.112:0/1877894293 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d68577f8d0 con 0x55d6858e5da0 2024-05-08T06:00:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.100+0000 7ff5d5000700 1 --2- 172.21.15.112:0/1233839935 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff5b40b2900 0x7ff5b40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.100+0000 7ff5cfe00700 1 --2- 172.21.15.112:0/1877894293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e66c0 0x55d6858e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:00:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.100+0000 7ff5cf400700 1 --2- 172.21.15.112:0/1877894293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e5da0 0x55d6858e6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55d6858e1410 tx=0x7ff5bc0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed2c58fa405dad6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:03.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.100+0000 7ff5c5600700 1 -- 172.21.15.112:0/1877894293 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc013e40 con 0x55d6858e5da0 2024-05-08T06:00:03.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.101+0000 7ff5c5600700 1 -- 172.21.15.112:0/1877894293 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5bc0044d0 con 0x55d6858e5da0 2024-05-08T06:00:03.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.101+0000 7ff5c5600700 1 -- 172.21.15.112:0/1877894293 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc005280 con 0x55d6858e5da0 2024-05-08T06:00:03.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.102+0000 7ff5d5000700 1 --2- 172.21.15.112:0/1233839935 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff5b40b2900 0x7ff5b40b4dc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80a8f00 tx=0x7ff5c8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:03.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.102+0000 7ff5e1942b80 1 -- 172.21.15.112:0/1877894293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e5da0 msgr2=0x55d6858e6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:03.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.102+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/1877894293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e5da0 0x55d6858e6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55d6858e1410 tx=0x7ff5bc0058e0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.102+0000 7ff5e1942b80 1 -- 172.21.15.112:0/1877894293 shutdown_connections 2024-05-08T06:00:03.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.102+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/1877894293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e0a50 0x55d6858e0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.102+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/1877894293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e5da0 0x55d6858e6180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.102+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/1877894293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e66c0 0x55d6858e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.102+0000 7ff5e1942b80 1 -- 172.21.15.112:0/1877894293 >> 172.21.15.112:0/1877894293 conn(0x55d6858586d0 msgr2=0x55d6858daca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:03.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.102+0000 7ff5e1942b80 1 -- 172.21.15.112:0/1877894293 shutdown_connections 2024-05-08T06:00:03.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.102+0000 7ff5e1942b80 1 -- 172.21.15.112:0/1877894293 wait complete. 2024-05-08T06:00:03.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.103+0000 7ff5e1942b80 1 -- start start 2024-05-08T06:00:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.103+0000 7ff5e1942b80 1 Processor -- start 2024-05-08T06:00:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.103+0000 7ff5e1942b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e0a50 0x55d6858ec400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.103+0000 7ff5e1942b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e5da0 0x55d6858e97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.103+0000 7ff5e1942b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e66c0 0x55d6858e9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.103+0000 7ff5e1942b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d6858ed150 con 0x55d6858e66c0 2024-05-08T06:00:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.103+0000 7ff5e1942b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d685798b20 con 0x55d6858e0a50 2024-05-08T06:00:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.103+0000 7ff5e1942b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d6858ed2d0 con 0x55d6858e5da0 2024-05-08T06:00:03.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.104+0000 7ff5e1942b80 10 client.0 _ll_register_callbacks cb 0x55d6857d9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:00:03.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.104+0000 7ff5cfe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e5da0 0x55d6858e97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.104+0000 7ff5cf400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e0a50 0x55d6858ec400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.104+0000 7ff5cfe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e5da0 0x55d6858e97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:50860/0 (socket says 172.21.15.112:50860) 2024-05-08T06:00:03.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.104+0000 7ff5cfe00700 1 -- 172.21.15.112:0/3719451604 learned_addr learned my addr 172.21.15.112:0/3719451604 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:03.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.104+0000 7ff5cf400700 1 --2- 172.21.15.112:0/3719451604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e66c0 0x55d6858e9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.104+0000 7ff5cfe00700 1 -- 172.21.15.112:0/3719451604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e0a50 msgr2=0x55d6858ec400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:03.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.104+0000 7ff5cfe00700 1 --2- 172.21.15.112:0/3719451604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e0a50 0x55d6858ec400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.104+0000 7ff5cfe00700 1 -- 172.21.15.112:0/3719451604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e66c0 msgr2=0x55d6858e9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:03.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.104+0000 7ff5cfe00700 1 --2- 172.21.15.112:0/3719451604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e66c0 0x55d6858e9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.104+0000 7ff5cfe00700 1 -- 172.21.15.112:0/3719451604 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6857f8420 con 0x55d6858e5da0 2024-05-08T06:00:03.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.105+0000 7ff5cfe00700 1 --2- 172.21.15.112:0/3719451604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e5da0 0x55d6858e97a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009990 tx=0x7ff5c0005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ec4bd9fb1e821e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:03.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.105+0000 7ff5ba000700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:00:03.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.105+0000 7ff5ba000700 1 -- 172.21.15.112:0/3719451604 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0029060 con 0x55d6858e5da0 2024-05-08T06:00:03.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.105+0000 7ff5ba000700 1 -- 172.21.15.112:0/3719451604 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7ff5c0003680 con 0x55d6858e5da0 2024-05-08T06:00:03.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.105+0000 7ff5e1942b80 1 -- 172.21.15.112:0/3719451604 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5bc0044d0 con 0x55d6858e5da0 2024-05-08T06:00:03.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.105+0000 7ff5ba000700 1 -- 172.21.15.112:0/3719451604 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff5c0003880 con 0x55d6858e5da0 2024-05-08T06:00:03.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.105+0000 7ff5e1942b80 10 client.25684 fetch_fsmap learned FSMap version 30 2024-05-08T06:00:03.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.105+0000 7ff5e1942b80 1 -- 172.21.15.112:0/3719451604 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d6858f7a50 con 0x55d6858e5da0 2024-05-08T06:00:03.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.106+0000 7ff5ba000700 1 -- 172.21.15.112:0/3719451604 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5bc0044d0 con 0x55d6858e5da0 2024-05-08T06:00:03.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.106+0000 7ff5ba000700 1 -- 172.21.15.112:0/3719451604 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff5c000f920 con 0x55d6858e5da0 2024-05-08T06:00:03.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.106+0000 7ff5e1942b80 10 client.25684 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T06:00:03.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.106+0000 7ff5e1942b80 10 client.25684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T06:00:03.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.106+0000 7ff5e1942b80 20 client.25684 populate_metadata read hostname 'smithi112' 2024-05-08T06:00:03.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.106+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/3719451604 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d6858fa0e0 0x55d6858fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.106+0000 7ff5e1942b80 4 client.25684 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:00:03.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.106+0000 7ff5e1942b80 1 -- 172.21.15.112:0/3719451604 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6858ad140 con 0x55d6858fa0e0 2024-05-08T06:00:03.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.107+0000 7ff5cf400700 1 --2- 172.21.15.112:0/3719451604 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d6858fa0e0 0x55d6858fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.107+0000 7ff5cf400700 1 --2- 172.21.15.112:0/3719451604 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d6858fa0e0 0x55d6858fc4d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b289909180c102e2 server_cookie=8e6086790cb92790 in_seq=0 out_seq=0 2024-05-08T06:00:03.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.107+0000 7ff5ba000700 10 client.25684 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T06:00:03.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.108+0000 7ff5ba000700 1 -- 172.21.15.112:0/3719451604 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6858ad140 con 0x55d6858fa0e0 2024-05-08T06:00:03.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.108+0000 7ff5ba000700 10 client.25684 handle_command_reply: tid=0 2024-05-08T06:00:03.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.108+0000 7ff5e1942b80 1 client.25684 shutdown 2024-05-08T06:00:03.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.108+0000 7ff5e1942b80 1 -- 172.21.15.112:0/3719451604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e5da0 msgr2=0x55d6858e97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:03.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.108+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/3719451604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e5da0 0x55d6858e97a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009990 tx=0x7ff5c0005800 comp rx=0 tx=0).stop 2024-05-08T06:00:03.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.108+0000 7ff5e1942b80 1 -- 172.21.15.112:0/3719451604 shutdown_connections 2024-05-08T06:00:03.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.108+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/3719451604 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d6858fa0e0 0x55d6858fc4d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.108+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/3719451604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e66c0 0x55d6858e9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.108+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/3719451604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e0a50 0x55d6858ec400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.108+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/3719451604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e5da0 0x55d6858e97a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.108+0000 7ff5e1942b80 1 -- 172.21.15.112:0/3719451604 >> 172.21.15.112:0/3719451604 conn(0x55d6858586d0 msgr2=0x55d6858d5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:03.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.108+0000 7ff5ba000700 0 client.25684 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T06:00:03.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.109+0000 7ff5e1942b80 1 -- 172.21.15.112:0/3719451604 shutdown_connections 2024-05-08T06:00:03.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.109+0000 7ff5e1942b80 1 -- 172.21.15.112:0/3719451604 wait complete. 2024-05-08T06:00:03.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.109+0000 7ff5e1942b80 20 client.25684 trim_cache size 0 max 16384 2024-05-08T06:00:03.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.126+0000 7ff5e1942b80 1 Processor -- start 2024-05-08T06:00:03.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.127+0000 7ff5e1942b80 1 -- start start 2024-05-08T06:00:03.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.127+0000 7ff5e1942b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e0a50 0x55d6858e6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.127+0000 7ff5e1942b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e5da0 0x55d6858e3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.127+0000 7ff5e1942b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e66c0 0x55d6858f6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.127+0000 7ff5e1942b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b000fd40 con 0x55d6858e0a50 2024-05-08T06:00:03.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.127+0000 7ff5e1942b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b000d570 con 0x55d6858e5da0 2024-05-08T06:00:03.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.127+0000 7ff5e1942b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b000b3c0 con 0x55d6858e66c0 2024-05-08T06:00:03.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.127+0000 7ff5cf400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e0a50 0x55d6858e6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.128+0000 7ff5cf400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e0a50 0x55d6858e6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53968/0 (socket says 172.21.15.112:53968) 2024-05-08T06:00:03.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.128+0000 7ff5cf400700 1 -- 172.21.15.112:0/2877662197 learned_addr learned my addr 172.21.15.112:0/2877662197 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:03.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.128+0000 7ff5cf400700 1 --2- 172.21.15.112:0/2877662197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e66c0 0x55d6858f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.128+0000 7ff5cfe00700 1 --2- 172.21.15.112:0/2877662197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e5da0 0x55d6858e3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.128+0000 7ff5cf400700 1 -- 172.21.15.112:0/2877662197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e5da0 msgr2=0x55d6858e3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:03.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.128+0000 7ff5cf400700 1 --2- 172.21.15.112:0/2877662197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e5da0 0x55d6858e3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.128+0000 7ff5cf400700 1 -- 172.21.15.112:0/2877662197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e0a50 msgr2=0x55d6858e6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:03.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.128+0000 7ff5cf400700 1 --2- 172.21.15.112:0/2877662197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e0a50 0x55d6858e6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.128+0000 7ff5cf400700 1 -- 172.21.15.112:0/2877662197 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6858dc550 con 0x55d6858e66c0 2024-05-08T06:00:03.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.129+0000 7ff5cf400700 1 --2- 172.21.15.112:0/2877662197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e66c0 0x55d6858f6240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc004ee0 tx=0x7ff5bc018040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bbc69d90ea153e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:03.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.129+0000 7ff5c4c00700 1 -- 172.21.15.112:0/2877662197 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc033070 con 0x55d6858e66c0 2024-05-08T06:00:03.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.129+0000 7ff5c4c00700 1 -- 172.21.15.112:0/2877662197 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5bc005430 con 0x55d6858e66c0 2024-05-08T06:00:03.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.129+0000 7ff5c4c00700 1 -- 172.21.15.112:0/2877662197 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc02d3c0 con 0x55d6858e66c0 2024-05-08T06:00:03.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.130+0000 7ff5e1942b80 1 -- 172.21.15.112:0/2877662197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e66c0 msgr2=0x55d6858f6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:03.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.130+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/2877662197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e66c0 0x55d6858f6240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc004ee0 tx=0x7ff5bc018040 comp rx=0 tx=0).stop 2024-05-08T06:00:03.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.130+0000 7ff5e1942b80 1 -- 172.21.15.112:0/2877662197 shutdown_connections 2024-05-08T06:00:03.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.130+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/2877662197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e66c0 0x55d6858f6240 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.130+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/2877662197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e0a50 0x55d6858e6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.130+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/2877662197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e5da0 0x55d6858e3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.130+0000 7ff5e1942b80 1 -- 172.21.15.112:0/2877662197 >> 172.21.15.112:0/2877662197 conn(0x55d6858586d0 msgr2=0x55d6858eae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:03.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.130+0000 7ff5e1942b80 1 -- 172.21.15.112:0/2877662197 shutdown_connections 2024-05-08T06:00:03.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.130+0000 7ff5e1942b80 1 -- 172.21.15.112:0/2877662197 wait complete. 2024-05-08T06:00:03.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.131+0000 7ff5e1942b80 1 -- start start 2024-05-08T06:00:03.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.131+0000 7ff5e1942b80 1 Processor -- start 2024-05-08T06:00:03.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5e1942b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e0a50 0x55d6858f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5e1942b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e5da0 0x55d6858f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5e1942b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e66c0 0x55d685900d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5e1942b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b000ba30 con 0x55d6858e0a50 2024-05-08T06:00:03.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5e1942b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b000cea0 con 0x55d6858e66c0 2024-05-08T06:00:03.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5e1942b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b000c100 con 0x55d6858e5da0 2024-05-08T06:00:03.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5e1942b80 10 client.0 _ll_register_callbacks cb 0x55d6857d9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:00:03.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5cf400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e0a50 0x55d6858f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5cfe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e5da0 0x55d6858f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5cf400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e0a50 0x55d6858f25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53974/0 (socket says 172.21.15.112:53974) 2024-05-08T06:00:03.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5cf400700 1 -- 172.21.15.112:0/570878932 learned_addr learned my addr 172.21.15.112:0/570878932 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:03.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5cf400700 1 --2- 172.21.15.112:0/570878932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e66c0 0x55d685900d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5cfe00700 1 -- 172.21.15.112:0/570878932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e66c0 msgr2=0x55d685900d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:03.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5cfe00700 1 --2- 172.21.15.112:0/570878932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e66c0 0x55d685900d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5cfe00700 1 -- 172.21.15.112:0/570878932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e0a50 msgr2=0x55d6858f25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:03.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.132+0000 7ff5cfe00700 1 --2- 172.21.15.112:0/570878932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e0a50 0x55d6858f25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.133+0000 7ff5cfe00700 1 -- 172.21.15.112:0/570878932 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d685901550 con 0x55d6858e5da0 2024-05-08T06:00:03.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.133+0000 7ff5cfe00700 1 --2- 172.21.15.112:0/570878932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e5da0 0x55d6858f2b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000fd40 tx=0x7ff5c002f640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aba275703f10ff9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:03.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.133+0000 7ff5ba000700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:00:03.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.133+0000 7ff5ba000700 1 -- 172.21.15.112:0/570878932 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0029070 con 0x55d6858e5da0 2024-05-08T06:00:03.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.133+0000 7ff5ba000700 1 -- 172.21.15.112:0/570878932 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7ff5c000b2f0 con 0x55d6858e5da0 2024-05-08T06:00:03.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.133+0000 7ff5e1942b80 1 -- 172.21.15.112:0/570878932 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5bc005430 con 0x55d6858e5da0 2024-05-08T06:00:03.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.134+0000 7ff5ba000700 1 -- 172.21.15.112:0/570878932 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff5c003e030 con 0x55d6858e5da0 2024-05-08T06:00:03.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.134+0000 7ff5e1942b80 10 client.25690 fetch_fsmap learned FSMap version 30 2024-05-08T06:00:03.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.134+0000 7ff5e1942b80 1 -- 172.21.15.112:0/570878932 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d685964400 con 0x55d6858e5da0 2024-05-08T06:00:03.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.134+0000 7ff5ba000700 1 -- 172.21.15.112:0/570878932 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5bc005430 con 0x55d6858e5da0 2024-05-08T06:00:03.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.134+0000 7ff5ba000700 1 -- 172.21.15.112:0/570878932 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff5c002f730 con 0x55d6858e5da0 2024-05-08T06:00:03.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.135+0000 7ff5e1942b80 10 client.25690 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T06:00:03.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.135+0000 7ff5e1942b80 10 client.25690 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T06:00:03.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.135+0000 7ff5e1942b80 20 client.25690 populate_metadata read hostname 'smithi112' 2024-05-08T06:00:03.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.135+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/570878932 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d685964b10 0x55d685966f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:03.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.135+0000 7ff5e1942b80 4 client.25690 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:00:03.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.135+0000 7ff5e1942b80 1 -- 172.21.15.112:0/570878932 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d685840400 con 0x55d685964b10 2024-05-08T06:00:03.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.135+0000 7ff5cf400700 1 --2- 172.21.15.112:0/570878932 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d685964b10 0x55d685966f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:03.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.135+0000 7ff5ba000700 10 client.25690 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T06:00:03.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.135+0000 7ff5cf400700 1 --2- 172.21.15.112:0/570878932 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d685964b10 0x55d685966f30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa3a227414732456 server_cookie=c8c1d8a3b9e7a82d in_seq=0 out_seq=0 2024-05-08T06:00:03.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.136+0000 7ff5ba000700 1 -- 172.21.15.112:0/570878932 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d685840400 con 0x55d685964b10 2024-05-08T06:00:03.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.136+0000 7ff5ba000700 10 client.25690 handle_command_reply: tid=0 2024-05-08T06:00:03.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.136+0000 7ff5e1942b80 1 client.25690 shutdown 2024-05-08T06:00:03.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.136+0000 7ff5e1942b80 1 -- 172.21.15.112:0/570878932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e5da0 msgr2=0x55d6858f2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:03.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.136+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/570878932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e5da0 0x55d6858f2b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000fd40 tx=0x7ff5c002f640 comp rx=0 tx=0).stop 2024-05-08T06:00:03.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.136+0000 7ff5e1942b80 1 -- 172.21.15.112:0/570878932 shutdown_connections 2024-05-08T06:00:03.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.136+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/570878932 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55d685964b10 0x55d685966f30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.136+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/570878932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6858e66c0 0x55d685900d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.136+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/570878932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6858e0a50 0x55d6858f25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.136+0000 7ff5e1942b80 1 --2- 172.21.15.112:0/570878932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d6858e5da0 0x55d6858f2b00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.136+0000 7ff5ba000700 0 client.25690 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T06:00:03.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.136+0000 7ff5e1942b80 1 -- 172.21.15.112:0/570878932 >> 172.21.15.112:0/570878932 conn(0x55d6858586d0 msgr2=0x55d6858eb630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:03.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.136+0000 7ff5e1942b80 1 -- 172.21.15.112:0/570878932 shutdown_connections 2024-05-08T06:00:03.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.136+0000 7ff5e1942b80 1 -- 172.21.15.112:0/570878932 wait complete. 2024-05-08T06:00:03.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.137+0000 7ff5e1942b80 20 client.25690 trim_cache size 0 max 16384 2024-05-08T06:00:03.214 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T06:00:03.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.137+0000 7ff5d5a00700 1 -- 172.21.15.112:0/1233839935 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff5b40b2900 msgr2=0x7ff5b40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:03.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.137+0000 7ff5d5a00700 1 --2- 172.21.15.112:0/1233839935 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff5b40b2900 0x7ff5b40b4dc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80a8f00 tx=0x7ff5c8009040 comp rx=0 tx=0).stop 2024-05-08T06:00:03.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.137+0000 7ff5d5a00700 1 -- 172.21.15.112:0/1233839935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d01143e0 msgr2=0x7ff5d01a49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:03.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.137+0000 7ff5d5a00700 1 --2- 172.21.15.112:0/1233839935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d01143e0 0x7ff5d01a49d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800ef90 tx=0x7ff5c800c3a0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.137+0000 7ff5d5a00700 1 -- 172.21.15.112:0/1233839935 shutdown_connections 2024-05-08T06:00:03.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.137+0000 7ff5d5a00700 1 --2- 172.21.15.112:0/1233839935 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ff5b40b2900 0x7ff5b40b4dc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.137+0000 7ff5d5a00700 1 --2- 172.21.15.112:0/1233839935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d01143e0 0x7ff5d01a49d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.137+0000 7ff5d5a00700 1 --2- 172.21.15.112:0/1233839935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5d0075840 0x7ff5d01a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.138+0000 7ff5d5a00700 1 --2- 172.21.15.112:0/1233839935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0076160 0x7ff5d01a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:03.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.138+0000 7ff5d5a00700 1 -- 172.21.15.112:0/1233839935 >> 172.21.15.112:0/1233839935 conn(0x7ff5d00fef20 msgr2=0x7ff5d0100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:03.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.138+0000 7ff5d5a00700 1 -- 172.21.15.112:0/1233839935 shutdown_connections 2024-05-08T06:00:03.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:03.138+0000 7ff5d5a00700 1 -- 172.21.15.112:0/1233839935 wait complete. 2024-05-08T06:00:03.216 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:02 smithi112 ceph-mon[30003]: pgmap v583: 97 pgs: 97 active+clean; 93 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.0 MiB/s wr, 383 op/s 2024-05-08T06:00:03.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:02 smithi162 ceph-mon[34800]: pgmap v583: 97 pgs: 97 active+clean; 93 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.0 MiB/s wr, 383 op/s 2024-05-08T06:00:03.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:02 smithi150 ceph-mon[34315]: pgmap v583: 97 pgs: 97 active+clean; 93 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.0 MiB/s wr, 383 op/s 2024-05-08T06:00:04.234 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-08T06:00:04.716 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:00:05.192 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:04 smithi112 ceph-mon[30003]: pgmap v584: 97 pgs: 97 active+clean; 93 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.0 MiB/s wr, 380 op/s 2024-05-08T06:00:05.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:04 smithi162 ceph-mon[34800]: pgmap v584: 97 pgs: 97 active+clean; 93 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.0 MiB/s wr, 380 op/s 2024-05-08T06:00:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:04 smithi150 ceph-mon[34315]: pgmap v584: 97 pgs: 97 active+clean; 93 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.0 MiB/s wr, 380 op/s 2024-05-08T06:00:07.295 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:06 smithi112 ceph-mon[30003]: pgmap v585: 97 pgs: 97 active+clean; 93 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.7 MiB/s wr, 493 op/s 2024-05-08T06:00:07.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:06 smithi162 ceph-mon[34800]: pgmap v585: 97 pgs: 97 active+clean; 93 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.7 MiB/s wr, 493 op/s 2024-05-08T06:00:07.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:06 smithi150 ceph-mon[34315]: pgmap v585: 97 pgs: 97 active+clean; 93 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.7 MiB/s wr, 493 op/s 2024-05-08T06:00:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.864+0000 7f2e1d600700 1 -- 172.21.15.112:0/1171168837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e181094c0 msgr2=0x7f2e181098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.864+0000 7f2e1d600700 1 --2- 172.21.15.112:0/1171168837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e181094c0 0x7f2e181098a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2e080099c0 tx=0x7f2e08009cd0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.864+0000 7f2e1d600700 1 -- 172.21.15.112:0/1171168837 shutdown_connections 2024-05-08T06:00:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.864+0000 7f2e1d600700 1 --2- 172.21.15.112:0/1171168837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e1806c3d0 0x7f2e18110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.864+0000 7f2e1d600700 1 --2- 172.21.15.112:0/1171168837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e181094c0 0x7f2e181098a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.864+0000 7f2e1d600700 1 --2- 172.21.15.112:0/1171168837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e1806ba10 0x7f2e1806be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.864+0000 7f2e1d600700 1 -- 172.21.15.112:0/1171168837 >> 172.21.15.112:0/1171168837 conn(0x7f2e180791e0 msgr2=0x7f2e180795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.865+0000 7f2e1d600700 1 -- 172.21.15.112:0/1171168837 shutdown_connections 2024-05-08T06:00:07.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.865+0000 7f2e1d600700 1 -- 172.21.15.112:0/1171168837 wait complete. 2024-05-08T06:00:07.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.866+0000 7f2e1d600700 1 Processor -- start 2024-05-08T06:00:07.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.866+0000 7f2e1d600700 1 -- start start 2024-05-08T06:00:07.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.867+0000 7f2e1d600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e1806ba10 0x7f2e181a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.867+0000 7f2e1d600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e1806c3d0 0x7f2e181a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.867+0000 7f2e1d600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e181094c0 0x7f2e1819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.867+0000 7f2e1d600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181135e0 con 0x7f2e181094c0 2024-05-08T06:00:07.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.867+0000 7f2e1d600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2e18113460 con 0x7f2e1806ba10 2024-05-08T06:00:07.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.867+0000 7f2e1d600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2e18113760 con 0x7f2e1806c3d0 2024-05-08T06:00:07.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.868+0000 7f2e17e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e1806ba10 0x7f2e181a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.868+0000 7f2e17e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e1806ba10 0x7f2e181a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:58398/0 (socket says 172.21.15.112:58398) 2024-05-08T06:00:07.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.868+0000 7f2e17e00700 1 -- 172.21.15.112:0/1129934305 learned_addr learned my addr 172.21.15.112:0/1129934305 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:07.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.868+0000 7f2e1cc00700 1 --2- 172.21.15.112:0/1129934305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e181094c0 0x7f2e1819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.868+0000 7f2e17400700 1 --2- 172.21.15.112:0/1129934305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e1806c3d0 0x7f2e181a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.868+0000 7f2e17e00700 1 -- 172.21.15.112:0/1129934305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e1806c3d0 msgr2=0x7f2e181a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:07.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.869+0000 7f2e17e00700 1 --2- 172.21.15.112:0/1129934305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e1806c3d0 0x7f2e181a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.869+0000 7f2e17e00700 1 -- 172.21.15.112:0/1129934305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e181094c0 msgr2=0x7f2e1819ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:07.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.869+0000 7f2e17e00700 1 --2- 172.21.15.112:0/1129934305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e181094c0 0x7f2e1819ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.869+0000 7f2e17e00700 1 -- 172.21.15.112:0/1129934305 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1819c1d0 con 0x7f2e1806ba10 2024-05-08T06:00:07.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.869+0000 7f2e1cc00700 1 --2- 172.21.15.112:0/1129934305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e181094c0 0x7f2e1819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:00:07.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.869+0000 7f2e17e00700 1 --2- 172.21.15.112:0/1129934305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e1806ba10 0x7f2e181a18b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2e08006010 tx=0x7f2e08005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:07.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.869+0000 7f2e14c00700 1 -- 172.21.15.112:0/1129934305 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08026070 con 0x7f2e1806ba10 2024-05-08T06:00:07.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.869+0000 7f2e14c00700 1 -- 172.21.15.112:0/1129934305 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e08005680 con 0x7f2e1806ba10 2024-05-08T06:00:07.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.869+0000 7f2e14c00700 1 -- 172.21.15.112:0/1129934305 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0800fb20 con 0x7f2e1806ba10 2024-05-08T06:00:07.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.869+0000 7f2e1d600700 1 -- 172.21.15.112:0/1129934305 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1819c460 con 0x7f2e1806ba10 2024-05-08T06:00:07.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.869+0000 7f2e1d600700 1 -- 172.21.15.112:0/1129934305 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e181a8780 con 0x7f2e1806ba10 2024-05-08T06:00:07.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.874+0000 7f2e14c00700 1 -- 172.21.15.112:0/1129934305 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2e08029070 con 0x7f2e1806ba10 2024-05-08T06:00:07.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.876+0000 7f2e295c9b80 1 Processor -- start 2024-05-08T06:00:07.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.876+0000 7f2e295c9b80 1 -- start start 2024-05-08T06:00:07.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.877+0000 7f2e295c9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 0x5617efd49dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.877+0000 7f2e295c9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd4a300 0x5617efd43fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.877+0000 7f2e295c9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd44600 0x5617efd44a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.877+0000 7f2e295c9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5617efc96a20 con 0x5617efd4a300 2024-05-08T06:00:07.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.877+0000 7f2e295c9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5617efc34760 con 0x5617efd44600 2024-05-08T06:00:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.877+0000 7f2e295c9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5617efc929e0 con 0x5617efd499e0 2024-05-08T06:00:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.877+0000 7f2e14c00700 1 --2- 172.21.15.112:0/1129934305 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2dfc0b2900 0x7f2dfc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.877+0000 7f2e1cc00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 0x5617efd49dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.877+0000 7f2e17e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd4a300 0x5617efd43fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.877+0000 7f2e17e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd4a300 0x5617efd43fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45686/0 (socket says 172.21.15.112:45686) 2024-05-08T06:00:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.877+0000 7f2e17e00700 1 -- 172.21.15.112:0/181222749 learned_addr learned my addr 172.21.15.112:0/181222749 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.877+0000 7f2e17400700 1 --2- 172.21.15.112:0/181222749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd44600 0x5617efd44a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.877+0000 7f2e1cc00700 1 --2- 172.21.15.112:0/1129934305 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2dfc0b2900 0x7f2dfc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.878+0000 7f2e17e00700 1 -- 172.21.15.112:0/181222749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 msgr2=0x5617efd49dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.878+0000 7f2e17e00700 1 --2- 172.21.15.112:0/181222749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 0x5617efd49dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.878+0000 7f2e17e00700 1 -- 172.21.15.112:0/181222749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd44600 msgr2=0x5617efd44a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.878+0000 7f2e17e00700 1 --2- 172.21.15.112:0/181222749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd44600 0x5617efd44a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.878+0000 7f2e14c00700 1 -- 172.21.15.112:0/1129934305 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f2e080e2950 con 0x7f2e1806ba10 2024-05-08T06:00:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.878+0000 7f2e17e00700 1 -- 172.21.15.112:0/181222749 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617efbe38d0 con 0x5617efd4a300 2024-05-08T06:00:07.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.878+0000 7f2e17400700 1 --2- 172.21.15.112:0/181222749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd44600 0x5617efd44a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:00:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.879+0000 7f2e1cc00700 1 --2- 172.21.15.112:0/1129934305 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2dfc0b2900 0x7f2dfc0b4dc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000ec40 tx=0x7f2e1000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.879+0000 7f2e17e00700 1 --2- 172.21.15.112:0/181222749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd4a300 0x5617efd43fe0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2e080e0390 tx=0x7f2e08009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e1c94771441f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.879+0000 7f2e0d600700 1 -- 172.21.15.112:0/181222749 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08003e20 con 0x5617efd4a300 2024-05-08T06:00:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.879+0000 7f2e0d600700 1 -- 172.21.15.112:0/181222749 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e080a9ae0 con 0x5617efd4a300 2024-05-08T06:00:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.879+0000 7f2e0d600700 1 -- 172.21.15.112:0/181222749 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e080b2da0 con 0x5617efd4a300 2024-05-08T06:00:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.880+0000 7f2e295c9b80 1 -- 172.21.15.112:0/181222749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd4a300 msgr2=0x5617efd43fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:07.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.880+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/181222749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd4a300 0x5617efd43fe0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2e080e0390 tx=0x7f2e08009550 comp rx=0 tx=0).stop 2024-05-08T06:00:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.880+0000 7f2e295c9b80 1 -- 172.21.15.112:0/181222749 shutdown_connections 2024-05-08T06:00:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.880+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/181222749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd44600 0x5617efd44a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.880+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/181222749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 0x5617efd49dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.880+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/181222749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd4a300 0x5617efd43fe0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.880+0000 7f2e295c9b80 1 -- 172.21.15.112:0/181222749 >> 172.21.15.112:0/181222749 conn(0x5617efc72b60 msgr2=0x5617efd3e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.880+0000 7f2e295c9b80 1 -- 172.21.15.112:0/181222749 shutdown_connections 2024-05-08T06:00:07.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.880+0000 7f2e295c9b80 1 -- 172.21.15.112:0/181222749 wait complete. 2024-05-08T06:00:07.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.881+0000 7f2e295c9b80 1 -- start start 2024-05-08T06:00:07.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.881+0000 7f2e295c9b80 1 Processor -- start 2024-05-08T06:00:07.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.882+0000 7f2e295c9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd44600 0x5617efd4fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.882+0000 7f2e295c9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 0x5617efd50340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.882+0000 7f2e295c9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd4a300 0x5617efd4d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.882+0000 7f2e295c9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5617efd50b70 con 0x5617efd44600 2024-05-08T06:00:07.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.882+0000 7f2e295c9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5617efbfcb20 con 0x5617efd4a300 2024-05-08T06:00:07.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.882+0000 7f2e295c9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5617efd50cf0 con 0x5617efd499e0 2024-05-08T06:00:07.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.882+0000 7f2e17400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 0x5617efd50340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.882+0000 7f2e295c9b80 10 client.0 _ll_register_callbacks cb 0x5617efc3dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:00:07.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.882+0000 7f2e17400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 0x5617efd50340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:36432/0 (socket says 172.21.15.112:36432) 2024-05-08T06:00:07.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.882+0000 7f2e17400700 1 -- 172.21.15.112:0/4062200070 learned_addr learned my addr 172.21.15.112:0/4062200070 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.882+0000 7f2e1cc00700 1 --2- 172.21.15.112:0/4062200070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd4a300 0x5617efd4d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.882+0000 7f2e17e00700 1 --2- 172.21.15.112:0/4062200070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd44600 0x5617efd4fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.883+0000 7f2e17400700 1 -- 172.21.15.112:0/4062200070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd4a300 msgr2=0x5617efd4d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.883+0000 7f2e17400700 1 --2- 172.21.15.112:0/4062200070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd4a300 0x5617efd4d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.883+0000 7f2e17400700 1 -- 172.21.15.112:0/4062200070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd44600 msgr2=0x5617efd4fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.883+0000 7f2e17400700 1 --2- 172.21.15.112:0/4062200070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd44600 0x5617efd4fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.883+0000 7f2e17400700 1 -- 172.21.15.112:0/4062200070 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617efc5c420 con 0x5617efd499e0 2024-05-08T06:00:07.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.884+0000 7f2e17400700 1 --2- 172.21.15.112:0/4062200070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 0x5617efd50340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2e040226e0 tx=0x7f2e040229f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86770581463aefc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:07.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.884+0000 7f2e02000700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:00:07.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.884+0000 7f2e02000700 1 -- 172.21.15.112:0/4062200070 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04030060 con 0x5617efd499e0 2024-05-08T06:00:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.884+0000 7f2e02000700 1 -- 172.21.15.112:0/4062200070 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f2e04004260 con 0x5617efd499e0 2024-05-08T06:00:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.884+0000 7f2e295c9b80 1 -- 172.21.15.112:0/4062200070 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e080a9ae0 con 0x5617efd499e0 2024-05-08T06:00:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.884+0000 7f2e02000700 1 -- 172.21.15.112:0/4062200070 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e04004460 con 0x5617efd499e0 2024-05-08T06:00:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.884+0000 7f2e295c9b80 10 client.25702 fetch_fsmap learned FSMap version 30 2024-05-08T06:00:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.884+0000 7f2e295c9b80 1 -- 172.21.15.112:0/4062200070 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5617efd5a040 con 0x5617efd499e0 2024-05-08T06:00:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.885+0000 7f2e02000700 1 -- 172.21.15.112:0/4062200070 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e080a9ae0 con 0x5617efd499e0 2024-05-08T06:00:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.885+0000 7f2e02000700 1 -- 172.21.15.112:0/4062200070 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2e0400f7f0 con 0x5617efd499e0 2024-05-08T06:00:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.885+0000 7f2e295c9b80 10 client.25702 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T06:00:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.885+0000 7f2e295c9b80 10 client.25702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:00:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.885+0000 7f2e295c9b80 20 client.25702 populate_metadata read hostname 'smithi112' 2024-05-08T06:00:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.885+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/4062200070 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5617efd5c680 0x5617efd5eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.885+0000 7f2e295c9b80 4 client.25702 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:00:07.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.885+0000 7f2e295c9b80 1 -- 172.21.15.112:0/4062200070 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617efd110a0 con 0x5617efd5c680 2024-05-08T06:00:07.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.885+0000 7f2e17e00700 1 --2- 172.21.15.112:0/4062200070 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5617efd5c680 0x5617efd5eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.886+0000 7f2e17e00700 1 --2- 172.21.15.112:0/4062200070 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5617efd5c680 0x5617efd5eb20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78ab73b71dc81d91 server_cookie=13a54207be8f841e in_seq=0 out_seq=0 2024-05-08T06:00:07.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.886+0000 7f2e02000700 10 client.25702 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:07.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.887+0000 7f2e02000700 1 -- 172.21.15.112:0/4062200070 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5617efd110a0 con 0x5617efd5c680 2024-05-08T06:00:07.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.887+0000 7f2e02000700 10 client.25702 handle_command_reply: tid=0 2024-05-08T06:00:07.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.887+0000 7f2e295c9b80 1 client.25702 shutdown 2024-05-08T06:00:07.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.888+0000 7f2e295c9b80 1 -- 172.21.15.112:0/4062200070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 msgr2=0x5617efd50340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:07.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.888+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/4062200070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 0x5617efd50340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2e040226e0 tx=0x7f2e040229f0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.888+0000 7f2e295c9b80 1 -- 172.21.15.112:0/4062200070 shutdown_connections 2024-05-08T06:00:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.888+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/4062200070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd4a300 0x5617efd4d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.888+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/4062200070 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5617efd5c680 0x5617efd5eb20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.888+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/4062200070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd44600 0x5617efd4fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.888+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/4062200070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 0x5617efd50340 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.888+0000 7f2e295c9b80 1 -- 172.21.15.112:0/4062200070 >> 172.21.15.112:0/4062200070 conn(0x5617efc72b60 msgr2=0x5617efd39460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.888+0000 7f2e02000700 0 client.25702 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.888+0000 7f2e295c9b80 1 -- 172.21.15.112:0/4062200070 shutdown_connections 2024-05-08T06:00:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.888+0000 7f2e295c9b80 1 -- 172.21.15.112:0/4062200070 wait complete. 2024-05-08T06:00:07.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.888+0000 7f2e295c9b80 20 client.25702 trim_cache size 0 max 16384 2024-05-08T06:00:07.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.907+0000 7f2e295c9b80 1 Processor -- start 2024-05-08T06:00:07.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.907+0000 7f2e295c9b80 1 -- start start 2024-05-08T06:00:07.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.908+0000 7f2e295c9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd44600 0x5617efd4db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.908+0000 7f2e295c9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 0x5617efd4bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.908+0000 7f2e295c9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd4a300 0x5617efd4bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.908+0000 7f2e295c9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181ae0e0 con 0x5617efd44600 2024-05-08T06:00:07.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.908+0000 7f2e295c9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181ab910 con 0x5617efd4a300 2024-05-08T06:00:07.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.908+0000 7f2e295c9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1810e1c0 con 0x5617efd499e0 2024-05-08T06:00:07.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.908+0000 7f2e17400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 0x5617efd4bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.908+0000 7f2e17400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 0x5617efd4bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:36444/0 (socket says 172.21.15.112:36444) 2024-05-08T06:00:07.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.908+0000 7f2e17400700 1 -- 172.21.15.112:0/3105735015 learned_addr learned my addr 172.21.15.112:0/3105735015 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:07.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.909+0000 7f2e1cc00700 1 --2- 172.21.15.112:0/3105735015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd4a300 0x5617efd4bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.909+0000 7f2e17e00700 1 --2- 172.21.15.112:0/3105735015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd44600 0x5617efd4db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.909+0000 7f2e1cc00700 1 -- 172.21.15.112:0/3105735015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 msgr2=0x5617efd4bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:07.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.909+0000 7f2e1cc00700 1 --2- 172.21.15.112:0/3105735015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 0x5617efd4bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.909+0000 7f2e1cc00700 1 -- 172.21.15.112:0/3105735015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd44600 msgr2=0x5617efd4db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:07.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.909+0000 7f2e1cc00700 1 --2- 172.21.15.112:0/3105735015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd44600 0x5617efd4db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.909+0000 7f2e1cc00700 1 -- 172.21.15.112:0/3105735015 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617efd4e0b0 con 0x5617efd4a300 2024-05-08T06:00:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.910+0000 7f2e1cc00700 1 --2- 172.21.15.112:0/3105735015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd4a300 0x5617efd4bff0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009e10 tx=0x7f2e10018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=955fe1363df86205 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.910+0000 7f2e0cc00700 1 -- 172.21.15.112:0/3105735015 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1001ae50 con 0x5617efd4a300 2024-05-08T06:00:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.910+0000 7f2e0cc00700 1 -- 172.21.15.112:0/3105735015 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e10004bb0 con 0x5617efd4a300 2024-05-08T06:00:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.910+0000 7f2e0cc00700 1 -- 172.21.15.112:0/3105735015 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10007500 con 0x5617efd4a300 2024-05-08T06:00:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.911+0000 7f2e295c9b80 1 -- 172.21.15.112:0/3105735015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd4a300 msgr2=0x5617efd4bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.911+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/3105735015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd4a300 0x5617efd4bff0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009e10 tx=0x7f2e10018000 comp rx=0 tx=0).stop 2024-05-08T06:00:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.911+0000 7f2e295c9b80 1 -- 172.21.15.112:0/3105735015 shutdown_connections 2024-05-08T06:00:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.911+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/3105735015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd4a300 0x5617efd4bff0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.911+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/3105735015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd44600 0x5617efd4db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.911+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/3105735015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd499e0 0x5617efd4bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.911+0000 7f2e295c9b80 1 -- 172.21.15.112:0/3105735015 >> 172.21.15.112:0/3105735015 conn(0x5617efc72b60 msgr2=0x5617efd4fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.911+0000 7f2e295c9b80 1 -- 172.21.15.112:0/3105735015 shutdown_connections 2024-05-08T06:00:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.912+0000 7f2e295c9b80 1 -- 172.21.15.112:0/3105735015 wait complete. 2024-05-08T06:00:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.912+0000 7f2e295c9b80 1 -- start start 2024-05-08T06:00:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.912+0000 7f2e295c9b80 1 Processor -- start 2024-05-08T06:00:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.913+0000 7f2e295c9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd44600 0x5617efd68b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.913+0000 7f2e295c9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd499e0 0x5617efd69080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.913+0000 7f2e295c9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd4a300 0x5617efd65a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.913+0000 7f2e295c9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181a9dd0 con 0x5617efd499e0 2024-05-08T06:00:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.913+0000 7f2e295c9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181ab240 con 0x5617efd44600 2024-05-08T06:00:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.913+0000 7f2e295c9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181aa4a0 con 0x5617efd4a300 2024-05-08T06:00:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.913+0000 7f2e295c9b80 10 client.0 _ll_register_callbacks cb 0x5617efc3dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:00:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.913+0000 7f2e17e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd44600 0x5617efd68b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.914+0000 7f2e17e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd44600 0x5617efd68b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:58416/0 (socket says 172.21.15.112:58416) 2024-05-08T06:00:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.914+0000 7f2e17e00700 1 -- 172.21.15.112:0/743934034 learned_addr learned my addr 172.21.15.112:0/743934034 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.914+0000 7f2e17400700 1 --2- 172.21.15.112:0/743934034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd499e0 0x5617efd69080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.914+0000 7f2e1cc00700 1 --2- 172.21.15.112:0/743934034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd4a300 0x5617efd65a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.914+0000 7f2e17e00700 1 -- 172.21.15.112:0/743934034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd4a300 msgr2=0x5617efd65a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.914+0000 7f2e17e00700 1 --2- 172.21.15.112:0/743934034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd4a300 0x5617efd65a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.914+0000 7f2e17e00700 1 -- 172.21.15.112:0/743934034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd499e0 msgr2=0x5617efd69080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.914+0000 7f2e17e00700 1 --2- 172.21.15.112:0/743934034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd499e0 0x5617efd69080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.914+0000 7f2e17e00700 1 -- 172.21.15.112:0/743934034 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617efd66240 con 0x5617efd44600 2024-05-08T06:00:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.914+0000 7f2e17400700 1 --2- 172.21.15.112:0/743934034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd499e0 0x5617efd69080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:00:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.914+0000 7f2e17e00700 1 --2- 172.21.15.112:0/743934034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd44600 0x5617efd68b40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2e080a9cd0 tx=0x7f2e08005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76a4e377ac11eaa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.914+0000 7f2e02000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:00:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.914+0000 7f2e02000700 1 -- 172.21.15.112:0/743934034 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e080ae380 con 0x5617efd44600 2024-05-08T06:00:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.915+0000 7f2e02000700 1 -- 172.21.15.112:0/743934034 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f2e0800f460 con 0x5617efd44600 2024-05-08T06:00:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.915+0000 7f2e295c9b80 1 -- 172.21.15.112:0/743934034 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e10004bb0 con 0x5617efd44600 2024-05-08T06:00:07.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.915+0000 7f2e02000700 1 -- 172.21.15.112:0/743934034 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e080a9ae0 con 0x5617efd44600 2024-05-08T06:00:07.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.915+0000 7f2e295c9b80 10 client.25670 fetch_fsmap learned FSMap version 30 2024-05-08T06:00:07.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.915+0000 7f2e295c9b80 1 -- 172.21.15.112:0/743934034 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5617efdc8230 con 0x5617efd44600 2024-05-08T06:00:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.915+0000 7f2e02000700 1 -- 172.21.15.112:0/743934034 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e10004bb0 con 0x5617efd44600 2024-05-08T06:00:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.915+0000 7f2e02000700 1 -- 172.21.15.112:0/743934034 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2e080aa230 con 0x5617efd44600 2024-05-08T06:00:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.915+0000 7f2e295c9b80 10 client.25670 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T06:00:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.915+0000 7f2e295c9b80 10 client.25670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:00:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.915+0000 7f2e295c9b80 20 client.25670 populate_metadata read hostname 'smithi112' 2024-05-08T06:00:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.915+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/743934034 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5617efdc8880 0x5617efdcaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.915+0000 7f2e295c9b80 4 client.25670 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:00:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.915+0000 7f2e295c9b80 1 -- 172.21.15.112:0/743934034 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5617efca4400 con 0x5617efdc8880 2024-05-08T06:00:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.916+0000 7f2e17400700 1 --2- 172.21.15.112:0/743934034 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5617efdc8880 0x5617efdcaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.917+0000 7f2e02000700 10 client.25670 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:07.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.917+0000 7f2e17400700 1 --2- 172.21.15.112:0/743934034 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5617efdc8880 0x5617efdcaca0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a43b611d5bde9f08 server_cookie=e25e517fb2fb87b0 in_seq=0 out_seq=0 2024-05-08T06:00:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.917+0000 7f2e02000700 1 -- 172.21.15.112:0/743934034 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617efca4400 con 0x5617efdc8880 2024-05-08T06:00:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.917+0000 7f2e02000700 10 client.25670 handle_command_reply: tid=0 2024-05-08T06:00:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.918+0000 7f2e295c9b80 1 client.25670 shutdown 2024-05-08T06:00:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.918+0000 7f2e295c9b80 1 -- 172.21.15.112:0/743934034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd44600 msgr2=0x5617efd68b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:07.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.918+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/743934034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd44600 0x5617efd68b40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2e080a9cd0 tx=0x7f2e08005850 comp rx=0 tx=0).stop 2024-05-08T06:00:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.918+0000 7f2e295c9b80 1 -- 172.21.15.112:0/743934034 shutdown_connections 2024-05-08T06:00:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.918+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/743934034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5617efd4a300 0x5617efd65a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.918+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/743934034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5617efd44600 0x5617efd68b40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.918+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/743934034 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5617efdc8880 0x5617efdcaca0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.918+0000 7f2e295c9b80 1 --2- 172.21.15.112:0/743934034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617efd499e0 0x5617efd69080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.918+0000 7f2e295c9b80 1 -- 172.21.15.112:0/743934034 >> 172.21.15.112:0/743934034 conn(0x5617efc72b60 msgr2=0x5617efd4ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:07.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.918+0000 7f2e02000700 0 client.25670 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.919+0000 7f2e295c9b80 1 -- 172.21.15.112:0/743934034 shutdown_connections 2024-05-08T06:00:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.919+0000 7f2e295c9b80 1 -- 172.21.15.112:0/743934034 wait complete. 2024-05-08T06:00:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.920+0000 7f2e295c9b80 20 client.25670 trim_cache size 0 max 16384 2024-05-08T06:00:07.921 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T06:00:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.921+0000 7f2e1d600700 1 -- 172.21.15.112:0/1129934305 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2dfc0b2900 msgr2=0x7f2dfc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:07.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.921+0000 7f2e1d600700 1 --2- 172.21.15.112:0/1129934305 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2dfc0b2900 0x7f2dfc0b4dc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000ec40 tx=0x7f2e1000b040 comp rx=0 tx=0).stop 2024-05-08T06:00:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.921+0000 7f2e1d600700 1 -- 172.21.15.112:0/1129934305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e1806ba10 msgr2=0x7f2e181a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.921+0000 7f2e1d600700 1 --2- 172.21.15.112:0/1129934305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e1806ba10 0x7f2e181a18b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2e08006010 tx=0x7f2e08005280 comp rx=0 tx=0).stop 2024-05-08T06:00:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.922+0000 7f2e1d600700 1 -- 172.21.15.112:0/1129934305 shutdown_connections 2024-05-08T06:00:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.922+0000 7f2e1d600700 1 --2- 172.21.15.112:0/1129934305 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f2dfc0b2900 0x7f2dfc0b4dc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.922+0000 7f2e1d600700 1 --2- 172.21.15.112:0/1129934305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e181094c0 0x7f2e1819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.922+0000 7f2e1d600700 1 --2- 172.21.15.112:0/1129934305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e1806ba10 0x7f2e181a18b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.922+0000 7f2e1d600700 1 --2- 172.21.15.112:0/1129934305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e1806c3d0 0x7f2e181a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.922+0000 7f2e1d600700 1 -- 172.21.15.112:0/1129934305 >> 172.21.15.112:0/1129934305 conn(0x7f2e180791e0 msgr2=0x7f2e1810e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.922+0000 7f2e1d600700 1 -- 172.21.15.112:0/1129934305 shutdown_connections 2024-05-08T06:00:07.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:07.922+0000 7f2e1d600700 1 -- 172.21.15.112:0/1129934305 wait complete. 2024-05-08T06:00:08.258 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x9ff773 2024-05-08T06:00:08.929 INFO:tasks.workunit.client.0.smithi112.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-05-08T06:00:09.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:09 smithi112 ceph-mon[30003]: pgmap v586: 97 pgs: 97 active+clean; 93 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.7 MiB/s wr, 318 op/s 2024-05-08T06:00:09.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:09 smithi162 ceph-mon[34800]: pgmap v586: 97 pgs: 97 active+clean; 93 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.7 MiB/s wr, 318 op/s 2024-05-08T06:00:09.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:09 smithi150 ceph-mon[34315]: pgmap v586: 97 pgs: 97 active+clean; 93 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.7 MiB/s wr, 318 op/s 2024-05-08T06:00:09.931 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T06:00:09.932 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T06:00:10.412 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:00:11.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:11 smithi112 ceph-mon[30003]: pgmap v587: 97 pgs: 97 active+clean; 88 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.5 MiB/s wr, 427 op/s 2024-05-08T06:00:11.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:11 smithi162 ceph-mon[34800]: pgmap v587: 97 pgs: 97 active+clean; 88 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.5 MiB/s wr, 427 op/s 2024-05-08T06:00:11.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:11 smithi150 ceph-mon[34315]: pgmap v587: 97 pgs: 97 active+clean; 88 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.5 MiB/s wr, 427 op/s 2024-05-08T06:00:13.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.079+0000 7ffbf2400700 1 -- 172.21.15.112:0/1686055536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbec109510 msgr2=0x7ffbec1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:13.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.079+0000 7ffbf2400700 1 --2- 172.21.15.112:0/1686055536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbec109510 0x7ffbec1098f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009990 tx=0x7ffbdc009ca0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.079+0000 7ffbf2400700 1 -- 172.21.15.112:0/1686055536 shutdown_connections 2024-05-08T06:00:13.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.079+0000 7ffbf2400700 1 --2- 172.21.15.112:0/1686055536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbec06bb40 0x7ffbec110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.079+0000 7ffbf2400700 1 --2- 172.21.15.112:0/1686055536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbec109510 0x7ffbec1098f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.079+0000 7ffbf2400700 1 --2- 172.21.15.112:0/1686055536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbec06b180 0x7ffbec06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.079+0000 7ffbf2400700 1 -- 172.21.15.112:0/1686055536 >> 172.21.15.112:0/1686055536 conn(0x7ffbec079350 msgr2=0x7ffbec079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:13.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.079+0000 7ffbf2400700 1 -- 172.21.15.112:0/1686055536 shutdown_connections 2024-05-08T06:00:13.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.079+0000 7ffbf2400700 1 -- 172.21.15.112:0/1686055536 wait complete. 2024-05-08T06:00:13.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.080+0000 7ffbf2400700 1 Processor -- start 2024-05-08T06:00:13.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.081+0000 7ffbf2400700 1 -- start start 2024-05-08T06:00:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.081+0000 7ffbf2400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbec06b180 0x7ffbec102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.081+0000 7ffbf2400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbec06bb40 0x7ffbec1029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.081+0000 7ffbf2400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbec109510 0x7ffbec102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.081+0000 7ffbf2400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec113670 con 0x7ffbec06bb40 2024-05-08T06:00:13.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.081+0000 7ffbf2400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec1134f0 con 0x7ffbec109510 2024-05-08T06:00:13.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.081+0000 7ffbf2400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec1137f0 con 0x7ffbec06b180 2024-05-08T06:00:13.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.082+0000 7ffbeb400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbec06b180 0x7ffbec102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.082+0000 7ffbeb400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbec06b180 0x7ffbec102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:36476/0 (socket says 172.21.15.112:36476) 2024-05-08T06:00:13.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.082+0000 7ffbeb400700 1 -- 172.21.15.112:0/2113913143 learned_addr learned my addr 172.21.15.112:0/2113913143 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:13.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.082+0000 7ffbebe00700 1 --2- 172.21.15.112:0/2113913143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbec109510 0x7ffbec102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.082+0000 7ffbeb400700 1 -- 172.21.15.112:0/2113913143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbec109510 msgr2=0x7ffbec102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:13.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.082+0000 7ffbeb400700 1 --2- 172.21.15.112:0/2113913143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbec109510 0x7ffbec102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.082+0000 7ffbeb400700 1 -- 172.21.15.112:0/2113913143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbec06bb40 msgr2=0x7ffbec1029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:13.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.082+0000 7ffbeb400700 1 --2- 172.21.15.112:0/2113913143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbec06bb40 0x7ffbec1029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.082+0000 7ffbeb400700 1 -- 172.21.15.112:0/2113913143 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbec1a80d0 con 0x7ffbec06b180 2024-05-08T06:00:13.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.083+0000 7ffbeb400700 1 --2- 172.21.15.112:0/2113913143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbec06b180 0x7ffbec102470 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc000c00 tx=0x7ffbdc0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:13.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.083+0000 7ffbe3e00700 1 -- 172.21.15.112:0/2113913143 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc027070 con 0x7ffbec06b180 2024-05-08T06:00:13.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.083+0000 7ffbf2400700 1 -- 172.21.15.112:0/2113913143 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbec1a8360 con 0x7ffbec06b180 2024-05-08T06:00:13.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.083+0000 7ffbe3e00700 1 -- 172.21.15.112:0/2113913143 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbdc022470 con 0x7ffbec06b180 2024-05-08T06:00:13.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.083+0000 7ffbe3e00700 1 -- 172.21.15.112:0/2113913143 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc017650 con 0x7ffbec06b180 2024-05-08T06:00:13.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.083+0000 7ffbf2400700 1 -- 172.21.15.112:0/2113913143 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbec1a8810 con 0x7ffbec06b180 2024-05-08T06:00:13.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.087+0000 7ffbe3e00700 1 -- 172.21.15.112:0/2113913143 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ffbdc02a070 con 0x7ffbec06b180 2024-05-08T06:00:13.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.088+0000 7ffbfcdebb80 1 Processor -- start 2024-05-08T06:00:13.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.088+0000 7ffbfcdebb80 1 -- start start 2024-05-08T06:00:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.089+0000 7ffbfcdebb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca7c3c40 0x55efca7c4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.089+0000 7ffbfcdebb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7c4560 0x55efca7be350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.089+0000 7ffbfcdebb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7be970 0x55efca7bedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.089+0000 7ffbfcdebb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55efca710aa0 con 0x55efca7c3c40 2024-05-08T06:00:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.089+0000 7ffbfcdebb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55efca6ae7e0 con 0x55efca7be970 2024-05-08T06:00:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.089+0000 7ffbfcdebb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55efca70ca60 con 0x55efca7c4560 2024-05-08T06:00:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.089+0000 7ffbe3e00700 1 --2- 172.21.15.112:0/2113913143 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffbd00b2880 0x7ffbd00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.089+0000 7ffbeaa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7be970 0x55efca7bedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.089+0000 7ffbeaa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7be970 0x55efca7bedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:58442/0 (socket says 172.21.15.112:58442) 2024-05-08T06:00:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.089+0000 7ffbeaa00700 1 -- 172.21.15.112:0/3701179797 learned_addr learned my addr 172.21.15.112:0/3701179797 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.089+0000 7ffbe3e00700 1 -- 172.21.15.112:0/2113913143 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7ffbdc0e2f80 con 0x7ffbec06b180 2024-05-08T06:00:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.089+0000 7ffbeb400700 1 --2- 172.21.15.112:0/3701179797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7c4560 0x55efca7be350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.089+0000 7ffbebe00700 1 --2- 172.21.15.112:0/2113913143 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffbd00b2880 0x7ffbd00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.090+0000 7ffbeaa00700 1 -- 172.21.15.112:0/3701179797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7c4560 msgr2=0x55efca7be350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.090+0000 7ffbeaa00700 1 --2- 172.21.15.112:0/3701179797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7c4560 0x55efca7be350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.090+0000 7ffbeaa00700 1 -- 172.21.15.112:0/3701179797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca7c3c40 msgr2=0x55efca7c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.090+0000 7ffbeaa00700 1 --2- 172.21.15.112:0/3701179797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca7c3c40 0x55efca7c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.090+0000 7ffbeaa00700 1 -- 172.21.15.112:0/3701179797 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efca65d950 con 0x55efca7be970 2024-05-08T06:00:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.090+0000 7ffbeaa00700 1 --2- 172.21.15.112:0/3701179797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7be970 0x55efca7bedf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009c40 tx=0x7ffbd8009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f68089b507a33c4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.090+0000 7ffbe0c00700 1 -- 172.21.15.112:0/3701179797 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd800eb50 con 0x55efca7be970 2024-05-08T06:00:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.090+0000 7ffbebe00700 1 --2- 172.21.15.112:0/2113913143 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffbd00b2880 0x7ffbd00b4d40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009ec0 tx=0x7ffbe4009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.091+0000 7ffbe0c00700 1 -- 172.21.15.112:0/3701179797 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbd8004ce0 con 0x55efca7be970 2024-05-08T06:00:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.091+0000 7ffbe0c00700 1 -- 172.21.15.112:0/3701179797 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8005670 con 0x55efca7be970 2024-05-08T06:00:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.091+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/3701179797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7be970 msgr2=0x55efca7bedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.091+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/3701179797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7be970 0x55efca7bedf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009c40 tx=0x7ffbd8009f50 comp rx=0 tx=0).stop 2024-05-08T06:00:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.091+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/3701179797 shutdown_connections 2024-05-08T06:00:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.092+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/3701179797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7be970 0x55efca7bedf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.092+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/3701179797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca7c3c40 0x55efca7c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.092+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/3701179797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7c4560 0x55efca7be350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.092+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/3701179797 >> 172.21.15.112:0/3701179797 conn(0x55efca736750 msgr2=0x55efca7b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.092+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/3701179797 shutdown_connections 2024-05-08T06:00:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.092+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/3701179797 wait complete. 2024-05-08T06:00:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.092+0000 7ffbfcdebb80 1 -- start start 2024-05-08T06:00:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.092+0000 7ffbfcdebb80 1 Processor -- start 2024-05-08T06:00:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.093+0000 7ffbfcdebb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.093+0000 7ffbfcdebb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca7c3c40 0x55efca7c7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.093+0000 7ffbfcdebb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c4560 0x55efca7c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.093+0000 7ffbfcdebb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55efca7cafa0 con 0x55efca7c3c40 2024-05-08T06:00:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.093+0000 7ffbfcdebb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55efca676ba0 con 0x55efca7c4560 2024-05-08T06:00:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.093+0000 7ffbfcdebb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55efca7cb120 con 0x55efca7be970 2024-05-08T06:00:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.093+0000 7ffbfcdebb80 10 client.0 _ll_register_callbacks cb 0x55efca6b7cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:00:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.093+0000 7ffbebe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c4560 0x55efca7c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.093+0000 7ffbebe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c4560 0x55efca7c7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:58454/0 (socket says 172.21.15.112:58454) 2024-05-08T06:00:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.093+0000 7ffbebe00700 1 -- 172.21.15.112:0/4276738141 learned_addr learned my addr 172.21.15.112:0/4276738141 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.093+0000 7ffbeb400700 1 --2- 172.21.15.112:0/4276738141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.094+0000 7ffbeaa00700 1 --2- 172.21.15.112:0/4276738141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca7c3c40 0x55efca7c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.094+0000 7ffbeb400700 1 -- 172.21.15.112:0/4276738141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c4560 msgr2=0x55efca7c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.094+0000 7ffbeb400700 1 --2- 172.21.15.112:0/4276738141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c4560 0x55efca7c7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.094+0000 7ffbeb400700 1 -- 172.21.15.112:0/4276738141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca7c3c40 msgr2=0x55efca7c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.094+0000 7ffbeb400700 1 --2- 172.21.15.112:0/4276738141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca7c3c40 0x55efca7c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.094+0000 7ffbeb400700 1 -- 172.21.15.112:0/4276738141 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efca6d64a0 con 0x55efca7be970 2024-05-08T06:00:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.094+0000 7ffbeaa00700 1 --2- 172.21.15.112:0/4276738141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca7c3c40 0x55efca7c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:00:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.094+0000 7ffbeb400700 1 --2- 172.21.15.112:0/4276738141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7ca2d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0e09f0 tx=0x7ffbdc0aa410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=901a1c4604065d97 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.094+0000 7ffbd5600700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:00:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.095+0000 7ffbd5600700 1 -- 172.21.15.112:0/4276738141 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc0aaca0 con 0x55efca7be970 2024-05-08T06:00:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.095+0000 7ffbd5600700 1 -- 172.21.15.112:0/4276738141 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7ffbdc0aa800 con 0x55efca7be970 2024-05-08T06:00:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.095+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/4276738141 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbd8004ce0 con 0x55efca7be970 2024-05-08T06:00:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.095+0000 7ffbd5600700 1 -- 172.21.15.112:0/4276738141 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffbdc0aa5a0 con 0x55efca7be970 2024-05-08T06:00:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.095+0000 7ffbfcdebb80 10 client.25720 fetch_fsmap learned FSMap version 30 2024-05-08T06:00:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.095+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/4276738141 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55efca7cb050 con 0x55efca7be970 2024-05-08T06:00:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.095+0000 7ffbd5600700 1 -- 172.21.15.112:0/4276738141 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbd8004ce0 con 0x55efca7be970 2024-05-08T06:00:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.096+0000 7ffbd5600700 1 -- 172.21.15.112:0/4276738141 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ffbdc0b2f70 con 0x55efca7be970 2024-05-08T06:00:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.097+0000 7ffbfcdebb80 10 client.25720 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T06:00:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.097+0000 7ffbfcdebb80 10 client.25720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:00:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.097+0000 7ffbfcdebb80 20 client.25720 populate_metadata read hostname 'smithi112' 2024-05-08T06:00:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.097+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/4276738141 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55efca7d7de0 0x55efca7da280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.097+0000 7ffbfcdebb80 4 client.25720 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:00:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.097+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/4276738141 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efca78b1c0 con 0x55efca7d7de0 2024-05-08T06:00:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.097+0000 7ffbeaa00700 1 --2- 172.21.15.112:0/4276738141 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55efca7d7de0 0x55efca7da280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.098+0000 7ffbeaa00700 1 --2- 172.21.15.112:0/4276738141 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55efca7d7de0 0x55efca7da280 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e11ad03dca6c84e server_cookie=11d70768b0f2f1a8 in_seq=0 out_seq=0 2024-05-08T06:00:13.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.098+0000 7ffbd5600700 10 client.25720 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.099+0000 7ffbd5600700 1 -- 172.21.15.112:0/4276738141 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55efca78b1c0 con 0x55efca7d7de0 2024-05-08T06:00:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.099+0000 7ffbd5600700 10 client.25720 handle_command_reply: tid=0 2024-05-08T06:00:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.099+0000 7ffbfcdebb80 1 client.25720 shutdown 2024-05-08T06:00:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.099+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/4276738141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 msgr2=0x55efca7ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.099+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/4276738141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7ca2d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0e09f0 tx=0x7ffbdc0aa410 comp rx=0 tx=0).stop 2024-05-08T06:00:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.099+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/4276738141 shutdown_connections 2024-05-08T06:00:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.099+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/4276738141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c4560 0x55efca7c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.099+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/4276738141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7ca2d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.099+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/4276738141 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55efca7d7de0 0x55efca7da280 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.099+0000 7ffbd5600700 0 client.25720 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.099+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/4276738141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca7c3c40 0x55efca7c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.099+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/4276738141 >> 172.21.15.112:0/4276738141 conn(0x55efca736750 msgr2=0x55efca7b8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:13.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.100+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/4276738141 shutdown_connections 2024-05-08T06:00:13.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.100+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/4276738141 wait complete. 2024-05-08T06:00:13.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.100+0000 7ffbfcdebb80 20 client.25720 trim_cache size 0 max 16384 2024-05-08T06:00:13.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.124+0000 7ffbfcdebb80 1 Processor -- start 2024-05-08T06:00:13.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.124+0000 7ffbfcdebb80 1 -- start start 2024-05-08T06:00:13.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.124+0000 7ffbfcdebb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca736750 0x55efca7d4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.124+0000 7ffbfcdebb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.124+0000 7ffbfcdebb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c3c40 0x55efca7c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.124+0000 7ffbfcdebb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec1b25a0 con 0x55efca736750 2024-05-08T06:00:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.124+0000 7ffbfcdebb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec1afdd0 con 0x55efca7c3c40 2024-05-08T06:00:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.124+0000 7ffbfcdebb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec1adc20 con 0x55efca7be970 2024-05-08T06:00:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.125+0000 7ffbebe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c3c40 0x55efca7c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.125+0000 7ffbebe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c3c40 0x55efca7c2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:58466/0 (socket says 172.21.15.112:58466) 2024-05-08T06:00:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.125+0000 7ffbeaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.125+0000 7ffbebe00700 1 -- 172.21.15.112:0/1995548894 learned_addr learned my addr 172.21.15.112:0/1995548894 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.125+0000 7ffbeb400700 1 --2- 172.21.15.112:0/1995548894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca736750 0x55efca7d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.125+0000 7ffbebe00700 1 -- 172.21.15.112:0/1995548894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 msgr2=0x55efca7b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:13.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.125+0000 7ffbebe00700 1 --2- 172.21.15.112:0/1995548894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7b8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.125+0000 7ffbebe00700 1 -- 172.21.15.112:0/1995548894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca736750 msgr2=0x55efca7d4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:13.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.125+0000 7ffbebe00700 1 --2- 172.21.15.112:0/1995548894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca736750 0x55efca7d4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.125+0000 7ffbebe00700 1 -- 172.21.15.112:0/1995548894 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efca7cae80 con 0x55efca7c3c40 2024-05-08T06:00:13.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.125+0000 7ffbeaa00700 1 --2- 172.21.15.112:0/1995548894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:00:13.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.126+0000 7ffbebe00700 1 --2- 172.21.15.112:0/1995548894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c3c40 0x55efca7c2eb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4008c30 tx=0x7ffbe4015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38ea613be3828d06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:13.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.126+0000 7ffbd7e00700 1 -- 172.21.15.112:0/1995548894 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe401a070 con 0x55efca7c3c40 2024-05-08T06:00:13.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.126+0000 7ffbd7e00700 1 -- 172.21.15.112:0/1995548894 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbe4027070 con 0x55efca7c3c40 2024-05-08T06:00:13.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.126+0000 7ffbd7e00700 1 -- 172.21.15.112:0/1995548894 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe40045f0 con 0x55efca7c3c40 2024-05-08T06:00:13.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.127+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/1995548894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c3c40 msgr2=0x55efca7c2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:13.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.127+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/1995548894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c3c40 0x55efca7c2eb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4008c30 tx=0x7ffbe4015000 comp rx=0 tx=0).stop 2024-05-08T06:00:13.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.127+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/1995548894 shutdown_connections 2024-05-08T06:00:13.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.127+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/1995548894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c3c40 0x55efca7c2eb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.127+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/1995548894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca736750 0x55efca7d4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.127+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/1995548894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.127+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/1995548894 >> 172.21.15.112:0/1995548894 conn(0x55efca7c6b10 msgr2=0x55efca7c27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.127+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/1995548894 shutdown_connections 2024-05-08T06:00:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.127+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/1995548894 wait complete. 2024-05-08T06:00:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.128+0000 7ffbfcdebb80 1 -- start start 2024-05-08T06:00:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.128+0000 7ffbfcdebb80 1 Processor -- start 2024-05-08T06:00:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.128+0000 7ffbfcdebb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca736750 0x55efca7cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.128+0000 7ffbfcdebb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.128+0000 7ffbfcdebb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c3c40 0x55efca7cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.128+0000 7ffbfcdebb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec1ae290 con 0x55efca736750 2024-05-08T06:00:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.129+0000 7ffbfcdebb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec1af700 con 0x55efca7c3c40 2024-05-08T06:00:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.129+0000 7ffbfcdebb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec1ae960 con 0x55efca7be970 2024-05-08T06:00:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.129+0000 7ffbfcdebb80 10 client.0 _ll_register_callbacks cb 0x55efca6b7cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:00:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.129+0000 7ffbeaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.129+0000 7ffbeb400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca736750 0x55efca7cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.129+0000 7ffbeaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:36520/0 (socket says 172.21.15.112:36520) 2024-05-08T06:00:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.129+0000 7ffbeaa00700 1 -- 172.21.15.112:0/3736613491 learned_addr learned my addr 172.21.15.112:0/3736613491 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.129+0000 7ffbebe00700 1 --2- 172.21.15.112:0/3736613491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c3c40 0x55efca7cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.129+0000 7ffbeaa00700 1 -- 172.21.15.112:0/3736613491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c3c40 msgr2=0x55efca7cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.129+0000 7ffbeaa00700 1 --2- 172.21.15.112:0/3736613491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c3c40 0x55efca7cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.129+0000 7ffbeaa00700 1 -- 172.21.15.112:0/3736613491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca736750 msgr2=0x55efca7cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.129+0000 7ffbeaa00700 1 --2- 172.21.15.112:0/3736613491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca736750 0x55efca7cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.129+0000 7ffbeaa00700 1 -- 172.21.15.112:0/3736613491 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efca7c4270 con 0x55efca7be970 2024-05-08T06:00:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.129+0000 7ffbeb400700 1 --2- 172.21.15.112:0/3736613491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca736750 0x55efca7cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:00:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.129+0000 7ffbebe00700 1 --2- 172.21.15.112:0/3736613491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c3c40 0x55efca7cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:00:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.130+0000 7ffbeaa00700 1 --2- 172.21.15.112:0/3736613491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7ce080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009c40 tx=0x7ffbd8005110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97a8409defb5eec7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.130+0000 7ffbd5600700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:00:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.130+0000 7ffbd5600700 1 -- 172.21.15.112:0/3736613491 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd801d040 con 0x55efca7be970 2024-05-08T06:00:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.130+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/3736613491 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbe4027070 con 0x55efca7be970 2024-05-08T06:00:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.130+0000 7ffbd5600700 1 -- 172.21.15.112:0/3736613491 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7ffbd8004040 con 0x55efca7be970 2024-05-08T06:00:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.130+0000 7ffbd5600700 1 -- 172.21.15.112:0/3736613491 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffbd8037050 con 0x55efca7be970 2024-05-08T06:00:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.130+0000 7ffbfcdebb80 10 client.25726 fetch_fsmap learned FSMap version 30 2024-05-08T06:00:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.130+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/3736613491 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55efca844140 con 0x55efca7be970 2024-05-08T06:00:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.131+0000 7ffbd5600700 1 -- 172.21.15.112:0/3736613491 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbe4027070 con 0x55efca7be970 2024-05-08T06:00:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.131+0000 7ffbd5600700 1 -- 172.21.15.112:0/3736613491 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ffbd8004950 con 0x55efca7be970 2024-05-08T06:00:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.131+0000 7ffbfcdebb80 10 client.25726 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T06:00:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.131+0000 7ffbfcdebb80 10 client.25726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:00:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.131+0000 7ffbfcdebb80 20 client.25726 populate_metadata read hostname 'smithi112' 2024-05-08T06:00:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.131+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/3736613491 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55efca7c4560 0x55efca846550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.131+0000 7ffbfcdebb80 4 client.25726 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T06:00:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.131+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/3736613491 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55efca71e480 con 0x55efca7c4560 2024-05-08T06:00:13.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.131+0000 7ffbeb400700 1 --2- 172.21.15.112:0/3736613491 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55efca7c4560 0x55efca846550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:13.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.132+0000 7ffbeb400700 1 --2- 172.21.15.112:0/3736613491 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55efca7c4560 0x55efca846550 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eaeb783e66a4b57 server_cookie=11c9b312daf0d064 in_seq=0 out_seq=0 2024-05-08T06:00:13.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.132+0000 7ffbd5600700 10 client.25726 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:13.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.133+0000 7ffbd5600700 1 -- 172.21.15.112:0/3736613491 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55efca71e480 con 0x55efca7c4560 2024-05-08T06:00:13.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.133+0000 7ffbd5600700 10 client.25726 handle_command_reply: tid=0 2024-05-08T06:00:13.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.133+0000 7ffbfcdebb80 1 client.25726 shutdown 2024-05-08T06:00:13.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.134+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/3736613491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 msgr2=0x55efca7ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:13.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.134+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/3736613491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7ce080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009c40 tx=0x7ffbd8005110 comp rx=0 tx=0).stop 2024-05-08T06:00:13.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.134+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/3736613491 shutdown_connections 2024-05-08T06:00:13.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.134+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/3736613491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55efca7c3c40 0x55efca7cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.134+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/3736613491 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55efca7c4560 0x55efca846550 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.134+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/3736613491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efca736750 0x55efca7cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.134+0000 7ffbfcdebb80 1 --2- 172.21.15.112:0/3736613491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55efca7be970 0x55efca7ce080 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.134+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/3736613491 >> 172.21.15.112:0/3736613491 conn(0x55efca7c6b10 msgr2=0x55efca7c3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:13.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.134+0000 7ffbd5600700 0 client.25726 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:13.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.134+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/3736613491 shutdown_connections 2024-05-08T06:00:13.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.134+0000 7ffbfcdebb80 1 -- 172.21.15.112:0/3736613491 wait complete. 2024-05-08T06:00:13.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.135+0000 7ffbfcdebb80 20 client.25726 trim_cache size 0 max 16384 2024-05-08T06:00:13.136 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:00:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-08T06:00:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "3eb3bca1-7bf6-480f-b034-5e2646300dcb", 2024-05-08T06:00:13.137 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-08T06:00:13.137 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:00:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.136+0000 7ffbf2400700 1 -- 172.21.15.112:0/2113913143 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffbd00b2880 msgr2=0x7ffbd00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.136+0000 7ffbf2400700 1 --2- 172.21.15.112:0/2113913143 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffbd00b2880 0x7ffbd00b4d40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009ec0 tx=0x7ffbe4009920 comp rx=0 tx=0).stop 2024-05-08T06:00:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.136+0000 7ffbf2400700 1 -- 172.21.15.112:0/2113913143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbec06b180 msgr2=0x7ffbec102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.136+0000 7ffbf2400700 1 --2- 172.21.15.112:0/2113913143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbec06b180 0x7ffbec102470 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc000c00 tx=0x7ffbdc0057c0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.136+0000 7ffbf2400700 1 -- 172.21.15.112:0/2113913143 shutdown_connections 2024-05-08T06:00:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.136+0000 7ffbf2400700 1 --2- 172.21.15.112:0/2113913143 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffbd00b2880 0x7ffbd00b4d40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.136+0000 7ffbf2400700 1 --2- 172.21.15.112:0/2113913143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbec109510 0x7ffbec102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.136+0000 7ffbf2400700 1 --2- 172.21.15.112:0/2113913143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbec06b180 0x7ffbec102470 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.136+0000 7ffbf2400700 1 --2- 172.21.15.112:0/2113913143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbec06bb40 0x7ffbec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.136+0000 7ffbf2400700 1 -- 172.21.15.112:0/2113913143 >> 172.21.15.112:0/2113913143 conn(0x7ffbec079350 msgr2=0x7ffbec101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.136+0000 7ffbf2400700 1 -- 172.21.15.112:0/2113913143 shutdown_connections 2024-05-08T06:00:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:13.136+0000 7ffbf2400700 1 -- 172.21.15.112:0/2113913143 wait complete. 2024-05-08T06:00:13.158 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:13 smithi112 ceph-mon[30003]: pgmap v588: 97 pgs: 97 active+clean; 89 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 354 op/s 2024-05-08T06:00:13.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:13 smithi162 ceph-mon[34800]: pgmap v588: 97 pgs: 97 active+clean; 89 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 354 op/s 2024-05-08T06:00:13.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:13 smithi150 ceph-mon[34315]: pgmap v588: 97 pgs: 97 active+clean; 89 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 354 op/s 2024-05-08T06:00:14.160 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T06:00:14.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:14 smithi112 ceph-mon[30003]: scrub queued for path: / 2024-05-08T06:00:14.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:14 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:00:14.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:14 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T06:00:14.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:14 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T06:00:14.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:14 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:00:14.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:14 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T06:00:14.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:14 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:00:14.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:14 smithi162 ceph-mon[34800]: scrub queued for path: / 2024-05-08T06:00:14.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:14 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:00:14.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:14 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T06:00:14.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:14 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T06:00:14.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:14 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:00:14.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:14 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T06:00:14.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:14 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:00:14.625 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:00:14.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:14 smithi150 ceph-mon[34315]: scrub queued for path: / 2024-05-08T06:00:14.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:14 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:00:14.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:14 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T06:00:14.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:14 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T06:00:14.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:14 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:00:14.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:14 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T06:00:14.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:14 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:00:15.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:15 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:00:15.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:15 smithi112 ceph-mon[30003]: pgmap v589: 97 pgs: 97 active+clean; 89 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 303 op/s 2024-05-08T06:00:15.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:15 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:00:15.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:15 smithi162 ceph-mon[34800]: pgmap v589: 97 pgs: 97 active+clean; 89 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 303 op/s 2024-05-08T06:00:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:15 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:00:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:15 smithi150 ceph-mon[34315]: pgmap v589: 97 pgs: 97 active+clean; 89 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 303 op/s 2024-05-08T06:00:17.208 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:17 smithi112 ceph-mon[30003]: pgmap v590: 97 pgs: 97 active+clean; 94 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.5 MiB/s wr, 468 op/s 2024-05-08T06:00:17.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.249+0000 7f25f0a00700 1 -- 172.21.15.112:0/2338129934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ec1033d0 msgr2=0x7f25ec103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.249+0000 7f25f0a00700 1 --2- 172.21.15.112:0/2338129934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ec1033d0 0x7f25ec103850 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009990 tx=0x7f25d8009ca0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.249+0000 7f25f0a00700 1 -- 172.21.15.112:0/2338129934 shutdown_connections 2024-05-08T06:00:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.249+0000 7f25f0a00700 1 --2- 172.21.15.112:0/2338129934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec103d90 0x7f25ec1102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.249+0000 7f25f0a00700 1 --2- 172.21.15.112:0/2338129934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25ec109460 0x7f25ec109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.249+0000 7f25f0a00700 1 --2- 172.21.15.112:0/2338129934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ec1033d0 0x7f25ec103850 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.249+0000 7f25f0a00700 1 -- 172.21.15.112:0/2338129934 >> 172.21.15.112:0/2338129934 conn(0x7f25ec0fef20 msgr2=0x7f25ec101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.249+0000 7f25f0a00700 1 -- 172.21.15.112:0/2338129934 shutdown_connections 2024-05-08T06:00:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.249+0000 7f25f0a00700 1 -- 172.21.15.112:0/2338129934 wait complete. 2024-05-08T06:00:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.251+0000 7f25f0a00700 1 Processor -- start 2024-05-08T06:00:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.251+0000 7f25f0a00700 1 -- start start 2024-05-08T06:00:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.251+0000 7f25f0a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec1033d0 0x7f25ec1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.251+0000 7f25f0a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25ec103d90 0x7f25ec1a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.251+0000 7f25f0a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ec109460 0x7f25ec19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.251+0000 7f25f0a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec076220 con 0x7f25ec1033d0 2024-05-08T06:00:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.251+0000 7f25f0a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec0760a0 con 0x7f25ec109460 2024-05-08T06:00:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.251+0000 7f25f0a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec0763a0 con 0x7f25ec103d90 2024-05-08T06:00:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.252+0000 7f25ebe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ec109460 0x7f25ec19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.252+0000 7f25eaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25ec103d90 0x7f25ec1a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.252+0000 7f25eb400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec1033d0 0x7f25ec1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.252+0000 7f25ebe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ec109460 0x7f25ec19bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:58848/0 (socket says 172.21.15.112:58848) 2024-05-08T06:00:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.252+0000 7f25eaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25ec103d90 0x7f25ec1a1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:58292/0 (socket says 172.21.15.112:58292) 2024-05-08T06:00:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.252+0000 7f25ebe00700 1 -- 172.21.15.112:0/969103808 learned_addr learned my addr 172.21.15.112:0/969103808 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.252+0000 7f25ebe00700 1 -- 172.21.15.112:0/969103808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25ec103d90 msgr2=0x7f25ec1a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.252+0000 7f25ebe00700 1 --2- 172.21.15.112:0/969103808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25ec103d90 0x7f25ec1a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.252+0000 7f25ebe00700 1 -- 172.21.15.112:0/969103808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec1033d0 msgr2=0x7f25ec1a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.252+0000 7f25ebe00700 1 --2- 172.21.15.112:0/969103808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec1033d0 0x7f25ec1a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.252+0000 7f25ebe00700 1 -- 172.21.15.112:0/969103808 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec19c340 con 0x7f25ec109460 2024-05-08T06:00:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.252+0000 7f25eaa00700 1 --2- 172.21.15.112:0/969103808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25ec103d90 0x7f25ec1a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:00:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.252+0000 7f25ebe00700 1 --2- 172.21.15.112:0/969103808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ec109460 0x7f25ec19bbb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f25e400ef30 tx=0x7f25e400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.253+0000 7f25e3e00700 1 -- 172.21.15.112:0/969103808 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e40092e0 con 0x7f25ec109460 2024-05-08T06:00:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.253+0000 7f25e3e00700 1 -- 172.21.15.112:0/969103808 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25e4004bb0 con 0x7f25ec109460 2024-05-08T06:00:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.253+0000 7f25e3e00700 1 -- 172.21.15.112:0/969103808 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4004020 con 0x7f25ec109460 2024-05-08T06:00:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.253+0000 7f25f0a00700 1 -- 172.21.15.112:0/969103808 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ec19c5d0 con 0x7f25ec109460 2024-05-08T06:00:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.253+0000 7f25f0a00700 1 -- 172.21.15.112:0/969103808 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ec1a87f0 con 0x7f25ec109460 2024-05-08T06:00:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.256+0000 7f25e3e00700 1 -- 172.21.15.112:0/969103808 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f25e401b030 con 0x7f25ec109460 2024-05-08T06:00:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.258+0000 7f25fc96fb80 1 Processor -- start 2024-05-08T06:00:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.258+0000 7f25fc96fb80 1 -- start start 2024-05-08T06:00:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.258+0000 7f25e3e00700 1 --2- 172.21.15.112:0/969103808 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f25d00b2900 0x7f25d00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.258+0000 7f25fc96fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9db9e0 0x56124a9dbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.258+0000 7f25fc96fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9dc300 0x56124a9d5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.258+0000 7f25fc96fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9d6600 0x56124a9d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.258+0000 7f25fc96fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56124a928a20 con 0x56124a9dc300 2024-05-08T06:00:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.258+0000 7f25fc96fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56124a8c6760 con 0x56124a9d6600 2024-05-08T06:00:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.258+0000 7f25fc96fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56124a9249e0 con 0x56124a9db9e0 2024-05-08T06:00:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.258+0000 7f25e3e00700 1 -- 172.21.15.112:0/969103808 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f25e40e1cc0 con 0x7f25ec109460 2024-05-08T06:00:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.259+0000 7f25eb400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9db9e0 0x56124a9dbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.259+0000 7f25eaa00700 1 --2- 172.21.15.112:0/969103808 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f25d00b2900 0x7f25d00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.259+0000 7f25eb400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9db9e0 0x56124a9dbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:58306/0 (socket says 172.21.15.112:58306) 2024-05-08T06:00:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.259+0000 7f25eb400700 1 -- 172.21.15.112:0/4108735167 learned_addr learned my addr 172.21.15.112:0/4108735167 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.259+0000 7f25eaa00700 1 --2- 172.21.15.112:0/4108735167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9dc300 0x56124a9d5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.259+0000 7f25ebe00700 1 --2- 172.21.15.112:0/4108735167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9d6600 0x56124a9d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.259+0000 7f25eb400700 1 -- 172.21.15.112:0/4108735167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9d6600 msgr2=0x56124a9d6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.259+0000 7f25eb400700 1 --2- 172.21.15.112:0/4108735167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9d6600 0x56124a9d6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.259+0000 7f25eb400700 1 -- 172.21.15.112:0/4108735167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9dc300 msgr2=0x56124a9d5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.259+0000 7f25eb400700 1 --2- 172.21.15.112:0/4108735167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9dc300 0x56124a9d5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.259+0000 7f25eb400700 1 -- 172.21.15.112:0/4108735167 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56124a8758d0 con 0x56124a9db9e0 2024-05-08T06:00:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.259+0000 7f25eb400700 1 --2- 172.21.15.112:0/4108735167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9db9e0 0x56124a9dbdc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009d30 tx=0x7f25dc011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=164af8bd4ef537cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.260+0000 7f25e0c00700 1 -- 172.21.15.112:0/4108735167 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc004230 con 0x56124a9db9e0 2024-05-08T06:00:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.260+0000 7f25e0c00700 1 -- 172.21.15.112:0/4108735167 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25dc0043d0 con 0x56124a9db9e0 2024-05-08T06:00:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.260+0000 7f25e0c00700 1 -- 172.21.15.112:0/4108735167 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc019820 con 0x56124a9db9e0 2024-05-08T06:00:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.260+0000 7f25fc96fb80 1 -- 172.21.15.112:0/4108735167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9db9e0 msgr2=0x56124a9dbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.260+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/4108735167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9db9e0 0x56124a9dbdc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009d30 tx=0x7f25dc011720 comp rx=0 tx=0).stop 2024-05-08T06:00:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.260+0000 7f25eaa00700 1 --2- 172.21.15.112:0/969103808 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f25d00b2900 0x7f25d00b4dc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f25d800b6c0 tx=0x7f25d8009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.261+0000 7f25fc96fb80 1 -- 172.21.15.112:0/4108735167 shutdown_connections 2024-05-08T06:00:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.261+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/4108735167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9d6600 0x56124a9d6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.261+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/4108735167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9db9e0 0x56124a9dbdc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.261+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/4108735167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9dc300 0x56124a9d5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.261+0000 7f25fc96fb80 1 -- 172.21.15.112:0/4108735167 >> 172.21.15.112:0/4108735167 conn(0x56124a904b60 msgr2=0x56124a9d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.261+0000 7f25fc96fb80 1 -- 172.21.15.112:0/4108735167 shutdown_connections 2024-05-08T06:00:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.261+0000 7f25fc96fb80 1 -- 172.21.15.112:0/4108735167 wait complete. 2024-05-08T06:00:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.261+0000 7f25fc96fb80 1 -- start start 2024-05-08T06:00:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.262+0000 7f25fc96fb80 1 Processor -- start 2024-05-08T06:00:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.262+0000 7f25fc96fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9d6600 0x56124a9e1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.262+0000 7f25fc96fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9db9e0 0x56124a9e2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.262+0000 7f25fc96fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9dc300 0x56124a9df3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.262+0000 7f25fc96fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56124a9e2db0 con 0x56124a9dc300 2024-05-08T06:00:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.262+0000 7f25fc96fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56124a88eb20 con 0x56124a9d6600 2024-05-08T06:00:17.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.262+0000 7f25fc96fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56124a9e2f30 con 0x56124a9db9e0 2024-05-08T06:00:17.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.262+0000 7f25fc96fb80 10 client.0 _ll_register_callbacks cb 0x56124a8cfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:00:17.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.262+0000 7f25eb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9d6600 0x56124a9e1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.262+0000 7f25eb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9d6600 0x56124a9e1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:58858/0 (socket says 172.21.15.112:58858) 2024-05-08T06:00:17.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.262+0000 7f25eb400700 1 -- 172.21.15.112:0/2835490464 learned_addr learned my addr 172.21.15.112:0/2835490464 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:17.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.262+0000 7f25eaa00700 1 --2- 172.21.15.112:0/2835490464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9db9e0 0x56124a9e2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.262+0000 7f25ebe00700 1 --2- 172.21.15.112:0/2835490464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9dc300 0x56124a9df3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.263+0000 7f25eb400700 1 -- 172.21.15.112:0/2835490464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9db9e0 msgr2=0x56124a9e2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:17.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.263+0000 7f25eb400700 1 --2- 172.21.15.112:0/2835490464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9db9e0 0x56124a9e2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.263+0000 7f25eb400700 1 -- 172.21.15.112:0/2835490464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9dc300 msgr2=0x56124a9df3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:17.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.263+0000 7f25eb400700 1 --2- 172.21.15.112:0/2835490464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9dc300 0x56124a9df3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.263+0000 7f25eb400700 1 -- 172.21.15.112:0/2835490464 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56124a8ee420 con 0x56124a9d6600 2024-05-08T06:00:17.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.263+0000 7f25eb400700 1 --2- 172.21.15.112:0/2835490464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9d6600 0x56124a9e1e50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f25dc003ed0 tx=0x7f25dc0038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23b9b876dd149db2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.263+0000 7f25d5600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:00:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.263+0000 7f25d5600700 1 -- 172.21.15.112:0/2835490464 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc027020 con 0x56124a9d6600 2024-05-08T06:00:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.263+0000 7f25d5600700 1 -- 172.21.15.112:0/2835490464 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f25dc005250 con 0x56124a9d6600 2024-05-08T06:00:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.263+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2835490464 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25dc0043d0 con 0x56124a9d6600 2024-05-08T06:00:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.264+0000 7f25d5600700 1 -- 172.21.15.112:0/2835490464 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f25dc032050 con 0x56124a9d6600 2024-05-08T06:00:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.264+0000 7f25fc96fb80 10 client.25700 fetch_fsmap learned FSMap version 30 2024-05-08T06:00:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.264+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2835490464 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56124a9ed430 con 0x56124a9d6600 2024-05-08T06:00:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.264+0000 7f25d5600700 1 -- 172.21.15.112:0/2835490464 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25dc0043d0 con 0x56124a9d6600 2024-05-08T06:00:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.264+0000 7f25d5600700 1 -- 172.21.15.112:0/2835490464 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f25dc023460 con 0x56124a9d6600 2024-05-08T06:00:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.264+0000 7f25fc96fb80 10 client.25700 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T06:00:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.264+0000 7f25fc96fb80 10 client.25700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:00:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.264+0000 7f25fc96fb80 20 client.25700 populate_metadata read hostname 'smithi112' 2024-05-08T06:00:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.264+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2835490464 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56124a9efa10 0x56124a9f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.264+0000 7f25fc96fb80 4 client.25700 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:00:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.264+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2835490464 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56124a9a30a0 con 0x56124a9efa10 2024-05-08T06:00:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.265+0000 7f25eaa00700 1 --2- 172.21.15.112:0/2835490464 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56124a9efa10 0x56124a9f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.265+0000 7f25eaa00700 1 --2- 172.21.15.112:0/2835490464 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56124a9efa10 0x56124a9f1e00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab1b98c9952c0373 server_cookie=b8ab3233fc2fe9a5 in_seq=0 out_seq=0 2024-05-08T06:00:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.265+0000 7f25d5600700 10 client.25700 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.266+0000 7f25d5600700 1 -- 172.21.15.112:0/2835490464 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56124a9a30a0 con 0x56124a9efa10 2024-05-08T06:00:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.266+0000 7f25d5600700 10 client.25700 handle_command_reply: tid=0 2024-05-08T06:00:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.266+0000 7f25fc96fb80 1 client.25700 shutdown 2024-05-08T06:00:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.267+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2835490464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9d6600 msgr2=0x56124a9e1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.267+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2835490464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9d6600 0x56124a9e1e50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f25dc003ed0 tx=0x7f25dc0038b0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.267+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2835490464 shutdown_connections 2024-05-08T06:00:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.267+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2835490464 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56124a9efa10 0x56124a9f1e00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.267+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2835490464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9dc300 0x56124a9df3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.267+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2835490464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9d6600 0x56124a9e1e50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.267+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2835490464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9db9e0 0x56124a9e2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.267+0000 7f25d5600700 0 client.25700 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.267+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2835490464 >> 172.21.15.112:0/2835490464 conn(0x56124a904b60 msgr2=0x56124a9cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.267+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2835490464 shutdown_connections 2024-05-08T06:00:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.267+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2835490464 wait complete. 2024-05-08T06:00:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.267+0000 7f25fc96fb80 20 client.25700 trim_cache size 0 max 16384 2024-05-08T06:00:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.290+0000 7f25fc96fb80 1 Processor -- start 2024-05-08T06:00:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.290+0000 7f25fc96fb80 1 -- start start 2024-05-08T06:00:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.291+0000 7f25fc96fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a904b60 0x56124a9d5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.291+0000 7f25fc96fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9d6600 0x56124a9dcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.291+0000 7f25fc96fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9db9e0 0x56124a9f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.291+0000 7f25fc96fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1b26a0 con 0x56124a9d6600 2024-05-08T06:00:17.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.291+0000 7f25fc96fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1afed0 con 0x56124a9db9e0 2024-05-08T06:00:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.291+0000 7f25fc96fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1add20 con 0x56124a904b60 2024-05-08T06:00:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.291+0000 7f25eb400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a904b60 0x56124a9d5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.291+0000 7f25eb400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a904b60 0x56124a9d5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:58332/0 (socket says 172.21.15.112:58332) 2024-05-08T06:00:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.291+0000 7f25eb400700 1 -- 172.21.15.112:0/2930989410 learned_addr learned my addr 172.21.15.112:0/2930989410 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.291+0000 7f25eaa00700 1 --2- 172.21.15.112:0/2930989410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9d6600 0x56124a9dcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.291+0000 7f25ebe00700 1 --2- 172.21.15.112:0/2930989410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9db9e0 0x56124a9f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.291+0000 7f25eb400700 1 -- 172.21.15.112:0/2930989410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9db9e0 msgr2=0x56124a9f75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.291+0000 7f25eb400700 1 --2- 172.21.15.112:0/2930989410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9db9e0 0x56124a9f75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.291+0000 7f25eb400700 1 -- 172.21.15.112:0/2930989410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9d6600 msgr2=0x56124a9dcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.291+0000 7f25eb400700 1 --2- 172.21.15.112:0/2930989410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9d6600 0x56124a9dcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.292+0000 7f25eb400700 1 -- 172.21.15.112:0/2930989410 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56124a9f2390 con 0x56124a904b60 2024-05-08T06:00:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.292+0000 7f25eaa00700 1 --2- 172.21.15.112:0/2930989410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9d6600 0x56124a9dcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:00:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.292+0000 7f25eb400700 1 --2- 172.21.15.112:0/2930989410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a904b60 0x56124a9d5fe0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0092a0 tx=0x7f25dc009730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ccd6a7930ddb54a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.292+0000 7f25d7e00700 1 -- 172.21.15.112:0/2930989410 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc027040 con 0x56124a904b60 2024-05-08T06:00:17.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.292+0000 7f25d7e00700 1 -- 172.21.15.112:0/2930989410 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25dc003920 con 0x56124a904b60 2024-05-08T06:00:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.292+0000 7f25d7e00700 1 -- 172.21.15.112:0/2930989410 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc023aa0 con 0x56124a904b60 2024-05-08T06:00:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.293+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2930989410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a904b60 msgr2=0x56124a9d5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.293+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2930989410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a904b60 0x56124a9d5fe0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0092a0 tx=0x7f25dc009730 comp rx=0 tx=0).stop 2024-05-08T06:00:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.293+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2930989410 shutdown_connections 2024-05-08T06:00:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.293+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2930989410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a9db9e0 0x56124a9f75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.293+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2930989410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a904b60 0x56124a9d5fe0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.293+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2930989410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9d6600 0x56124a9dcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.293+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2930989410 >> 172.21.15.112:0/2930989410 conn(0x56124a9dff50 msgr2=0x56124a9dc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.293+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2930989410 shutdown_connections 2024-05-08T06:00:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.294+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2930989410 wait complete. 2024-05-08T06:00:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.294+0000 7f25fc96fb80 1 -- start start 2024-05-08T06:00:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.294+0000 7f25fc96fb80 1 Processor -- start 2024-05-08T06:00:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.295+0000 7f25fc96fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a904b60 0x56124a9e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.295+0000 7f25fc96fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9d6600 0x56124a9e5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.295+0000 7f25fc96fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9db9e0 0x56124a9e56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.295+0000 7f25fc96fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1ae390 con 0x56124a9db9e0 2024-05-08T06:00:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.295+0000 7f25fc96fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1af800 con 0x56124a904b60 2024-05-08T06:00:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.295+0000 7f25fc96fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1aea60 con 0x56124a9d6600 2024-05-08T06:00:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.295+0000 7f25fc96fb80 10 client.0 _ll_register_callbacks cb 0x56124a8cfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:00:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.295+0000 7f25eb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a904b60 0x56124a9e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.295+0000 7f25eaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9d6600 0x56124a9e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.295+0000 7f25ebe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9db9e0 0x56124a9e56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.295+0000 7f25eb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a904b60 0x56124a9e86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:58874/0 (socket says 172.21.15.112:58874) 2024-05-08T06:00:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.295+0000 7f25eaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9d6600 0x56124a9e5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:58344/0 (socket says 172.21.15.112:58344) 2024-05-08T06:00:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.295+0000 7f25eb400700 1 -- 172.21.15.112:0/2537513507 learned_addr learned my addr 172.21.15.112:0/2537513507 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.296+0000 7f25eaa00700 1 -- 172.21.15.112:0/2537513507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a904b60 msgr2=0x56124a9e86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:17.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.296+0000 7f25eaa00700 1 --2- 172.21.15.112:0/2537513507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a904b60 0x56124a9e86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.296+0000 7f25eaa00700 1 -- 172.21.15.112:0/2537513507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9db9e0 msgr2=0x56124a9e56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.296+0000 7f25eaa00700 1 --2- 172.21.15.112:0/2537513507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9db9e0 0x56124a9e56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.296+0000 7f25eaa00700 1 -- 172.21.15.112:0/2537513507 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56124a9e5ea0 con 0x56124a9d6600 2024-05-08T06:00:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.296+0000 7f25eb400700 1 --2- 172.21.15.112:0/2537513507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a904b60 0x56124a9e86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:00:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.296+0000 7f25eaa00700 1 --2- 172.21.15.112:0/2537513507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9d6600 0x56124a9e5190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f25d8006010 tx=0x7f25d801a070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2abd0cc998317de2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.296+0000 7f25d5600700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:00:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.296+0000 7f25d5600700 1 -- 172.21.15.112:0/2537513507 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8003a20 con 0x56124a9d6600 2024-05-08T06:00:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.296+0000 7f25d5600700 1 -- 172.21.15.112:0/2537513507 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f25d8003bc0 con 0x56124a9d6600 2024-05-08T06:00:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.296+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2537513507 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25dc003920 con 0x56124a9d6600 2024-05-08T06:00:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.297+0000 7f25d5600700 1 -- 172.21.15.112:0/2537513507 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f25d8042050 con 0x56124a9d6600 2024-05-08T06:00:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.297+0000 7f25fc96fb80 10 client.25744 fetch_fsmap learned FSMap version 30 2024-05-08T06:00:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.297+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2537513507 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56124aa5b130 con 0x56124a9d6600 2024-05-08T06:00:17.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.297+0000 7f25d5600700 1 -- 172.21.15.112:0/2537513507 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25dc003920 con 0x56124a9d6600 2024-05-08T06:00:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.297+0000 7f25d5600700 1 -- 172.21.15.112:0/2537513507 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f25d800f620 con 0x56124a9d6600 2024-05-08T06:00:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.297+0000 7f25fc96fb80 10 client.25744 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T06:00:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.297+0000 7f25fc96fb80 10 client.25744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:00:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.297+0000 7f25fc96fb80 20 client.25744 populate_metadata read hostname 'smithi112' 2024-05-08T06:00:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.297+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2537513507 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56124a9dc300 0x56124aa5d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.298+0000 7f25fc96fb80 4 client.25744 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T06:00:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.298+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2537513507 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56124a936400 con 0x56124a9dc300 2024-05-08T06:00:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.298+0000 7f25eb400700 1 --2- 172.21.15.112:0/2537513507 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56124a9dc300 0x56124aa5d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.299+0000 7f25eb400700 1 --2- 172.21.15.112:0/2537513507 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56124a9dc300 0x56124aa5d910 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2041d31102095a4d server_cookie=46f035656cc3af34 in_seq=0 out_seq=0 2024-05-08T06:00:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.299+0000 7f25d5600700 10 client.25744 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.299+0000 7f25d5600700 1 -- 172.21.15.112:0/2537513507 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56124a936400 con 0x56124a9dc300 2024-05-08T06:00:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.299+0000 7f25d5600700 10 client.25744 handle_command_reply: tid=0 2024-05-08T06:00:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.299+0000 7f25fc96fb80 1 client.25744 shutdown 2024-05-08T06:00:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.299+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2537513507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9d6600 msgr2=0x56124a9e5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:17.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.299+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2537513507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9d6600 0x56124a9e5190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f25d8006010 tx=0x7f25d801a070 comp rx=0 tx=0).stop 2024-05-08T06:00:17.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.299+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2537513507 shutdown_connections 2024-05-08T06:00:17.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.299+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2537513507 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56124a9dc300 0x56124aa5d910 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.299+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2537513507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56124a9db9e0 0x56124a9e56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.299+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2537513507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56124a904b60 0x56124a9e86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.299+0000 7f25fc96fb80 1 --2- 172.21.15.112:0/2537513507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56124a9d6600 0x56124a9e5190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.299+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2537513507 >> 172.21.15.112:0/2537513507 conn(0x56124a9dff50 msgr2=0x56124a9cabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:17.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.299+0000 7f25d5600700 0 client.25744 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:17.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.300+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2537513507 shutdown_connections 2024-05-08T06:00:17.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.300+0000 7f25fc96fb80 1 -- 172.21.15.112:0/2537513507 wait complete. 2024-05-08T06:00:17.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.300+0000 7f25fc96fb80 20 client.25744 trim_cache size 0 max 16384 2024-05-08T06:00:17.448 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:00:17.449 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T06:00:17.449 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-05-08T06:00:17.449 INFO:teuthology.orchestra.run.smithi112.stdout: "3eb3bca1-7bf6-480f-b034-5e2646300dcb": { 2024-05-08T06:00:17.449 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-05-08T06:00:17.449 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "3eb3bca1-7bf6-480f-b034-5e2646300dcb", 2024-05-08T06:00:17.449 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-05-08T06:00:17.449 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:00:17.449 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:00:17.449 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:00:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.301+0000 7f25f0a00700 1 -- 172.21.15.112:0/969103808 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f25d00b2900 msgr2=0x7f25d00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.301+0000 7f25f0a00700 1 --2- 172.21.15.112:0/969103808 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f25d00b2900 0x7f25d00b4dc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f25d800b6c0 tx=0x7f25d8009590 comp rx=0 tx=0).stop 2024-05-08T06:00:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.301+0000 7f25f0a00700 1 -- 172.21.15.112:0/969103808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ec109460 msgr2=0x7f25ec19bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.301+0000 7f25f0a00700 1 --2- 172.21.15.112:0/969103808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ec109460 0x7f25ec19bbb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f25e400ef30 tx=0x7f25e400c3a0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.301+0000 7f25f0a00700 1 -- 172.21.15.112:0/969103808 shutdown_connections 2024-05-08T06:00:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.301+0000 7f25f0a00700 1 --2- 172.21.15.112:0/969103808 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f25d00b2900 0x7f25d00b4dc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.301+0000 7f25f0a00700 1 --2- 172.21.15.112:0/969103808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ec109460 0x7f25ec19bbb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.301+0000 7f25f0a00700 1 --2- 172.21.15.112:0/969103808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25ec1033d0 0x7f25ec1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.301+0000 7f25f0a00700 1 --2- 172.21.15.112:0/969103808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25ec103d90 0x7f25ec1a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.301+0000 7f25f0a00700 1 -- 172.21.15.112:0/969103808 >> 172.21.15.112:0/969103808 conn(0x7f25ec0fef20 msgr2=0x7f25ec106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.302+0000 7f25f0a00700 1 -- 172.21.15.112:0/969103808 shutdown_connections 2024-05-08T06:00:17.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:17.302+0000 7f25f0a00700 1 -- 172.21.15.112:0/969103808 wait complete. 2024-05-08T06:00:17.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:17 smithi162 ceph-mon[34800]: pgmap v590: 97 pgs: 97 active+clean; 94 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.5 MiB/s wr, 468 op/s 2024-05-08T06:00:17.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:17 smithi150 ceph-mon[34315]: pgmap v590: 97 pgs: 97 active+clean; 94 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.5 MiB/s wr, 468 op/s 2024-05-08T06:00:18.589 INFO:tasks.cephfs.filesystem:scrub status for tag:3eb3bca1-7bf6-480f-b034-5e2646300dcb - {'path': '/', 'tag': '3eb3bca1-7bf6-480f-b034-5e2646300dcb', 'options': 'recursive,force'} 2024-05-08T06:00:19.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:19 smithi112 ceph-mon[30003]: pgmap v591: 97 pgs: 97 active+clean; 94 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 321 op/s 2024-05-08T06:00:19.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:19 smithi162 ceph-mon[34800]: pgmap v591: 97 pgs: 97 active+clean; 94 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 321 op/s 2024-05-08T06:00:19.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:19 smithi150 ceph-mon[34315]: pgmap v591: 97 pgs: 97 active+clean; 94 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 321 op/s 2024-05-08T06:00:21.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:21 smithi112 ceph-mon[30003]: pgmap v592: 97 pgs: 97 active+clean; 93 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.4 MiB/s wr, 430 op/s 2024-05-08T06:00:21.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:21 smithi162 ceph-mon[34800]: pgmap v592: 97 pgs: 97 active+clean; 93 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.4 MiB/s wr, 430 op/s 2024-05-08T06:00:21.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:21 smithi150 ceph-mon[34315]: pgmap v592: 97 pgs: 97 active+clean; 93 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.4 MiB/s wr, 430 op/s 2024-05-08T06:00:23.478 INFO:tasks.workunit.client.0.smithi112.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-05-08T06:00:23.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:23 smithi112 ceph-mon[30003]: pgmap v593: 97 pgs: 97 active+clean; 94 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 365 op/s 2024-05-08T06:00:23.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:23 smithi162 ceph-mon[34800]: pgmap v593: 97 pgs: 97 active+clean; 94 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 365 op/s 2024-05-08T06:00:23.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:23 smithi150 ceph-mon[34315]: pgmap v593: 97 pgs: 97 active+clean; 94 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 365 op/s 2024-05-08T06:00:24.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:24 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T06:00:24.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:24 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T06:00:24.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:24 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T06:00:24.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:24 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T06:00:24.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:24 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T06:00:24.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:24 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T06:00:24.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:24 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T06:00:24.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:24 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T06:00:24.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:24 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T06:00:25.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:25 smithi112 ceph-mon[30003]: pgmap v594: 97 pgs: 97 active+clean; 94 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 320 op/s 2024-05-08T06:00:25.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:25 smithi162 ceph-mon[34800]: pgmap v594: 97 pgs: 97 active+clean; 94 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 320 op/s 2024-05-08T06:00:25.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:25 smithi150 ceph-mon[34315]: pgmap v594: 97 pgs: 97 active+clean; 94 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 320 op/s 2024-05-08T06:00:27.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:27 smithi112 ceph-mon[30003]: pgmap v595: 97 pgs: 97 active+clean; 96 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.5 MiB/s wr, 457 op/s 2024-05-08T06:00:27.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:27 smithi162 ceph-mon[34800]: pgmap v595: 97 pgs: 97 active+clean; 96 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.5 MiB/s wr, 457 op/s 2024-05-08T06:00:27.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:27 smithi150 ceph-mon[34315]: pgmap v595: 97 pgs: 97 active+clean; 96 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.5 MiB/s wr, 457 op/s 2024-05-08T06:00:29.500 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:29 smithi150 ceph-mon[34315]: pgmap v596: 97 pgs: 97 active+clean; 96 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 310 op/s 2024-05-08T06:00:29.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:29 smithi112 ceph-mon[30003]: pgmap v596: 97 pgs: 97 active+clean; 96 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 310 op/s 2024-05-08T06:00:29.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:29 smithi162 ceph-mon[34800]: pgmap v596: 97 pgs: 97 active+clean; 96 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 310 op/s 2024-05-08T06:00:31.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:31 smithi112 ceph-mon[30003]: pgmap v597: 97 pgs: 97 active+clean; 95 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.3 MiB/s wr, 390 op/s 2024-05-08T06:00:31.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:31 smithi162 ceph-mon[34800]: pgmap v597: 97 pgs: 97 active+clean; 95 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.3 MiB/s wr, 390 op/s 2024-05-08T06:00:31.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:31 smithi150 ceph-mon[34315]: pgmap v597: 97 pgs: 97 active+clean; 95 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.3 MiB/s wr, 390 op/s 2024-05-08T06:00:33.339 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x9ffb01 2024-05-08T06:00:33.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:33 smithi112 ceph-mon[30003]: pgmap v598: 97 pgs: 97 active+clean; 95 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-05-08T06:00:33.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:33 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:33.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:33 smithi162 ceph-mon[34800]: pgmap v598: 97 pgs: 97 active+clean; 95 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-05-08T06:00:33.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:33 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:33.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:33 smithi150 ceph-mon[34315]: pgmap v598: 97 pgs: 97 active+clean; 95 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-05-08T06:00:33.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:33 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:34.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:34.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:34.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:34.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:34.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:34.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:34.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:34.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:34.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:34.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:34.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:34.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:35.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:35 smithi112 ceph-mon[30003]: pgmap v599: 97 pgs: 97 active+clean; 95 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-05-08T06:00:35.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:35 smithi112 ceph-mon[30003]: Adjusting osd_memory_target on smithi162 to 176.2M 2024-05-08T06:00:35.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:35 smithi112 ceph-mon[30003]: Unable to set osd_memory_target on smithi162 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-08T06:00:35.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:35 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:00:35.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:35 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:35.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:35 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:00:35.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:35 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:00:35.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:35 smithi162 ceph-mon[34800]: pgmap v599: 97 pgs: 97 active+clean; 95 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-05-08T06:00:35.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:35 smithi162 ceph-mon[34800]: Adjusting osd_memory_target on smithi162 to 176.2M 2024-05-08T06:00:35.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:35 smithi162 ceph-mon[34800]: Unable to set osd_memory_target on smithi162 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-08T06:00:35.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:35 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:00:35.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:35 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:35.568 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:35 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:00:35.569 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:35 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:00:35.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:35 smithi150 ceph-mon[34315]: pgmap v599: 97 pgs: 97 active+clean; 95 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-05-08T06:00:35.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:35 smithi150 ceph-mon[34315]: Adjusting osd_memory_target on smithi162 to 176.2M 2024-05-08T06:00:35.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:35 smithi150 ceph-mon[34315]: Unable to set osd_memory_target on smithi162 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-08T06:00:35.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:35 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:00:35.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:35 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:35.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:35 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:00:35.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:35 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:00:37.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:37 smithi112 ceph-mon[30003]: pgmap v600: 97 pgs: 97 active+clean; 94 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.1 MiB/s wr, 351 op/s 2024-05-08T06:00:37.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:37 smithi162 ceph-mon[34800]: pgmap v600: 97 pgs: 97 active+clean; 94 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.1 MiB/s wr, 351 op/s 2024-05-08T06:00:37.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:37 smithi150 ceph-mon[34315]: pgmap v600: 97 pgs: 97 active+clean; 94 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.1 MiB/s wr, 351 op/s 2024-05-08T06:00:39.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:39 smithi112 ceph-mon[30003]: pgmap v601: 97 pgs: 97 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.4 MiB/s wr, 254 op/s 2024-05-08T06:00:39.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:39 smithi162 ceph-mon[34800]: pgmap v601: 97 pgs: 97 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.4 MiB/s wr, 254 op/s 2024-05-08T06:00:39.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:39 smithi150 ceph-mon[34315]: pgmap v601: 97 pgs: 97 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.4 MiB/s wr, 254 op/s 2024-05-08T06:00:41.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:41 smithi112 ceph-mon[30003]: pgmap v602: 97 pgs: 97 active+clean; 99 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.9 MiB/s wr, 318 op/s 2024-05-08T06:00:41.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:41 smithi162 ceph-mon[34800]: pgmap v602: 97 pgs: 97 active+clean; 99 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.9 MiB/s wr, 318 op/s 2024-05-08T06:00:41.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:41 smithi150 ceph-mon[34315]: pgmap v602: 97 pgs: 97 active+clean; 99 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.9 MiB/s wr, 318 op/s 2024-05-08T06:00:42.713 INFO:tasks.workunit.client.0.smithi112.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-05-08T06:00:43.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:43 smithi112 ceph-mon[30003]: pgmap v603: 97 pgs: 97 active+clean; 99 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 238 op/s 2024-05-08T06:00:43.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:43 smithi162 ceph-mon[34800]: pgmap v603: 97 pgs: 97 active+clean; 99 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 238 op/s 2024-05-08T06:00:43.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:43 smithi150 ceph-mon[34315]: pgmap v603: 97 pgs: 97 active+clean; 99 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 238 op/s 2024-05-08T06:00:45.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:45 smithi112 ceph-mon[30003]: pgmap v604: 97 pgs: 97 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 272 op/s 2024-05-08T06:00:45.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:45 smithi162 ceph-mon[34800]: pgmap v604: 97 pgs: 97 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 272 op/s 2024-05-08T06:00:45.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:45 smithi150 ceph-mon[34315]: pgmap v604: 97 pgs: 97 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 272 op/s 2024-05-08T06:00:47.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:47 smithi112 ceph-mon[30003]: pgmap v605: 97 pgs: 97 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.8 MiB/s wr, 315 op/s 2024-05-08T06:00:47.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:47 smithi162 ceph-mon[34800]: pgmap v605: 97 pgs: 97 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.8 MiB/s wr, 315 op/s 2024-05-08T06:00:47.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:47 smithi150 ceph-mon[34315]: pgmap v605: 97 pgs: 97 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.8 MiB/s wr, 315 op/s 2024-05-08T06:00:48.589 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T06:00:49.045 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:00:49.561 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:49 smithi112 ceph-mon[30003]: pgmap v606: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 266 op/s 2024-05-08T06:00:49.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:49 smithi162 ceph-mon[34800]: pgmap v606: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 266 op/s 2024-05-08T06:00:49.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:49 smithi150 ceph-mon[34315]: pgmap v606: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 266 op/s 2024-05-08T06:00:51.350 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:51 smithi112 ceph-mon[30003]: pgmap v607: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 325 op/s 2024-05-08T06:00:51.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:51 smithi162 ceph-mon[34800]: pgmap v607: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 325 op/s 2024-05-08T06:00:51.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.661+0000 7fd51da00700 1 -- 172.21.15.112:0/3925564351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd518109600 msgr2=0x7fd5181099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:51.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.661+0000 7fd51da00700 1 --2- 172.21.15.112:0/3925564351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd518109600 0x7fd5181099e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd508009990 tx=0x7fd508009ca0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.662+0000 7fd51da00700 1 -- 172.21.15.112:0/3925564351 shutdown_connections 2024-05-08T06:00:51.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.662+0000 7fd51da00700 1 --2- 172.21.15.112:0/3925564351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd518103eb0 0x7fd518110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.662+0000 7fd51da00700 1 --2- 172.21.15.112:0/3925564351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd518109600 0x7fd5181099e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.662+0000 7fd51da00700 1 --2- 172.21.15.112:0/3925564351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5181034f0 0x7fd518103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.662+0000 7fd51da00700 1 -- 172.21.15.112:0/3925564351 >> 172.21.15.112:0/3925564351 conn(0x7fd51807a650 msgr2=0x7fd51807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:51.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.662+0000 7fd51da00700 1 -- 172.21.15.112:0/3925564351 shutdown_connections 2024-05-08T06:00:51.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.662+0000 7fd51da00700 1 -- 172.21.15.112:0/3925564351 wait complete. 2024-05-08T06:00:51.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.663+0000 7fd51da00700 1 Processor -- start 2024-05-08T06:00:51.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.663+0000 7fd51da00700 1 -- start start 2024-05-08T06:00:51.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.664+0000 7fd51da00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5181034f0 0x7fd5181a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.664+0000 7fd51da00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd518103eb0 0x7fd5181a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.664+0000 7fd51da00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd518109600 0x7fd51810f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.664+0000 7fd51da00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5181136a0 con 0x7fd518109600 2024-05-08T06:00:51.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.664+0000 7fd51da00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd518113520 con 0x7fd518103eb0 2024-05-08T06:00:51.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.664+0000 7fd51da00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd518113820 con 0x7fd5181034f0 2024-05-08T06:00:51.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.664+0000 7fd516000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd518103eb0 0x7fd5181a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.664+0000 7fd516000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd518103eb0 0x7fd5181a51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:44342/0 (socket says 172.21.15.112:44342) 2024-05-08T06:00:51.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.664+0000 7fd516000700 1 -- 172.21.15.112:0/2458882421 learned_addr learned my addr 172.21.15.112:0/2458882421 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:51.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.664+0000 7fd516a00700 1 --2- 172.21.15.112:0/2458882421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5181034f0 0x7fd5181a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.665+0000 7fd516000700 1 -- 172.21.15.112:0/2458882421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5181034f0 msgr2=0x7fd5181a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:51.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.665+0000 7fd516000700 1 --2- 172.21.15.112:0/2458882421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5181034f0 0x7fd5181a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.665+0000 7fd516000700 1 -- 172.21.15.112:0/2458882421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd518109600 msgr2=0x7fd51810f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:51.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.665+0000 7fd516000700 1 --2- 172.21.15.112:0/2458882421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd518109600 0x7fd51810f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.665+0000 7fd516000700 1 -- 172.21.15.112:0/2458882421 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51810fb60 con 0x7fd518103eb0 2024-05-08T06:00:51.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.665+0000 7fd516a00700 1 --2- 172.21.15.112:0/2458882421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5181034f0 0x7fd5181a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:00:51.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.665+0000 7fd516000700 1 --2- 172.21.15.112:0/2458882421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd518103eb0 0x7fd5181a51e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd50400e9a0 tx=0x7fd50400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:51.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.665+0000 7fd50f400700 1 -- 172.21.15.112:0/2458882421 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50400cb50 con 0x7fd518103eb0 2024-05-08T06:00:51.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.665+0000 7fd50f400700 1 -- 172.21.15.112:0/2458882421 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd50400ccf0 con 0x7fd518103eb0 2024-05-08T06:00:51.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.665+0000 7fd50f400700 1 -- 172.21.15.112:0/2458882421 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd504018680 con 0x7fd518103eb0 2024-05-08T06:00:51.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.666+0000 7fd51da00700 1 -- 172.21.15.112:0/2458882421 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51810fe50 con 0x7fd518103eb0 2024-05-08T06:00:51.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.666+0000 7fd51da00700 1 -- 172.21.15.112:0/2458882421 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5181acc40 con 0x7fd518103eb0 2024-05-08T06:00:51.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.671+0000 7fd528424b80 1 Processor -- start 2024-05-08T06:00:51.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.671+0000 7fd50f400700 1 -- 172.21.15.112:0/2458882421 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd504018820 con 0x7fd518103eb0 2024-05-08T06:00:51.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.671+0000 7fd528424b80 1 -- start start 2024-05-08T06:00:51.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.671+0000 7fd528424b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f060 0x55917a70f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.671+0000 7fd528424b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a70f980 0x55917a713490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.671+0000 7fd528424b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a7139d0 0x55917a713e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.671+0000 7fd528424b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55917a661a20 con 0x55917a70f060 2024-05-08T06:00:51.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.671+0000 7fd528424b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55917a5ff760 con 0x55917a7139d0 2024-05-08T06:00:51.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.671+0000 7fd528424b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55917a65d9e0 con 0x55917a70f980 2024-05-08T06:00:51.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.672+0000 7fd517400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f060 0x55917a70f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.672+0000 7fd517400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f060 0x55917a70f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47190/0 (socket says 172.21.15.112:47190) 2024-05-08T06:00:51.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.672+0000 7fd517400700 1 -- 172.21.15.112:0/2714429185 learned_addr learned my addr 172.21.15.112:0/2714429185 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:51.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.672+0000 7fd516000700 1 --2- 172.21.15.112:0/2714429185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a7139d0 0x55917a713e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.673+0000 7fd517400700 1 -- 172.21.15.112:0/2714429185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a70f980 msgr2=0x55917a713490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:51.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.673+0000 7fd50f400700 1 --2- 172.21.15.112:0/2458882421 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4fc0b2900 0x7fd4fc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.672+0000 7fd516a00700 1 --2- 172.21.15.112:0/2714429185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a70f980 0x55917a713490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.673+0000 7fd517400700 1 --2- 172.21.15.112:0/2714429185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a70f980 0x55917a713490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.673+0000 7fd50f400700 1 -- 172.21.15.112:0/2458882421 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd5040e2560 con 0x7fd518103eb0 2024-05-08T06:00:51.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.673+0000 7fd517400700 1 -- 172.21.15.112:0/2714429185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a7139d0 msgr2=0x55917a713e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:51.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.673+0000 7fd517400700 1 --2- 172.21.15.112:0/2714429185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a7139d0 0x55917a713e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.673+0000 7fd517400700 1 -- 172.21.15.112:0/2714429185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917a5ae8d0 con 0x55917a70f060 2024-05-08T06:00:51.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.673+0000 7fd516000700 1 --2- 172.21.15.112:0/2714429185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a7139d0 0x55917a713e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:00:51.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.673+0000 7fd516a00700 1 --2- 172.21.15.112:0/2714429185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a70f980 0x55917a713490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:00:51.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.674+0000 7fd517400700 1 --2- 172.21.15.112:0/2714429185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f060 0x55917a70f440 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd51000e490 tx=0x7fd51000e7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc694ddef970e87 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:51.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.674+0000 7fd517400700 1 --2- 172.21.15.112:0/2458882421 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4fc0b2900 0x7fd4fc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.674+0000 7fd503e00700 1 -- 172.21.15.112:0/2714429185 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5100090d0 con 0x55917a70f060 2024-05-08T06:00:51.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.674+0000 7fd503e00700 1 -- 172.21.15.112:0/2714429185 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd510019070 con 0x55917a70f060 2024-05-08T06:00:51.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.674+0000 7fd503e00700 1 -- 172.21.15.112:0/2714429185 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd510014d50 con 0x55917a70f060 2024-05-08T06:00:51.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.675+0000 7fd528424b80 1 -- 172.21.15.112:0/2714429185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f060 msgr2=0x55917a70f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:51.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.675+0000 7fd528424b80 1 --2- 172.21.15.112:0/2714429185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f060 0x55917a70f440 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd51000e490 tx=0x7fd51000e7a0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.675+0000 7fd517400700 1 --2- 172.21.15.112:0/2458882421 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4fc0b2900 0x7fd4fc0b4dc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd510018000 tx=0x7fd510014600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:51.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.675+0000 7fd528424b80 1 -- 172.21.15.112:0/2714429185 shutdown_connections 2024-05-08T06:00:51.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.675+0000 7fd528424b80 1 --2- 172.21.15.112:0/2714429185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a7139d0 0x55917a713e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.675+0000 7fd528424b80 1 --2- 172.21.15.112:0/2714429185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f060 0x55917a70f440 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.675+0000 7fd528424b80 1 --2- 172.21.15.112:0/2714429185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a70f980 0x55917a713490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.675+0000 7fd528424b80 1 -- 172.21.15.112:0/2714429185 >> 172.21.15.112:0/2714429185 conn(0x55917a63db60 msgr2=0x55917a7097a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:51.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.675+0000 7fd528424b80 1 -- 172.21.15.112:0/2714429185 shutdown_connections 2024-05-08T06:00:51.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.675+0000 7fd528424b80 1 -- 172.21.15.112:0/2714429185 wait complete. 2024-05-08T06:00:51.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.676+0000 7fd528424b80 1 -- start start 2024-05-08T06:00:51.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.676+0000 7fd528424b80 1 Processor -- start 2024-05-08T06:00:51.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.677+0000 7fd528424b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f060 0x55917a71ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.677+0000 7fd528424b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f980 0x55917a7181a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.677+0000 7fd528424b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a7139d0 0x55917a7186e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.677+0000 7fd528424b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55917a71bb80 con 0x55917a70f060 2024-05-08T06:00:51.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.677+0000 7fd528424b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55917a5c7b20 con 0x55917a70f980 2024-05-08T06:00:51.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.677+0000 7fd528424b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55917a71bd00 con 0x55917a7139d0 2024-05-08T06:00:51.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.677+0000 7fd528424b80 10 client.0 _ll_register_callbacks cb 0x55917a608c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:00:51.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.677+0000 7fd516000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f980 0x55917a7181a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.677+0000 7fd517400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a7139d0 0x55917a7186e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.677+0000 7fd516a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f060 0x55917a71ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.677+0000 7fd517400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a7139d0 0x55917a7186e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:40820/0 (socket says 172.21.15.112:40820) 2024-05-08T06:00:51.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.677+0000 7fd517400700 1 -- 172.21.15.112:0/1231467252 learned_addr learned my addr 172.21.15.112:0/1231467252 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:51.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.677+0000 7fd516000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f980 0x55917a7181a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:44354/0 (socket says 172.21.15.112:44354) 2024-05-08T06:00:51.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.678+0000 7fd516000700 1 -- 172.21.15.112:0/1231467252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a7139d0 msgr2=0x55917a7186e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:51.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.678+0000 7fd516000700 1 --2- 172.21.15.112:0/1231467252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a7139d0 0x55917a7186e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.678+0000 7fd516000700 1 -- 172.21.15.112:0/1231467252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f060 msgr2=0x55917a71ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:51.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.678+0000 7fd516000700 1 --2- 172.21.15.112:0/1231467252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f060 0x55917a71ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.678+0000 7fd516000700 1 -- 172.21.15.112:0/1231467252 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917a627420 con 0x55917a70f980 2024-05-08T06:00:51.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.678+0000 7fd516000700 1 --2- 172.21.15.112:0/1231467252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f980 0x55917a7181a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd504010fd0 tx=0x7fd5040b27a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7145e2c501401861 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:51.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.678+0000 7fd500c00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:00:51.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.678+0000 7fd500c00700 1 -- 172.21.15.112:0/1231467252 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5040a98c0 con 0x55917a70f980 2024-05-08T06:00:51.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.678+0000 7fd528424b80 1 -- 172.21.15.112:0/1231467252 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd510019070 con 0x55917a70f980 2024-05-08T06:00:51.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.678+0000 7fd500c00700 1 -- 172.21.15.112:0/1231467252 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd5040cc2f0 con 0x55917a70f980 2024-05-08T06:00:51.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.679+0000 7fd500c00700 1 -- 172.21.15.112:0/1231467252 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd5040a9260 con 0x55917a70f980 2024-05-08T06:00:51.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.679+0000 7fd528424b80 10 client.25718 fetch_fsmap learned FSMap version 31 2024-05-08T06:00:51.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.679+0000 7fd528424b80 1 -- 172.21.15.112:0/1231467252 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55917a726360 con 0x55917a70f980 2024-05-08T06:00:51.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.679+0000 7fd500c00700 1 -- 172.21.15.112:0/1231467252 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd510019070 con 0x55917a70f980 2024-05-08T06:00:51.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.679+0000 7fd500c00700 1 -- 172.21.15.112:0/1231467252 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fd5040e20d0 con 0x55917a70f980 2024-05-08T06:00:51.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.679+0000 7fd528424b80 10 client.25718 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T06:00:51.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.679+0000 7fd528424b80 10 client.25718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:00:51.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.679+0000 7fd528424b80 20 client.25718 populate_metadata read hostname 'smithi112' 2024-05-08T06:00:51.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.679+0000 7fd528424b80 1 --2- 172.21.15.112:0/1231467252 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55917a728940 0x55917a72ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.679+0000 7fd528424b80 4 client.25718 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:00:51.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.679+0000 7fd528424b80 1 -- 172.21.15.112:0/1231467252 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55917a6dc0a0 con 0x55917a728940 2024-05-08T06:00:51.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.680+0000 7fd516a00700 1 --2- 172.21.15.112:0/1231467252 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55917a728940 0x55917a72ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.681+0000 7fd516a00700 1 --2- 172.21.15.112:0/1231467252 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55917a728940 0x55917a72ad30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fb08ec2421896f2 server_cookie=b81541fb96eb8f36 in_seq=0 out_seq=0 2024-05-08T06:00:51.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.681+0000 7fd500c00700 10 client.25718 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:51.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.682+0000 7fd500c00700 1 -- 172.21.15.112:0/1231467252 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55917a6dc0a0 con 0x55917a728940 2024-05-08T06:00:51.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.682+0000 7fd500c00700 10 client.25718 handle_command_reply: tid=0 2024-05-08T06:00:51.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.682+0000 7fd528424b80 1 client.25718 shutdown 2024-05-08T06:00:51.684 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:51 smithi150 ceph-mon[34315]: pgmap v607: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 325 op/s 2024-05-08T06:00:51.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.682+0000 7fd528424b80 1 -- 172.21.15.112:0/1231467252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f980 msgr2=0x55917a7181a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:51.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.682+0000 7fd528424b80 1 --2- 172.21.15.112:0/1231467252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f980 0x55917a7181a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd504010fd0 tx=0x7fd5040b27a0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.683+0000 7fd528424b80 1 -- 172.21.15.112:0/1231467252 shutdown_connections 2024-05-08T06:00:51.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.683+0000 7fd528424b80 1 --2- 172.21.15.112:0/1231467252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a7139d0 0x55917a7186e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.683+0000 7fd528424b80 1 --2- 172.21.15.112:0/1231467252 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55917a728940 0x55917a72ad30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.683+0000 7fd528424b80 1 --2- 172.21.15.112:0/1231467252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f060 0x55917a71ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.683+0000 7fd528424b80 1 --2- 172.21.15.112:0/1231467252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f980 0x55917a7181a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.683+0000 7fd500c00700 0 client.25718 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:51.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.683+0000 7fd528424b80 1 -- 172.21.15.112:0/1231467252 >> 172.21.15.112:0/1231467252 conn(0x55917a63db60 msgr2=0x55917a7042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:51.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.683+0000 7fd528424b80 1 -- 172.21.15.112:0/1231467252 shutdown_connections 2024-05-08T06:00:51.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.683+0000 7fd528424b80 1 -- 172.21.15.112:0/1231467252 wait complete. 2024-05-08T06:00:51.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.683+0000 7fd528424b80 20 client.25718 trim_cache size 0 max 16384 2024-05-08T06:00:51.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.707+0000 7fd528424b80 1 Processor -- start 2024-05-08T06:00:51.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.707+0000 7fd528424b80 1 -- start start 2024-05-08T06:00:51.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.707+0000 7fd528424b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f060 0x55917a724c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.707+0000 7fd528424b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f980 0x55917a712aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.707+0000 7fd528424b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a7139d0 0x55917a730640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.707+0000 7fd528424b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5181b6be0 con 0x55917a70f980 2024-05-08T06:00:51.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.707+0000 7fd528424b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5181b4410 con 0x55917a70f060 2024-05-08T06:00:51.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.707+0000 7fd528424b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd5181b2260 con 0x55917a7139d0 2024-05-08T06:00:51.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.708+0000 7fd516a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f060 0x55917a724c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.708+0000 7fd516000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f980 0x55917a712aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.708+0000 7fd517400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a7139d0 0x55917a730640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.708+0000 7fd516a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f060 0x55917a724c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:44368/0 (socket says 172.21.15.112:44368) 2024-05-08T06:00:51.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.708+0000 7fd516a00700 1 -- 172.21.15.112:0/3024588208 learned_addr learned my addr 172.21.15.112:0/3024588208 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:51.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.708+0000 7fd517400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a7139d0 0x55917a730640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:40828/0 (socket says 172.21.15.112:40828) 2024-05-08T06:00:51.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.708+0000 7fd516a00700 1 -- 172.21.15.112:0/3024588208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a7139d0 msgr2=0x55917a730640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:51.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.708+0000 7fd516a00700 1 --2- 172.21.15.112:0/3024588208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a7139d0 0x55917a730640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.708+0000 7fd516a00700 1 -- 172.21.15.112:0/3024588208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f980 msgr2=0x55917a712aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:51.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.708+0000 7fd516a00700 1 --2- 172.21.15.112:0/3024588208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f980 0x55917a712aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.708+0000 7fd516a00700 1 -- 172.21.15.112:0/3024588208 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917a705f90 con 0x55917a70f060 2024-05-08T06:00:51.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.708+0000 7fd516a00700 1 --2- 172.21.15.112:0/3024588208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f060 0x55917a724c10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd508009990 tx=0x7fd508005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a11ab03539e4c0c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:51.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.709+0000 7fd503400700 1 -- 172.21.15.112:0/3024588208 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50802d070 con 0x55917a70f060 2024-05-08T06:00:51.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.709+0000 7fd503400700 1 -- 172.21.15.112:0/3024588208 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd50800bdd0 con 0x55917a70f060 2024-05-08T06:00:51.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.709+0000 7fd503400700 1 -- 172.21.15.112:0/3024588208 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508003e50 con 0x55917a70f060 2024-05-08T06:00:51.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.710+0000 7fd528424b80 1 -- 172.21.15.112:0/3024588208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f060 msgr2=0x55917a724c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:51.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.710+0000 7fd528424b80 1 --2- 172.21.15.112:0/3024588208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f060 0x55917a724c10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd508009990 tx=0x7fd508005ad0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.710+0000 7fd528424b80 1 -- 172.21.15.112:0/3024588208 shutdown_connections 2024-05-08T06:00:51.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.710+0000 7fd528424b80 1 --2- 172.21.15.112:0/3024588208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a7139d0 0x55917a730640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.710+0000 7fd528424b80 1 --2- 172.21.15.112:0/3024588208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f060 0x55917a724c10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.710+0000 7fd528424b80 1 --2- 172.21.15.112:0/3024588208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a70f980 0x55917a712aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.710+0000 7fd528424b80 1 -- 172.21.15.112:0/3024588208 >> 172.21.15.112:0/3024588208 conn(0x55917a63db60 msgr2=0x55917a712390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:51.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.710+0000 7fd528424b80 1 -- 172.21.15.112:0/3024588208 shutdown_connections 2024-05-08T06:00:51.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.710+0000 7fd528424b80 1 -- 172.21.15.112:0/3024588208 wait complete. 2024-05-08T06:00:51.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.711+0000 7fd528424b80 1 -- start start 2024-05-08T06:00:51.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.711+0000 7fd528424b80 1 Processor -- start 2024-05-08T06:00:51.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd528424b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a70f060 0x55917a720de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd528424b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f980 0x55917a721320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd528424b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a7139d0 0x55917a71f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd528424b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5181b28d0 con 0x55917a7139d0 2024-05-08T06:00:51.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd528424b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5181b3d40 con 0x55917a70f980 2024-05-08T06:00:51.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd528424b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd5181b2fa0 con 0x55917a70f060 2024-05-08T06:00:51.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd528424b80 10 client.0 _ll_register_callbacks cb 0x55917a608c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:00:51.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd516a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a70f060 0x55917a720de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd517400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a7139d0 0x55917a71f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd516a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a70f060 0x55917a720de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:40842/0 (socket says 172.21.15.112:40842) 2024-05-08T06:00:51.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd516a00700 1 -- 172.21.15.112:0/169776503 learned_addr learned my addr 172.21.15.112:0/169776503 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:51.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd516a00700 1 -- 172.21.15.112:0/169776503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f980 msgr2=0x55917a721320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:51.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd516a00700 1 --2- 172.21.15.112:0/169776503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f980 0x55917a721320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd516a00700 1 -- 172.21.15.112:0/169776503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a7139d0 msgr2=0x55917a71f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:00:51.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd516a00700 1 --2- 172.21.15.112:0/169776503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a7139d0 0x55917a71f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.712+0000 7fd516a00700 1 -- 172.21.15.112:0/169776503 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917a71faa0 con 0x55917a70f060 2024-05-08T06:00:51.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.713+0000 7fd516a00700 1 --2- 172.21.15.112:0/169776503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a70f060 0x55917a720de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd508003990 tx=0x7fd508005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5786adaf37d0639c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:51.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.713+0000 7fd500c00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:00:51.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.713+0000 7fd500c00700 1 -- 172.21.15.112:0/169776503 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50802d040 con 0x55917a70f060 2024-05-08T06:00:51.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.713+0000 7fd500c00700 1 -- 172.21.15.112:0/169776503 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd508003a90 con 0x55917a70f060 2024-05-08T06:00:51.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.713+0000 7fd528424b80 1 -- 172.21.15.112:0/169776503 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd50800bdd0 con 0x55917a70f060 2024-05-08T06:00:51.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.714+0000 7fd500c00700 1 -- 172.21.15.112:0/169776503 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd50800b720 con 0x55917a70f060 2024-05-08T06:00:51.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.714+0000 7fd528424b80 10 client.25759 fetch_fsmap learned FSMap version 31 2024-05-08T06:00:51.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.714+0000 7fd528424b80 1 -- 172.21.15.112:0/169776503 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55917a793c40 con 0x55917a70f060 2024-05-08T06:00:51.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.714+0000 7fd500c00700 1 -- 172.21.15.112:0/169776503 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd50800bdd0 con 0x55917a70f060 2024-05-08T06:00:51.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.714+0000 7fd500c00700 1 -- 172.21.15.112:0/169776503 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fd508030070 con 0x55917a70f060 2024-05-08T06:00:51.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.714+0000 7fd528424b80 10 client.25759 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T06:00:51.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.714+0000 7fd528424b80 10 client.25759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:00:51.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.714+0000 7fd528424b80 20 client.25759 populate_metadata read hostname 'smithi112' 2024-05-08T06:00:51.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.714+0000 7fd528424b80 1 --2- 172.21.15.112:0/169776503 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55917a794340 0x55917a796760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:51.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.714+0000 7fd528424b80 4 client.25759 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T06:00:51.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.714+0000 7fd528424b80 1 -- 172.21.15.112:0/169776503 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55917a66f400 con 0x55917a794340 2024-05-08T06:00:51.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.715+0000 7fd516000700 1 --2- 172.21.15.112:0/169776503 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55917a794340 0x55917a796760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:51.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.716+0000 7fd516000700 1 --2- 172.21.15.112:0/169776503 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55917a794340 0x55917a796760 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd316954f931c0d3 server_cookie=5b19340a3aebce94 in_seq=0 out_seq=0 2024-05-08T06:00:51.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.716+0000 7fd500c00700 10 client.25759 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.716+0000 7fd500c00700 1 -- 172.21.15.112:0/169776503 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55917a66f400 con 0x55917a794340 2024-05-08T06:00:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.717+0000 7fd500c00700 10 client.25759 handle_command_reply: tid=0 2024-05-08T06:00:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.717+0000 7fd528424b80 1 client.25759 shutdown 2024-05-08T06:00:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.717+0000 7fd528424b80 1 -- 172.21.15.112:0/169776503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a70f060 msgr2=0x55917a720de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.717+0000 7fd528424b80 1 --2- 172.21.15.112:0/169776503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a70f060 0x55917a720de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd508003990 tx=0x7fd508005c00 comp rx=0 tx=0).stop 2024-05-08T06:00:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.717+0000 7fd528424b80 1 -- 172.21.15.112:0/169776503 shutdown_connections 2024-05-08T06:00:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.717+0000 7fd528424b80 1 --2- 172.21.15.112:0/169776503 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55917a794340 0x55917a796760 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.717+0000 7fd528424b80 1 --2- 172.21.15.112:0/169776503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55917a7139d0 0x55917a71f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.717+0000 7fd528424b80 1 --2- 172.21.15.112:0/169776503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55917a70f060 0x55917a720de0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.717+0000 7fd528424b80 1 --2- 172.21.15.112:0/169776503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55917a70f980 0x55917a721320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.717+0000 7fd528424b80 1 -- 172.21.15.112:0/169776503 >> 172.21.15.112:0/169776503 conn(0x55917a63db60 msgr2=0x55917a712c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:51.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.717+0000 7fd500c00700 0 client.25759 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:00:51.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.718+0000 7fd528424b80 1 -- 172.21.15.112:0/169776503 shutdown_connections 2024-05-08T06:00:51.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.718+0000 7fd528424b80 1 -- 172.21.15.112:0/169776503 wait complete. 2024-05-08T06:00:51.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.718+0000 7fd528424b80 20 client.25759 trim_cache size 0 max 16384 2024-05-08T06:00:51.719 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:00:51.720 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-08T06:00:51.720 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-08T06:00:51.720 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:00:51.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.719+0000 7fd51da00700 1 -- 172.21.15.112:0/2458882421 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4fc0b2900 msgr2=0x7fd4fc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:51.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.719+0000 7fd51da00700 1 --2- 172.21.15.112:0/2458882421 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4fc0b2900 0x7fd4fc0b4dc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd510018000 tx=0x7fd510014600 comp rx=0 tx=0).stop 2024-05-08T06:00:51.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.719+0000 7fd51da00700 1 -- 172.21.15.112:0/2458882421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd518103eb0 msgr2=0x7fd5181a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:51.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.719+0000 7fd51da00700 1 --2- 172.21.15.112:0/2458882421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd518103eb0 0x7fd5181a51e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd50400e9a0 tx=0x7fd50400ed60 comp rx=0 tx=0).stop 2024-05-08T06:00:51.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.719+0000 7fd51da00700 1 -- 172.21.15.112:0/2458882421 shutdown_connections 2024-05-08T06:00:51.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.719+0000 7fd51da00700 1 --2- 172.21.15.112:0/2458882421 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4fc0b2900 0x7fd4fc0b4dc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.719+0000 7fd51da00700 1 --2- 172.21.15.112:0/2458882421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd518109600 0x7fd51810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.719+0000 7fd51da00700 1 --2- 172.21.15.112:0/2458882421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5181034f0 0x7fd5181a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.719+0000 7fd51da00700 1 --2- 172.21.15.112:0/2458882421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd518103eb0 0x7fd5181a51e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:51.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.719+0000 7fd51da00700 1 -- 172.21.15.112:0/2458882421 >> 172.21.15.112:0/2458882421 conn(0x7fd51807a650 msgr2=0x7fd518101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:51.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.719+0000 7fd51da00700 1 -- 172.21.15.112:0/2458882421 shutdown_connections 2024-05-08T06:00:51.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:51.719+0000 7fd51da00700 1 -- 172.21.15.112:0/2458882421 wait complete. 2024-05-08T06:00:52.762 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T06:00:52.762 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:00:53.223 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:00:53.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:53 smithi112 ceph-mon[30003]: pgmap v608: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 244 op/s 2024-05-08T06:00:53.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:53 smithi162 ceph-mon[34800]: pgmap v608: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 244 op/s 2024-05-08T06:00:53.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:53 smithi150 ceph-mon[34315]: pgmap v608: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 244 op/s 2024-05-08T06:00:55.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:55 smithi112 ceph-mon[30003]: pgmap v609: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 251 op/s 2024-05-08T06:00:55.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:55 smithi162 ceph-mon[34800]: pgmap v609: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 251 op/s 2024-05-08T06:00:55.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:55 smithi150 ceph-mon[34315]: pgmap v609: 97 pgs: 97 active+clean; 96 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 251 op/s 2024-05-08T06:00:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.123+0000 7f699c200700 1 -- 172.21.15.112:0/2821246892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6994103520 msgr2=0x7f69941039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.123+0000 7f699c200700 1 --2- 172.21.15.112:0/2821246892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6994103520 0x7f69941039a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f69800098f0 tx=0x7f6980009c00 comp rx=0 tx=0).stop 2024-05-08T06:00:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.124+0000 7f699c200700 1 -- 172.21.15.112:0/2821246892 shutdown_connections 2024-05-08T06:00:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.124+0000 7f699c200700 1 --2- 172.21.15.112:0/2821246892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6994103ee0 0x7f6994110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.124+0000 7f699c200700 1 --2- 172.21.15.112:0/2821246892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6994109530 0x7f6994109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.124+0000 7f699c200700 1 --2- 172.21.15.112:0/2821246892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6994103520 0x7f69941039a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.124+0000 7f699c200700 1 -- 172.21.15.112:0/2821246892 >> 172.21.15.112:0/2821246892 conn(0x7f699407a570 msgr2=0x7f699407a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.124+0000 7f699c200700 1 -- 172.21.15.112:0/2821246892 shutdown_connections 2024-05-08T06:00:56.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.124+0000 7f699c200700 1 -- 172.21.15.112:0/2821246892 wait complete. 2024-05-08T06:00:56.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.125+0000 7f699c200700 1 Processor -- start 2024-05-08T06:00:56.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.125+0000 7f699c200700 1 -- start start 2024-05-08T06:00:56.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.126+0000 7f699c200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6994103520 0x7f69940757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:56.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.126+0000 7f699c200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6994103ee0 0x7f6994075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.126+0000 7f699c200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6994109530 0x7f6994076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.126+0000 7f699c200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f69941136b0 con 0x7f6994103520 2024-05-08T06:00:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.126+0000 7f699c200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6994113530 con 0x7f6994109530 2024-05-08T06:00:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.126+0000 7f699c200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6994113830 con 0x7f6994103ee0 2024-05-08T06:00:56.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.126+0000 7f6998e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6994103ee0 0x7f6994075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.126+0000 7f6999800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6994103520 0x7f69940757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.126+0000 7f6998e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6994103ee0 0x7f6994075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:45638/0 (socket says 172.21.15.112:45638) 2024-05-08T06:00:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.126+0000 7f6998e00700 1 -- 172.21.15.112:0/1046503750 learned_addr learned my addr 172.21.15.112:0/1046503750 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:00:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.126+0000 7f699a200700 1 --2- 172.21.15.112:0/1046503750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6994109530 0x7f6994076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.127+0000 7f6999800700 1 -- 172.21.15.112:0/1046503750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6994103ee0 msgr2=0x7f6994075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.127+0000 7f6999800700 1 --2- 172.21.15.112:0/1046503750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6994103ee0 0x7f6994075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.127+0000 7f6999800700 1 -- 172.21.15.112:0/1046503750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6994109530 msgr2=0x7f6994076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.127+0000 7f6999800700 1 --2- 172.21.15.112:0/1046503750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6994109530 0x7f6994076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.127+0000 7f6999800700 1 -- 172.21.15.112:0/1046503750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69941a8130 con 0x7f6994103520 2024-05-08T06:00:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.127+0000 7f6998e00700 1 --2- 172.21.15.112:0/1046503750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6994103ee0 0x7f6994075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:00:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.127+0000 7f699a200700 1 --2- 172.21.15.112:0/1046503750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6994109530 0x7f6994076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:00:56.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.127+0000 7f6999800700 1 --2- 172.21.15.112:0/1046503750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6994103520 0x7f69940757a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f698800e9c0 tx=0x7f698800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:56.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.127+0000 7f6992000700 1 -- 172.21.15.112:0/1046503750 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698800c9c0 con 0x7f6994103520 2024-05-08T06:00:56.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.128+0000 7f6992000700 1 -- 172.21.15.112:0/1046503750 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f698800cb60 con 0x7f6994103520 2024-05-08T06:00:56.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.128+0000 7f699c200700 1 -- 172.21.15.112:0/1046503750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69941a8420 con 0x7f6994103520 2024-05-08T06:00:56.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.128+0000 7f6992000700 1 -- 172.21.15.112:0/1046503750 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988018420 con 0x7f6994103520 2024-05-08T06:00:56.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.128+0000 7f699c200700 1 -- 172.21.15.112:0/1046503750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69941a8960 con 0x7f6994103520 2024-05-08T06:00:56.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.129+0000 7f699c200700 1 -- 172.21.15.112:0/1046503750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f699406a190 con 0x7f6994103520 2024-05-08T06:00:56.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.131+0000 7f6992000700 1 -- 172.21.15.112:0/1046503750 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6988010430 con 0x7f6994103520 2024-05-08T06:00:56.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.133+0000 7f6992000700 1 --2- 172.21.15.112:0/1046503750 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f697c0b2880 0x7f697c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:56.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.133+0000 7f6992000700 1 -- 172.21.15.112:0/1046503750 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f69880e2390 con 0x7f6994103520 2024-05-08T06:00:56.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.133+0000 7f6998e00700 1 --2- 172.21.15.112:0/1046503750 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f697c0b2880 0x7f697c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:56.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.134+0000 7f6998e00700 1 --2- 172.21.15.112:0/1046503750 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f697c0b2880 0x7f697c0b4d40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f699407ac90 tx=0x7f6980019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:56.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.137+0000 7f6992000700 1 -- 172.21.15.112:0/1046503750 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f69880aa020 con 0x7f6994103520 2024-05-08T06:00:56.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.453+0000 7f699c200700 1 -- 172.21.15.112:0/1046503750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f699402e510 con 0x7f6994103520 2024-05-08T06:00:56.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.454+0000 7f6992000700 1 -- 172.21.15.112:0/1046503750 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11895 (secure 0 0 0) 0x7f69880a9920 con 0x7f6994103520 2024-05-08T06:00:56.455 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T06:00:56.455 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T06:00:33.308080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24518,24511]},"id":1}]} 2024-05-08T06:00:56.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.457+0000 7f699c200700 1 -- 172.21.15.112:0/1046503750 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f697c0b2880 msgr2=0x7f697c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.457+0000 7f699c200700 1 --2- 172.21.15.112:0/1046503750 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f697c0b2880 0x7f697c0b4d40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f699407ac90 tx=0x7f6980019040 comp rx=0 tx=0).stop 2024-05-08T06:00:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.457+0000 7f699c200700 1 -- 172.21.15.112:0/1046503750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6994103520 msgr2=0x7f69940757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.457+0000 7f699c200700 1 --2- 172.21.15.112:0/1046503750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6994103520 0x7f69940757a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f698800e9c0 tx=0x7f698800ecd0 comp rx=0 tx=0).stop 2024-05-08T06:00:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.457+0000 7f699c200700 1 -- 172.21.15.112:0/1046503750 shutdown_connections 2024-05-08T06:00:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.457+0000 7f699c200700 1 --2- 172.21.15.112:0/1046503750 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f697c0b2880 0x7f697c0b4d40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.457+0000 7f699c200700 1 --2- 172.21.15.112:0/1046503750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6994109530 0x7f6994076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.457+0000 7f699c200700 1 --2- 172.21.15.112:0/1046503750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6994103520 0x7f69940757a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.457+0000 7f699c200700 1 --2- 172.21.15.112:0/1046503750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6994103ee0 0x7f6994075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.457+0000 7f699c200700 1 -- 172.21.15.112:0/1046503750 >> 172.21.15.112:0/1046503750 conn(0x7f699407a570 msgr2=0x7f6994102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.457+0000 7f699c200700 1 -- 172.21.15.112:0/1046503750 shutdown_connections 2024-05-08T06:00:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:00:56.457+0000 7f699c200700 1 -- 172.21.15.112:0/1046503750 wait complete. 2024-05-08T06:00:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 31 2024-05-08T06:00:57.424 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-08T06:00:57.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:57 smithi150 ceph-mon[34315]: pgmap v610: 97 pgs: 97 active+clean; 100 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 374 op/s 2024-05-08T06:00:57.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:57 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1046503750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:00:57.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:57 smithi112 ceph-mon[30003]: pgmap v610: 97 pgs: 97 active+clean; 100 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 374 op/s 2024-05-08T06:00:57.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:57 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1046503750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:00:57.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:57 smithi162 ceph-mon[34800]: pgmap v610: 97 pgs: 97 active+clean; 100 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 374 op/s 2024-05-08T06:00:57.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:57 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1046503750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:00:57.880 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:00:59.143 INFO:tasks.workunit.client.0.smithi112.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-05-08T06:00:59.143 INFO:tasks.workunit.client.0.smithi112.stdout:All 50000 operations completed A-OK! 2024-05-08T06:00:59.153 INFO:tasks.workunit.client.0.smithi112.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-05-08T06:00:59.157 INFO:tasks.workunit.client.0.smithi112.stdout:Seed set to 1 2024-05-08T06:00:59.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:00:59 smithi150 ceph-mon[34315]: pgmap v611: 97 pgs: 97 active+clean; 100 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 299 op/s 2024-05-08T06:00:59.774 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:00:59 smithi112 ceph-mon[30003]: pgmap v611: 97 pgs: 97 active+clean; 100 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 299 op/s 2024-05-08T06:00:59.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:00:59 smithi162 ceph-mon[34800]: pgmap v611: 97 pgs: 97 active+clean; 100 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 299 op/s 2024-05-08T06:01:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.301+0000 7fc347200700 1 -- 172.21.15.112:0/1957509923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc348102b30 msgr2=0x7fc34810f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.301+0000 7fc347200700 1 --2- 172.21.15.112:0/1957509923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc348102b30 0x7fc34810f7f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc334009990 tx=0x7fc334009ca0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.302+0000 7fc347200700 1 -- 172.21.15.112:0/1957509923 shutdown_connections 2024-05-08T06:01:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.302+0000 7fc347200700 1 --2- 172.21.15.112:0/1957509923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc34810fd30 0x7fc348112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.302+0000 7fc347200700 1 --2- 172.21.15.112:0/1957509923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc348102180 0x7fc348102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.302+0000 7fc347200700 1 --2- 172.21.15.112:0/1957509923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc348102b30 0x7fc34810f7f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.302+0000 7fc347200700 1 -- 172.21.15.112:0/1957509923 >> 172.21.15.112:0/1957509923 conn(0x7fc3480fddc0 msgr2=0x7fc3481001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.302+0000 7fc347200700 1 -- 172.21.15.112:0/1957509923 shutdown_connections 2024-05-08T06:01:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.302+0000 7fc347200700 1 -- 172.21.15.112:0/1957509923 wait complete. 2024-05-08T06:01:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.303+0000 7fc347200700 1 Processor -- start 2024-05-08T06:01:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.303+0000 7fc347200700 1 -- start start 2024-05-08T06:01:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.303+0000 7fc347200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc348102180 0x7fc348104460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.303+0000 7fc347200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc348102b30 0x7fc3481049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.303+0000 7fc347200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc34810fd30 0x7fc348104ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.303+0000 7fc347200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc348115800 con 0x7fc34810fd30 2024-05-08T06:01:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.303+0000 7fc347200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc348115680 con 0x7fc348102b30 2024-05-08T06:01:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.303+0000 7fc347200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc348115980 con 0x7fc348102180 2024-05-08T06:01:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.304+0000 7fc345e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc348102180 0x7fc348104460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.304+0000 7fc345e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc348102180 0x7fc348104460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:45650/0 (socket says 172.21.15.112:45650) 2024-05-08T06:01:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.304+0000 7fc345e00700 1 -- 172.21.15.112:0/3247717823 learned_addr learned my addr 172.21.15.112:0/3247717823 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.304+0000 7fc346800700 1 --2- 172.21.15.112:0/3247717823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc34810fd30 0x7fc348104ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.304+0000 7fc345400700 1 --2- 172.21.15.112:0/3247717823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc348102b30 0x7fc3481049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.304+0000 7fc345e00700 1 -- 172.21.15.112:0/3247717823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc348102b30 msgr2=0x7fc3481049a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.304+0000 7fc345e00700 1 --2- 172.21.15.112:0/3247717823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc348102b30 0x7fc3481049a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.304+0000 7fc345e00700 1 -- 172.21.15.112:0/3247717823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc34810fd30 msgr2=0x7fc348104ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.304+0000 7fc345e00700 1 --2- 172.21.15.112:0/3247717823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc34810fd30 0x7fc348104ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.304+0000 7fc345e00700 1 -- 172.21.15.112:0/3247717823 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3481ac360 con 0x7fc348102180 2024-05-08T06:01:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.304+0000 7fc346800700 1 --2- 172.21.15.112:0/3247717823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc34810fd30 0x7fc348104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:01:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.304+0000 7fc345e00700 1 --2- 172.21.15.112:0/3247717823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc348102180 0x7fc348104460 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc33800eb60 tx=0x7fc33800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.305+0000 7fc33ea00700 1 -- 172.21.15.112:0/3247717823 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33800cd40 con 0x7fc348102180 2024-05-08T06:01:00.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.305+0000 7fc33ea00700 1 -- 172.21.15.112:0/3247717823 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3380044d0 con 0x7fc348102180 2024-05-08T06:01:00.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.305+0000 7fc33ea00700 1 -- 172.21.15.112:0/3247717823 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc338010690 con 0x7fc348102180 2024-05-08T06:01:00.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.305+0000 7fc347200700 1 -- 172.21.15.112:0/3247717823 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3481ac650 con 0x7fc348102180 2024-05-08T06:01:00.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.305+0000 7fc347200700 1 -- 172.21.15.112:0/3247717823 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3481acb90 con 0x7fc348102180 2024-05-08T06:01:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.310+0000 7fc33ea00700 1 -- 172.21.15.112:0/3247717823 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc3380037a0 con 0x7fc348102180 2024-05-08T06:01:00.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.310+0000 7fc3572c0b80 1 Processor -- start 2024-05-08T06:01:00.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.311+0000 7fc3572c0b80 1 -- start start 2024-05-08T06:01:00.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.311+0000 7fc3572c0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070c0da0 0x55d7070c1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.311+0000 7fc3572c0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070c16c0 0x55d7070bb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.311+0000 7fc3572c0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070bba50 0x55d7070bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.311+0000 7fc3572c0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d70700da20 con 0x55d7070c16c0 2024-05-08T06:01:00.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.311+0000 7fc3572c0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d706fab760 con 0x55d7070c0da0 2024-05-08T06:01:00.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.311+0000 7fc3572c0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d7070099e0 con 0x55d7070bba50 2024-05-08T06:01:00.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.312+0000 7fc346800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070c0da0 0x55d7070c1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.312+0000 7fc346800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070c0da0 0x55d7070c1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:49216/0 (socket says 172.21.15.112:49216) 2024-05-08T06:01:00.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.312+0000 7fc346800700 1 -- 172.21.15.112:0/328780619 learned_addr learned my addr 172.21.15.112:0/328780619 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:00.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.312+0000 7fc345400700 1 --2- 172.21.15.112:0/328780619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070bba50 0x55d7070bbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.312+0000 7fc345e00700 1 --2- 172.21.15.112:0/328780619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070c16c0 0x55d7070bb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.312+0000 7fc33ea00700 1 --2- 172.21.15.112:0/3247717823 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc32c0b2880 0x7fc32c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.312+0000 7fc345400700 1 -- 172.21.15.112:0/328780619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070c0da0 msgr2=0x55d7070c1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:00.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.312+0000 7fc345400700 1 --2- 172.21.15.112:0/328780619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070c0da0 0x55d7070c1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.312+0000 7fc345400700 1 -- 172.21.15.112:0/328780619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070c16c0 msgr2=0x55d7070bb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:00.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.312+0000 7fc33ea00700 1 -- 172.21.15.112:0/3247717823 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fc3380e25b0 con 0x7fc348102180 2024-05-08T06:01:00.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.312+0000 7fc345400700 1 --2- 172.21.15.112:0/328780619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070c16c0 0x55d7070bb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.312+0000 7fc345400700 1 -- 172.21.15.112:0/328780619 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d706f5a8d0 con 0x55d7070bba50 2024-05-08T06:01:00.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.313+0000 7fc345e00700 1 --2- 172.21.15.112:0/328780619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070c16c0 0x55d7070bb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:01:00.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.313+0000 7fc346800700 1 --2- 172.21.15.112:0/3247717823 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc32c0b2880 0x7fc32c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.313+0000 7fc345400700 1 --2- 172.21.15.112:0/328780619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070bba50 0x55d7070bbed0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc348105cb0 tx=0x7fc334005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c73e67cc89b59a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:00.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.313+0000 7fc333400700 1 -- 172.21.15.112:0/328780619 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33400bb80 con 0x55d7070bba50 2024-05-08T06:01:00.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.313+0000 7fc333400700 1 -- 172.21.15.112:0/328780619 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc33400bd20 con 0x55d7070bba50 2024-05-08T06:01:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.313+0000 7fc333400700 1 -- 172.21.15.112:0/328780619 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334003c80 con 0x55d7070bba50 2024-05-08T06:01:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.313+0000 7fc346800700 1 --2- 172.21.15.112:0/3247717823 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc32c0b2880 0x7fc32c0b4d40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc340009fd0 tx=0x7fc340004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.314+0000 7fc3572c0b80 1 -- 172.21.15.112:0/328780619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070bba50 msgr2=0x55d7070bbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.314+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/328780619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070bba50 0x55d7070bbed0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc348105cb0 tx=0x7fc334005f00 comp rx=0 tx=0).stop 2024-05-08T06:01:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.314+0000 7fc3572c0b80 1 -- 172.21.15.112:0/328780619 shutdown_connections 2024-05-08T06:01:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.314+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/328780619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070bba50 0x55d7070bbed0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.314+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/328780619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070c0da0 0x55d7070c1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.314+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/328780619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070c16c0 0x55d7070bb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.314+0000 7fc3572c0b80 1 -- 172.21.15.112:0/328780619 >> 172.21.15.112:0/328780619 conn(0x55d7070336d0 msgr2=0x55d7070b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.315+0000 7fc3572c0b80 1 -- 172.21.15.112:0/328780619 shutdown_connections 2024-05-08T06:01:00.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.315+0000 7fc3572c0b80 1 -- 172.21.15.112:0/328780619 wait complete. 2024-05-08T06:01:00.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.315+0000 7fc3572c0b80 1 -- start start 2024-05-08T06:01:00.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.315+0000 7fc3572c0b80 1 Processor -- start 2024-05-08T06:01:00.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.316+0000 7fc3572c0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070bba50 0x55d7070c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.316+0000 7fc3572c0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070c0da0 0x55d7070c4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.316+0000 7fc3572c0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 0x55d7070c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.316+0000 7fc3572c0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d7070c8090 con 0x55d7070bba50 2024-05-08T06:01:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.316+0000 7fc3572c0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d706f73b20 con 0x55d7070c0da0 2024-05-08T06:01:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.316+0000 7fc3572c0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d7070c8210 con 0x55d7070c16c0 2024-05-08T06:01:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.316+0000 7fc3572c0b80 10 client.0 _ll_register_callbacks cb 0x55d706fb4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:01:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.316+0000 7fc345e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070bba50 0x55d7070c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.316+0000 7fc345e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070bba50 0x55d7070c73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40440/0 (socket says 172.21.15.112:40440) 2024-05-08T06:01:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.316+0000 7fc345e00700 1 -- 172.21.15.112:0/4235917838 learned_addr learned my addr 172.21.15.112:0/4235917838 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.316+0000 7fc345400700 1 --2- 172.21.15.112:0/4235917838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070c0da0 0x55d7070c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.316+0000 7fc346800700 1 --2- 172.21.15.112:0/4235917838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 0x55d7070c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.317+0000 7fc345400700 1 -- 172.21.15.112:0/4235917838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 msgr2=0x55d7070c4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.317+0000 7fc345400700 1 --2- 172.21.15.112:0/4235917838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 0x55d7070c4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.317+0000 7fc345400700 1 -- 172.21.15.112:0/4235917838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070bba50 msgr2=0x55d7070c73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.317+0000 7fc345400700 1 --2- 172.21.15.112:0/4235917838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070bba50 0x55d7070c73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.317+0000 7fc345400700 1 -- 172.21.15.112:0/4235917838 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d706fd3420 con 0x55d7070c0da0 2024-05-08T06:01:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.317+0000 7fc345e00700 1 --2- 172.21.15.112:0/4235917838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070bba50 0x55d7070c73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:01:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.317+0000 7fc345400700 1 --2- 172.21.15.112:0/4235917838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070c0da0 0x55d7070c4750 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc334006010 tx=0x7fc3340096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb31a9e2864dd6c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.317+0000 7fc327e00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:01:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.317+0000 7fc327e00700 1 -- 172.21.15.112:0/4235917838 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334031060 con 0x55d7070c0da0 2024-05-08T06:01:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.317+0000 7fc327e00700 1 -- 172.21.15.112:0/4235917838 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fc334003680 con 0x55d7070c0da0 2024-05-08T06:01:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.317+0000 7fc3572c0b80 1 -- 172.21.15.112:0/4235917838 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc33400bd20 con 0x55d7070c0da0 2024-05-08T06:01:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.318+0000 7fc327e00700 1 -- 172.21.15.112:0/4235917838 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc3340054d0 con 0x55d7070c0da0 2024-05-08T06:01:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.318+0000 7fc3572c0b80 10 client.25739 fetch_fsmap learned FSMap version 31 2024-05-08T06:01:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.318+0000 7fc3572c0b80 1 -- 172.21.15.112:0/4235917838 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d7070d28b0 con 0x55d7070c0da0 2024-05-08T06:01:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.318+0000 7fc327e00700 1 -- 172.21.15.112:0/4235917838 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc33400bd20 con 0x55d7070c0da0 2024-05-08T06:01:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.318+0000 7fc327e00700 1 -- 172.21.15.112:0/4235917838 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fc33400fab0 con 0x55d7070c0da0 2024-05-08T06:01:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.318+0000 7fc3572c0b80 10 client.25739 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T06:01:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.319+0000 7fc3572c0b80 10 client.25739 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:01:00.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.319+0000 7fc3572c0b80 20 client.25739 populate_metadata read hostname 'smithi112' 2024-05-08T06:01:00.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.319+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/4235917838 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d7070d4d80 0x55d7070d7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.319+0000 7fc3572c0b80 4 client.25739 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:01:00.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.319+0000 7fc3572c0b80 1 -- 172.21.15.112:0/4235917838 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d707088140 con 0x55d7070d4d80 2024-05-08T06:01:00.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.319+0000 7fc345e00700 1 --2- 172.21.15.112:0/4235917838 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d7070d4d80 0x55d7070d7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.319+0000 7fc345e00700 1 --2- 172.21.15.112:0/4235917838 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d7070d4d80 0x55d7070d7170 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3201d53164dae1a server_cookie=3369607dc8d71675 in_seq=0 out_seq=0 2024-05-08T06:01:00.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.319+0000 7fc327e00700 10 client.25739 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T06:01:00.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.320+0000 7fc327e00700 1 -- 172.21.15.112:0/4235917838 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d707088140 con 0x55d7070d4d80 2024-05-08T06:01:00.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.320+0000 7fc327e00700 10 client.25739 handle_command_reply: tid=0 2024-05-08T06:01:00.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.320+0000 7fc3572c0b80 1 client.25739 shutdown 2024-05-08T06:01:00.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.320+0000 7fc3572c0b80 1 -- 172.21.15.112:0/4235917838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070c0da0 msgr2=0x55d7070c4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:00.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.320+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/4235917838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070c0da0 0x55d7070c4750 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc334006010 tx=0x7fc3340096e0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.321+0000 7fc3572c0b80 1 -- 172.21.15.112:0/4235917838 shutdown_connections 2024-05-08T06:01:00.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.321+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/4235917838 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d7070d4d80 0x55d7070d7170 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.321+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/4235917838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 0x55d7070c4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.321+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/4235917838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070bba50 0x55d7070c73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.321+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/4235917838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070c0da0 0x55d7070c4750 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.321+0000 7fc3572c0b80 1 -- 172.21.15.112:0/4235917838 >> 172.21.15.112:0/4235917838 conn(0x55d7070336d0 msgr2=0x55d7070b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:00.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.321+0000 7fc327e00700 0 client.25739 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T06:01:00.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.321+0000 7fc3572c0b80 1 -- 172.21.15.112:0/4235917838 shutdown_connections 2024-05-08T06:01:00.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.321+0000 7fc3572c0b80 1 -- 172.21.15.112:0/4235917838 wait complete. 2024-05-08T06:01:00.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.321+0000 7fc3572c0b80 20 client.25739 trim_cache size 0 max 16384 2024-05-08T06:01:00.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.337+0000 7fc3572c0b80 1 Processor -- start 2024-05-08T06:01:00.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.337+0000 7fc3572c0b80 1 -- start start 2024-05-08T06:01:00.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.338+0000 7fc3572c0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070bba50 0x55d7070bb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.338+0000 7fc3572c0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070c0da0 0x55d7070b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.338+0000 7fc3572c0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 0x55d7070c3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.338+0000 7fc3572c0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3481b68d0 con 0x55d7070c0da0 2024-05-08T06:01:00.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.338+0000 7fc3572c0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc3481b4100 con 0x55d7070bba50 2024-05-08T06:01:00.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.338+0000 7fc3572c0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc3481b1f50 con 0x55d7070c16c0 2024-05-08T06:01:00.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.338+0000 7fc345400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070c0da0 0x55d7070b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.338+0000 7fc345400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070c0da0 0x55d7070b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40450/0 (socket says 172.21.15.112:40450) 2024-05-08T06:01:00.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.338+0000 7fc345400700 1 -- 172.21.15.112:0/1753693895 learned_addr learned my addr 172.21.15.112:0/1753693895 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:00.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.338+0000 7fc345e00700 1 --2- 172.21.15.112:0/1753693895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070bba50 0x55d7070bb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.338+0000 7fc346800700 1 --2- 172.21.15.112:0/1753693895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 0x55d7070c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.338+0000 7fc345400700 1 -- 172.21.15.112:0/1753693895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 msgr2=0x55d7070c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:00.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.339+0000 7fc345400700 1 --2- 172.21.15.112:0/1753693895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 0x55d7070c3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.339+0000 7fc345400700 1 -- 172.21.15.112:0/1753693895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070bba50 msgr2=0x55d7070bb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:00.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.339+0000 7fc345400700 1 --2- 172.21.15.112:0/1753693895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070bba50 0x55d7070bb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.339+0000 7fc345400700 1 -- 172.21.15.112:0/1753693895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7070b6b90 con 0x55d7070c0da0 2024-05-08T06:01:00.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.339+0000 7fc345e00700 1 --2- 172.21.15.112:0/1753693895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070bba50 0x55d7070bb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:01:00.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.339+0000 7fc345400700 1 --2- 172.21.15.112:0/1753693895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070c0da0 0x55d7070b0860 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc33400fe50 tx=0x7fc334022630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114e94ceb6dd0298 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:00.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.339+0000 7fc332a00700 1 -- 172.21.15.112:0/1753693895 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334031040 con 0x55d7070c0da0 2024-05-08T06:01:00.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.339+0000 7fc332a00700 1 -- 172.21.15.112:0/1753693895 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc334022e00 con 0x55d7070c0da0 2024-05-08T06:01:00.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.339+0000 7fc332a00700 1 -- 172.21.15.112:0/1753693895 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33402c420 con 0x55d7070c0da0 2024-05-08T06:01:00.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.340+0000 7fc3572c0b80 1 -- 172.21.15.112:0/1753693895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070c0da0 msgr2=0x55d7070b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:00.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.340+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/1753693895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070c0da0 0x55d7070b0860 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc33400fe50 tx=0x7fc334022630 comp rx=0 tx=0).stop 2024-05-08T06:01:00.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.340+0000 7fc3572c0b80 1 -- 172.21.15.112:0/1753693895 shutdown_connections 2024-05-08T06:01:00.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.340+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/1753693895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 0x55d7070c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.340+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/1753693895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070bba50 0x55d7070bb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.340+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/1753693895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070c0da0 0x55d7070b0860 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.340+0000 7fc3572c0b80 1 -- 172.21.15.112:0/1753693895 >> 172.21.15.112:0/1753693895 conn(0x55d7070336d0 msgr2=0x55d7070c4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:00.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.340+0000 7fc3572c0b80 1 -- 172.21.15.112:0/1753693895 shutdown_connections 2024-05-08T06:01:00.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.340+0000 7fc3572c0b80 1 -- 172.21.15.112:0/1753693895 wait complete. 2024-05-08T06:01:00.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.341+0000 7fc3572c0b80 1 -- start start 2024-05-08T06:01:00.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.341+0000 7fc3572c0b80 1 Processor -- start 2024-05-08T06:01:00.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.341+0000 7fc3572c0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070bba50 0x55d7070d6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.341+0000 7fc3572c0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 0x55d7070d72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.341+0000 7fc3572c0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070d7810 0x55d70713d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.341+0000 7fc3572c0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3481b25c0 con 0x55d7070bba50 2024-05-08T06:01:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.341+0000 7fc3572c0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc3481b3a30 con 0x55d7070d7810 2024-05-08T06:01:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.341+0000 7fc3572c0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc3481b2c90 con 0x55d7070c16c0 2024-05-08T06:01:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.341+0000 7fc3572c0b80 10 client.0 _ll_register_callbacks cb 0x55d706fb4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:01:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.342+0000 7fc345400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 0x55d7070d72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.342+0000 7fc345e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070bba50 0x55d7070d6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.342+0000 7fc345400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 0x55d7070d72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:45686/0 (socket says 172.21.15.112:45686) 2024-05-08T06:01:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.342+0000 7fc345400700 1 -- 172.21.15.112:0/2638952326 learned_addr learned my addr 172.21.15.112:0/2638952326 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.342+0000 7fc346800700 1 --2- 172.21.15.112:0/2638952326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070d7810 0x55d70713d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.342+0000 7fc345400700 1 -- 172.21.15.112:0/2638952326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070d7810 msgr2=0x55d70713d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.342+0000 7fc345400700 1 --2- 172.21.15.112:0/2638952326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070d7810 0x55d70713d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.342+0000 7fc345400700 1 -- 172.21.15.112:0/2638952326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070bba50 msgr2=0x55d7070d6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.342+0000 7fc345400700 1 --2- 172.21.15.112:0/2638952326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070bba50 0x55d7070d6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.342+0000 7fc345400700 1 -- 172.21.15.112:0/2638952326 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d70713d820 con 0x55d7070c16c0 2024-05-08T06:01:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.342+0000 7fc345e00700 1 --2- 172.21.15.112:0/2638952326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070bba50 0x55d7070d6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:01:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.342+0000 7fc346800700 1 --2- 172.21.15.112:0/2638952326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070d7810 0x55d70713d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:01:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.342+0000 7fc345400700 1 --2- 172.21.15.112:0/2638952326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 0x55d7070d72d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc33400b2e0 tx=0x7fc33400bdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbf39de89e3e48b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.342+0000 7fc327e00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:01:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.343+0000 7fc327e00700 1 -- 172.21.15.112:0/2638952326 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334031040 con 0x55d7070c16c0 2024-05-08T06:01:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.343+0000 7fc327e00700 1 -- 172.21.15.112:0/2638952326 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fc33400f460 con 0x55d7070c16c0 2024-05-08T06:01:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.343+0000 7fc3572c0b80 1 -- 172.21.15.112:0/2638952326 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc334022e00 con 0x55d7070c16c0 2024-05-08T06:01:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.343+0000 7fc327e00700 1 -- 172.21.15.112:0/2638952326 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc3340052d0 con 0x55d7070c16c0 2024-05-08T06:01:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.343+0000 7fc3572c0b80 10 client.25783 fetch_fsmap learned FSMap version 31 2024-05-08T06:01:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.343+0000 7fc3572c0b80 1 -- 172.21.15.112:0/2638952326 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d70713eaa0 con 0x55d7070c16c0 2024-05-08T06:01:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.343+0000 7fc327e00700 1 -- 172.21.15.112:0/2638952326 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc334022e00 con 0x55d7070c16c0 2024-05-08T06:01:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.344+0000 7fc327e00700 1 -- 172.21.15.112:0/2638952326 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fc334049430 con 0x55d7070c16c0 2024-05-08T06:01:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.344+0000 7fc3572c0b80 10 client.25783 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T06:01:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.344+0000 7fc3572c0b80 10 client.25783 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:01:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.344+0000 7fc3572c0b80 20 client.25783 populate_metadata read hostname 'smithi112' 2024-05-08T06:01:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.344+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/2638952326 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d70713ef00 0x55d7071412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.344+0000 7fc3572c0b80 4 client.25783 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:01:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.344+0000 7fc3572c0b80 1 -- 172.21.15.112:0/2638952326 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d70701b400 con 0x55d70713ef00 2024-05-08T06:01:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.344+0000 7fc345e00700 1 --2- 172.21.15.112:0/2638952326 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d70713ef00 0x55d7071412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.344+0000 7fc345e00700 1 --2- 172.21.15.112:0/2638952326 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d70713ef00 0x55d7071412f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=471584340ee0228b server_cookie=e2c9ee146336c88b in_seq=0 out_seq=0 2024-05-08T06:01:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.344+0000 7fc327e00700 10 client.25783 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.345+0000 7fc327e00700 1 -- 172.21.15.112:0/2638952326 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d70701b400 con 0x55d70713ef00 2024-05-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.345+0000 7fc327e00700 10 client.25783 handle_command_reply: tid=0 2024-05-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.345+0000 7fc3572c0b80 1 client.25783 shutdown 2024-05-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.345+0000 7fc3572c0b80 1 -- 172.21.15.112:0/2638952326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 msgr2=0x55d7070d72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.345+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/2638952326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 0x55d7070d72d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc33400b2e0 tx=0x7fc33400bdb0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.345+0000 7fc3572c0b80 1 -- 172.21.15.112:0/2638952326 shutdown_connections 2024-05-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.345+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/2638952326 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d70713ef00 0x55d7071412f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.345+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/2638952326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7070d7810 0x55d70713d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.345+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/2638952326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7070bba50 0x55d7070d6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.345+0000 7fc3572c0b80 1 --2- 172.21.15.112:0/2638952326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d7070c16c0 0x55d7070d72d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.345+0000 7fc3572c0b80 1 -- 172.21.15.112:0/2638952326 >> 172.21.15.112:0/2638952326 conn(0x55d7070336d0 msgr2=0x55d7070c1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.345+0000 7fc327e00700 0 client.25783 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.345+0000 7fc3572c0b80 1 -- 172.21.15.112:0/2638952326 shutdown_connections 2024-05-08T06:01:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.345+0000 7fc3572c0b80 1 -- 172.21.15.112:0/2638952326 wait complete. 2024-05-08T06:01:00.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.345+0000 7fc3572c0b80 20 client.25783 trim_cache size 0 max 16384 2024-05-08T06:01:00.350 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T06:01:00.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.346+0000 7fc347200700 1 -- 172.21.15.112:0/3247717823 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc32c0b2880 msgr2=0x7fc32c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.346+0000 7fc347200700 1 --2- 172.21.15.112:0/3247717823 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc32c0b2880 0x7fc32c0b4d40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc340009fd0 tx=0x7fc340004ad0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.346+0000 7fc347200700 1 -- 172.21.15.112:0/3247717823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc348102180 msgr2=0x7fc348104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.346+0000 7fc347200700 1 --2- 172.21.15.112:0/3247717823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc348102180 0x7fc348104460 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc33800eb60 tx=0x7fc33800ef20 comp rx=0 tx=0).stop 2024-05-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.346+0000 7fc347200700 1 -- 172.21.15.112:0/3247717823 shutdown_connections 2024-05-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.346+0000 7fc347200700 1 --2- 172.21.15.112:0/3247717823 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc32c0b2880 0x7fc32c0b4d40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.346+0000 7fc347200700 1 --2- 172.21.15.112:0/3247717823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc34810fd30 0x7fc348104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.346+0000 7fc347200700 1 --2- 172.21.15.112:0/3247717823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc348102180 0x7fc348104460 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.346+0000 7fc347200700 1 --2- 172.21.15.112:0/3247717823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc348102b30 0x7fc3481049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.346+0000 7fc347200700 1 -- 172.21.15.112:0/3247717823 >> 172.21.15.112:0/3247717823 conn(0x7fc3480fddc0 msgr2=0x7fc3481001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.346+0000 7fc347200700 1 -- 172.21.15.112:0/3247717823 shutdown_connections 2024-05-08T06:01:00.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:00.346+0000 7fc347200700 1 -- 172.21.15.112:0/3247717823 wait complete. 2024-05-08T06:01:00.558 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-05-08T06:01:00.558 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-05-08T06:01:00.561 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-05-08T06:01:00.561 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-05-08T06:01:00.561 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support clone range, disabling! 2024-05-08T06:01:00.566 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support dedupe range, disabling! 2024-05-08T06:01:00.571 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero size read 2024-05-08T06:01:00.571 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support exchange range, disabling! 2024-05-08T06:01:00.593 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x1c37b90 2024-05-08T06:01:00.606 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x5bd330d 2024-05-08T06:01:00.638 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x637bf99 2024-05-08T06:01:01.048 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x63d9b9c 2024-05-08T06:01:01.201 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-08T06:01:01.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:01 smithi112 ceph-mon[30003]: pgmap v612: 97 pgs: 97 active+clean; 102 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.4 MiB/s wr, 410 op/s 2024-05-08T06:01:01.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:01 smithi162 ceph-mon[34800]: pgmap v612: 97 pgs: 97 active+clean; 102 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.4 MiB/s wr, 410 op/s 2024-05-08T06:01:01.676 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:01:01.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:01 smithi150 ceph-mon[34315]: pgmap v612: 97 pgs: 97 active+clean; 102 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.4 MiB/s wr, 410 op/s 2024-05-08T06:01:02.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:02 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:01:02.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:02 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:01:02.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:02 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:01:03.564 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:03 smithi112 ceph-mon[30003]: pgmap v613: 97 pgs: 97 active+clean; 111 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 319 op/s 2024-05-08T06:01:03.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:03 smithi150 ceph-mon[34315]: pgmap v613: 97 pgs: 97 active+clean; 111 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 319 op/s 2024-05-08T06:01:03.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:03 smithi162 ceph-mon[34800]: pgmap v613: 97 pgs: 97 active+clean; 111 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 319 op/s 2024-05-08T06:01:04.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.139+0000 7fb9c1000700 1 -- 172.21.15.112:0/3980007826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc1083d0 msgr2=0x7fb9bc06b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:04.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.139+0000 7fb9c1000700 1 --2- 172.21.15.112:0/3980007826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc1083d0 0x7fb9bc06b1b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009990 tx=0x7fb9ac009ca0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.139+0000 7fb9c1000700 1 -- 172.21.15.112:0/3980007826 shutdown_connections 2024-05-08T06:01:04.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.139+0000 7fb9c1000700 1 --2- 172.21.15.112:0/3980007826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9bc105ee0 0x7fb9bc106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.139+0000 7fb9c1000700 1 --2- 172.21.15.112:0/3980007826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc1083d0 0x7fb9bc06b1b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.139+0000 7fb9c1000700 1 --2- 172.21.15.112:0/3980007826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc06b6f0 0x7fb9bc06bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.139+0000 7fb9c1000700 1 -- 172.21.15.112:0/3980007826 >> 172.21.15.112:0/3980007826 conn(0x7fb9bc079350 msgr2=0x7fb9bc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:04.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.139+0000 7fb9c1000700 1 -- 172.21.15.112:0/3980007826 shutdown_connections 2024-05-08T06:01:04.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.139+0000 7fb9c1000700 1 -- 172.21.15.112:0/3980007826 wait complete. 2024-05-08T06:01:04.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.140+0000 7fb9c1000700 1 Processor -- start 2024-05-08T06:01:04.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.140+0000 7fb9c1000700 1 -- start start 2024-05-08T06:01:04.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.141+0000 7fb9c1000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc06b6f0 0x7fb9bc1a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.141+0000 7fb9c1000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9bc105ee0 0x7fb9bc1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.141+0000 7fb9c1000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc1083d0 0x7fb9bc1a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.141+0000 7fb9c1000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc119c80 con 0x7fb9bc06b6f0 2024-05-08T06:01:04.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.141+0000 7fb9c1000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc119b00 con 0x7fb9bc1083d0 2024-05-08T06:01:04.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.141+0000 7fb9c1000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc119e00 con 0x7fb9bc105ee0 2024-05-08T06:01:04.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.141+0000 7fb9b9400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9bc105ee0 0x7fb9bc1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.141+0000 7fb9ba800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc1083d0 0x7fb9bc1a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.141+0000 7fb9b9e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc06b6f0 0x7fb9bc1a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.141+0000 7fb9b9400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9bc105ee0 0x7fb9bc1a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38492/0 (socket says 172.21.15.112:38492) 2024-05-08T06:01:04.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.141+0000 7fb9ba800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc1083d0 0x7fb9bc1a4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:49314/0 (socket says 172.21.15.112:49314) 2024-05-08T06:01:04.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.141+0000 7fb9b9400700 1 -- 172.21.15.112:0/841438216 learned_addr learned my addr 172.21.15.112:0/841438216 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:04.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.142+0000 7fb9b9400700 1 -- 172.21.15.112:0/841438216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc1083d0 msgr2=0x7fb9bc1a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:04.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.142+0000 7fb9b9400700 1 --2- 172.21.15.112:0/841438216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc1083d0 0x7fb9bc1a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.142+0000 7fb9b9400700 1 -- 172.21.15.112:0/841438216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc06b6f0 msgr2=0x7fb9bc1a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:04.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.142+0000 7fb9b9400700 1 --2- 172.21.15.112:0/841438216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc06b6f0 0x7fb9bc1a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.142+0000 7fb9b9400700 1 -- 172.21.15.112:0/841438216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9bc1a5200 con 0x7fb9bc105ee0 2024-05-08T06:01:04.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.142+0000 7fb9ba800700 1 --2- 172.21.15.112:0/841438216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc1083d0 0x7fb9bc1a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:01:04.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.142+0000 7fb9b9400700 1 --2- 172.21.15.112:0/841438216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9bc105ee0 0x7fb9bc1a0730 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800e9a0 tx=0x7fb9a800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:04.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.142+0000 7fb9b2a00700 1 -- 172.21.15.112:0/841438216 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a800ca10 con 0x7fb9bc105ee0 2024-05-08T06:01:04.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.143+0000 7fb9b2a00700 1 -- 172.21.15.112:0/841438216 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9a800cbb0 con 0x7fb9bc105ee0 2024-05-08T06:01:04.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.143+0000 7fb9c1000700 1 -- 172.21.15.112:0/841438216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9bc1a54f0 con 0x7fb9bc105ee0 2024-05-08T06:01:04.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.143+0000 7fb9b2a00700 1 -- 172.21.15.112:0/841438216 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8018680 con 0x7fb9bc105ee0 2024-05-08T06:01:04.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.143+0000 7fb9c1000700 1 -- 172.21.15.112:0/841438216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9bc1acd70 con 0x7fb9bc105ee0 2024-05-08T06:01:04.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.147+0000 7fb9b2a00700 1 -- 172.21.15.112:0/841438216 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb9a8010430 con 0x7fb9bc105ee0 2024-05-08T06:01:04.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.148+0000 7fb9cb975b80 1 Processor -- start 2024-05-08T06:01:04.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.148+0000 7fb9cb975b80 1 -- start start 2024-05-08T06:01:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.148+0000 7fb9cb975b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d11060 0x562b00d11440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.148+0000 7fb9cb975b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11980 0x562b00d15490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.148+0000 7fb9cb975b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d159d0 0x562b00d15e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.148+0000 7fb9cb975b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562b00c63a20 con 0x562b00d11980 2024-05-08T06:01:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.148+0000 7fb9cb975b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562b00c01760 con 0x562b00d11060 2024-05-08T06:01:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.148+0000 7fb9cb975b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562b00c5f9e0 con 0x562b00d159d0 2024-05-08T06:01:04.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.149+0000 7fb9b9e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11980 0x562b00d15490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.149+0000 7fb9ba800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d11060 0x562b00d11440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.149+0000 7fb9b9e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11980 0x562b00d15490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36718/0 (socket says 172.21.15.112:36718) 2024-05-08T06:01:04.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.149+0000 7fb9b9e00700 1 -- 172.21.15.112:0/1654665629 learned_addr learned my addr 172.21.15.112:0/1654665629 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:04.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.149+0000 7fb9b9400700 1 --2- 172.21.15.112:0/1654665629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d159d0 0x562b00d15e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.149+0000 7fb9ba800700 1 -- 172.21.15.112:0/1654665629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d159d0 msgr2=0x562b00d15e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:04.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.149+0000 7fb9ba800700 1 --2- 172.21.15.112:0/1654665629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d159d0 0x562b00d15e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.149+0000 7fb9ba800700 1 -- 172.21.15.112:0/1654665629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11980 msgr2=0x562b00d15490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:04.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.149+0000 7fb9ba800700 1 --2- 172.21.15.112:0/1654665629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11980 0x562b00d15490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.149+0000 7fb9ba800700 1 -- 172.21.15.112:0/1654665629 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b00bb08d0 con 0x562b00d11060 2024-05-08T06:01:04.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.149+0000 7fb9b9e00700 1 --2- 172.21.15.112:0/1654665629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11980 0x562b00d15490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:01:04.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.149+0000 7fb9b2a00700 1 --2- 172.21.15.112:0/841438216 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb9a00b2900 0x7fb9a00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.150+0000 7fb9b9e00700 1 --2- 172.21.15.112:0/841438216 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb9a00b2900 0x7fb9a00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.150+0000 7fb9b2a00700 1 -- 172.21.15.112:0/841438216 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fb9a80e2330 con 0x7fb9bc105ee0 2024-05-08T06:01:04.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.150+0000 7fb9ba800700 1 --2- 172.21.15.112:0/1654665629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d11060 0x562b00d11440 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4003db0 tx=0x7fb9b4009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc9671fd7fd47a69 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:04.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.150+0000 7fb9a7400700 1 -- 172.21.15.112:0/1654665629 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4019030 con 0x562b00d11060 2024-05-08T06:01:04.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.150+0000 7fb9a7400700 1 -- 172.21.15.112:0/1654665629 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9b4004bb0 con 0x562b00d11060 2024-05-08T06:01:04.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.150+0000 7fb9a7400700 1 -- 172.21.15.112:0/1654665629 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4007d30 con 0x562b00d11060 2024-05-08T06:01:04.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.151+0000 7fb9cb975b80 1 -- 172.21.15.112:0/1654665629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d11060 msgr2=0x562b00d11440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:04.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.151+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/1654665629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d11060 0x562b00d11440 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4003db0 tx=0x7fb9b4009920 comp rx=0 tx=0).stop 2024-05-08T06:01:04.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.151+0000 7fb9b9e00700 1 --2- 172.21.15.112:0/841438216 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb9a00b2900 0x7fb9a00b4dc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac015000 tx=0x7fb9ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:04.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.151+0000 7fb9cb975b80 1 -- 172.21.15.112:0/1654665629 shutdown_connections 2024-05-08T06:01:04.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.151+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/1654665629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d159d0 0x562b00d15e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.151+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/1654665629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d11060 0x562b00d11440 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.151+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/1654665629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11980 0x562b00d15490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.151+0000 7fb9cb975b80 1 -- 172.21.15.112:0/1654665629 >> 172.21.15.112:0/1654665629 conn(0x562b00c3fb60 msgr2=0x562b00d0b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:04.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.152+0000 7fb9cb975b80 1 -- 172.21.15.112:0/1654665629 shutdown_connections 2024-05-08T06:01:04.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.152+0000 7fb9cb975b80 1 -- 172.21.15.112:0/1654665629 wait complete. 2024-05-08T06:01:04.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.152+0000 7fb9cb975b80 1 -- start start 2024-05-08T06:01:04.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.153+0000 7fb9cb975b80 1 Processor -- start 2024-05-08T06:01:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.153+0000 7fb9cb975b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 0x562b00d1ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.153+0000 7fb9cb975b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d11980 0x562b00d1a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.153+0000 7fb9cb975b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d159d0 0x562b00d1a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.153+0000 7fb9cb975b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562b00d1dbe0 con 0x562b00d11060 2024-05-08T06:01:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.153+0000 7fb9cb975b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562b00bc9b20 con 0x562b00d159d0 2024-05-08T06:01:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.153+0000 7fb9cb975b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562b00d1dd60 con 0x562b00d11980 2024-05-08T06:01:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.153+0000 7fb9cb975b80 10 client.0 _ll_register_callbacks cb 0x562b00c0ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:01:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.154+0000 7fb9b9400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d11980 0x562b00d1a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.154+0000 7fb9b9400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d11980 0x562b00d1a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38512/0 (socket says 172.21.15.112:38512) 2024-05-08T06:01:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.154+0000 7fb9b9400700 1 -- 172.21.15.112:0/2167154866 learned_addr learned my addr 172.21.15.112:0/2167154866 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.154+0000 7fb9b9400700 1 -- 172.21.15.112:0/2167154866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d159d0 msgr2=0x562b00d1a770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T06:01:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.154+0000 7fb9ba800700 1 --2- 172.21.15.112:0/2167154866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d159d0 0x562b00d1a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.154+0000 7fb9b9400700 1 --2- 172.21.15.112:0/2167154866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d159d0 0x562b00d1a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.154+0000 7fb9b9400700 1 -- 172.21.15.112:0/2167154866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 msgr2=0x562b00d1ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.154+0000 7fb9b9e00700 1 --2- 172.21.15.112:0/2167154866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 0x562b00d1ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.155+0000 7fb9b9400700 1 --2- 172.21.15.112:0/2167154866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 0x562b00d1ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.155+0000 7fb9b9400700 1 -- 172.21.15.112:0/2167154866 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b00c29420 con 0x562b00d11980 2024-05-08T06:01:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.155+0000 7fb9ba800700 1 --2- 172.21.15.112:0/2167154866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d159d0 0x562b00d1a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:01:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.155+0000 7fb9b9e00700 1 --2- 172.21.15.112:0/2167154866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 0x562b00d1ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:01:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.155+0000 7fb9b9400700 1 --2- 172.21.15.112:0/2167154866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d11980 0x562b00d1a230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80a59a0 tx=0x7fb9a80e5f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c2041b942f55b09 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.155+0000 7fb99be00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:01:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.155+0000 7fb99be00700 1 -- 172.21.15.112:0/2167154866 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a80b20c0 con 0x562b00d11980 2024-05-08T06:01:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.155+0000 7fb99be00700 1 -- 172.21.15.112:0/2167154866 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fb9a80cabd0 con 0x562b00d11980 2024-05-08T06:01:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.155+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2167154866 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b4004bb0 con 0x562b00d11980 2024-05-08T06:01:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.156+0000 7fb99be00700 1 -- 172.21.15.112:0/2167154866 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb9a80a9430 con 0x562b00d11980 2024-05-08T06:01:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.156+0000 7fb9cb975b80 10 client.25795 fetch_fsmap learned FSMap version 32 2024-05-08T06:01:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.156+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2167154866 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562b00d284f0 con 0x562b00d11980 2024-05-08T06:01:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.156+0000 7fb99be00700 1 -- 172.21.15.112:0/2167154866 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9b4004bb0 con 0x562b00d11980 2024-05-08T06:01:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.156+0000 7fb99be00700 1 -- 172.21.15.112:0/2167154866 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 32) ==== 8342+0+0 (secure 0 0 0) 0x7fb9a80b22f0 con 0x562b00d11980 2024-05-08T06:01:04.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.156+0000 7fb9cb975b80 10 client.25795 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T06:01:04.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.156+0000 7fb9cb975b80 10 client.25795 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T06:01:04.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.156+0000 7fb9cb975b80 20 client.25795 populate_metadata read hostname 'smithi112' 2024-05-08T06:01:04.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.156+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/2167154866 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x562b00d2aa80 0x562b00d2ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.156+0000 7fb9cb975b80 4 client.25795 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:01:04.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.156+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2167154866 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b00cde0a0 con 0x562b00d2aa80 2024-05-08T06:01:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.157+0000 7fb9b9e00700 1 --2- 172.21.15.112:0/2167154866 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x562b00d2aa80 0x562b00d2ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.158+0000 7fb9b9e00700 1 --2- 172.21.15.112:0/2167154866 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x562b00d2aa80 0x562b00d2ce70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86c63a51e527277e server_cookie=d206d38a343c2bd0 in_seq=0 out_seq=0 2024-05-08T06:01:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.158+0000 7fb99be00700 10 client.25795 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T06:01:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.159+0000 7fb99be00700 1 -- 172.21.15.112:0/2167154866 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b00cde0a0 con 0x562b00d2aa80 2024-05-08T06:01:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.159+0000 7fb99be00700 10 client.25795 handle_command_reply: tid=0 2024-05-08T06:01:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.159+0000 7fb9cb975b80 1 client.25795 shutdown 2024-05-08T06:01:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.159+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2167154866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d11980 msgr2=0x562b00d1a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.159+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/2167154866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d11980 0x562b00d1a230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80a59a0 tx=0x7fb9a80e5f40 comp rx=0 tx=0).stop 2024-05-08T06:01:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.159+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2167154866 shutdown_connections 2024-05-08T06:01:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.159+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/2167154866 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x562b00d2aa80 0x562b00d2ce70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.159+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/2167154866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d159d0 0x562b00d1a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.159+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/2167154866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 0x562b00d1ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.159+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/2167154866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d11980 0x562b00d1a230 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.159+0000 7fb99be00700 0 client.25795 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T06:01:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.159+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2167154866 >> 172.21.15.112:0/2167154866 conn(0x562b00c3fb60 msgr2=0x562b00d062b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:04.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.160+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2167154866 shutdown_connections 2024-05-08T06:01:04.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.160+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2167154866 wait complete. 2024-05-08T06:01:04.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.160+0000 7fb9cb975b80 20 client.25795 trim_cache size 0 max 16384 2024-05-08T06:01:04.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.178+0000 7fb9cb975b80 1 Processor -- start 2024-05-08T06:01:04.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.178+0000 7fb9cb975b80 1 -- start start 2024-05-08T06:01:04.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.178+0000 7fb9cb975b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 0x562b00d26c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.178+0000 7fb9cb975b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d11980 0x562b00d06130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.178+0000 7fb9cb975b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d159d0 0x562b00d1b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.178+0000 7fb9cb975b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1b6b90 con 0x562b00d11060 2024-05-08T06:01:04.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.178+0000 7fb9cb975b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1b4420 con 0x562b00d159d0 2024-05-08T06:01:04.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.178+0000 7fb9cb975b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1b2270 con 0x562b00d11980 2024-05-08T06:01:04.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.179+0000 7fb9b9400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d11980 0x562b00d06130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.179+0000 7fb9b9400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d11980 0x562b00d06130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38516/0 (socket says 172.21.15.112:38516) 2024-05-08T06:01:04.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.179+0000 7fb9b9400700 1 -- 172.21.15.112:0/508593119 learned_addr learned my addr 172.21.15.112:0/508593119 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:04.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.179+0000 7fb9b9e00700 1 --2- 172.21.15.112:0/508593119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 0x562b00d26c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.179+0000 7fb9ba800700 1 --2- 172.21.15.112:0/508593119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d159d0 0x562b00d1b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.179+0000 7fb9ba800700 1 -- 172.21.15.112:0/508593119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d11980 msgr2=0x562b00d06130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:04.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.179+0000 7fb9ba800700 1 --2- 172.21.15.112:0/508593119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d11980 0x562b00d06130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.179+0000 7fb9ba800700 1 -- 172.21.15.112:0/508593119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 msgr2=0x562b00d26c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:04.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.179+0000 7fb9ba800700 1 --2- 172.21.15.112:0/508593119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 0x562b00d26c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.179+0000 7fb9ba800700 1 -- 172.21.15.112:0/508593119 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b00d0c610 con 0x562b00d159d0 2024-05-08T06:01:04.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.180+0000 7fb9ba800700 1 --2- 172.21.15.112:0/508593119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d159d0 0x562b00d1b8e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4003d80 tx=0x7fb9b4009ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1f25a79745e1def server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:04.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.181+0000 7fb9a6a00700 1 -- 172.21.15.112:0/508593119 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40092d0 con 0x562b00d159d0 2024-05-08T06:01:04.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.181+0000 7fb9a6a00700 1 -- 172.21.15.112:0/508593119 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9b40079b0 con 0x562b00d159d0 2024-05-08T06:01:04.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.181+0000 7fb9a6a00700 1 -- 172.21.15.112:0/508593119 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b400ea70 con 0x562b00d159d0 2024-05-08T06:01:04.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.183+0000 7fb9cb975b80 1 -- 172.21.15.112:0/508593119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d159d0 msgr2=0x562b00d1b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:04.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.183+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/508593119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d159d0 0x562b00d1b8e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4003d80 tx=0x7fb9b4009ac0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.184+0000 7fb9cb975b80 1 -- 172.21.15.112:0/508593119 shutdown_connections 2024-05-08T06:01:04.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.184+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/508593119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d159d0 0x562b00d1b8e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.184+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/508593119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 0x562b00d26c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.184+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/508593119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d11980 0x562b00d06130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.184+0000 7fb9cb975b80 1 -- 172.21.15.112:0/508593119 >> 172.21.15.112:0/508593119 conn(0x562b00c3fb60 msgr2=0x562b00d05a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:04.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.184+0000 7fb9cb975b80 1 -- 172.21.15.112:0/508593119 shutdown_connections 2024-05-08T06:01:04.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.184+0000 7fb9cb975b80 1 -- 172.21.15.112:0/508593119 wait complete. 2024-05-08T06:01:04.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.185+0000 7fb9cb975b80 1 -- start start 2024-05-08T06:01:04.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.185+0000 7fb9cb975b80 1 Processor -- start 2024-05-08T06:01:04.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9cb975b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 0x562b00d19910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9cb975b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d11980 0x562b00d19e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9cb975b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d159d0 0x562b00d1a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9cb975b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1b28e0 con 0x562b00d11060 2024-05-08T06:01:04.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9cb975b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1b3d50 con 0x562b00d11980 2024-05-08T06:01:04.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9cb975b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1b2fb0 con 0x562b00d159d0 2024-05-08T06:01:04.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9cb975b80 10 client.0 _ll_register_callbacks cb 0x562b00c0ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:01:04.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9ba800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d159d0 0x562b00d1a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9ba800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d159d0 0x562b00d1a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38530/0 (socket says 172.21.15.112:38530) 2024-05-08T06:01:04.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9ba800700 1 -- 172.21.15.112:0/2245492349 learned_addr learned my addr 172.21.15.112:0/2245492349 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:04.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9b9400700 1 --2- 172.21.15.112:0/2245492349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d11980 0x562b00d19e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9b9e00700 1 --2- 172.21.15.112:0/2245492349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 0x562b00d19910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9ba800700 1 -- 172.21.15.112:0/2245492349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d11980 msgr2=0x562b00d19e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:04.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9ba800700 1 --2- 172.21.15.112:0/2245492349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d11980 0x562b00d19e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9ba800700 1 -- 172.21.15.112:0/2245492349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 msgr2=0x562b00d19910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:04.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9ba800700 1 --2- 172.21.15.112:0/2245492349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 0x562b00d19910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.186+0000 7fb9ba800700 1 -- 172.21.15.112:0/2245492349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b00d083f0 con 0x562b00d159d0 2024-05-08T06:01:04.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.187+0000 7fb9b9400700 1 --2- 172.21.15.112:0/2245492349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d11980 0x562b00d19e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:01:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.187+0000 7fb9ba800700 1 --2- 172.21.15.112:0/2245492349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d159d0 0x562b00d1a390 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40046b0 tx=0x7fb9b4009380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de5ad1e43fac0e1f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.187+0000 7fb99be00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:01:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.187+0000 7fb99be00700 1 -- 172.21.15.112:0/2245492349 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b402c030 con 0x562b00d159d0 2024-05-08T06:01:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.187+0000 7fb99be00700 1 -- 172.21.15.112:0/2245492349 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fb9b40076f0 con 0x562b00d159d0 2024-05-08T06:01:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.188+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2245492349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b40079b0 con 0x562b00d159d0 2024-05-08T06:01:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.188+0000 7fb99be00700 1 -- 172.21.15.112:0/2245492349 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb9b400e670 con 0x562b00d159d0 2024-05-08T06:01:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.188+0000 7fb9cb975b80 10 client.25801 fetch_fsmap learned FSMap version 32 2024-05-08T06:01:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.188+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2245492349 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562b00da6390 con 0x562b00d159d0 2024-05-08T06:01:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.188+0000 7fb99be00700 1 -- 172.21.15.112:0/2245492349 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9b40079b0 con 0x562b00d159d0 2024-05-08T06:01:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.188+0000 7fb99be00700 1 -- 172.21.15.112:0/2245492349 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 32) ==== 8342+0+0 (secure 0 0 0) 0x7fb9b4024020 con 0x562b00d159d0 2024-05-08T06:01:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.188+0000 7fb9cb975b80 10 client.25801 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T06:01:04.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.188+0000 7fb9cb975b80 10 client.25801 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T06:01:04.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.188+0000 7fb9cb975b80 20 client.25801 populate_metadata read hostname 'smithi112' 2024-05-08T06:01:04.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.188+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/2245492349 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x562b00da6820 0x562b00da8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:04.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.189+0000 7fb9cb975b80 4 client.25801 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:01:04.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.189+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2245492349 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b00c71400 con 0x562b00da6820 2024-05-08T06:01:04.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.189+0000 7fb9b9e00700 1 --2- 172.21.15.112:0/2245492349 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x562b00da6820 0x562b00da8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:04.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.189+0000 7fb9b9e00700 1 --2- 172.21.15.112:0/2245492349 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x562b00da6820 0x562b00da8c40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73b4baad6dc67d95 server_cookie=d7d8e83c638d84e2 in_seq=0 out_seq=0 2024-05-08T06:01:04.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.189+0000 7fb99be00700 10 client.25801 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T06:01:04.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.190+0000 7fb99be00700 1 -- 172.21.15.112:0/2245492349 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b00c71400 con 0x562b00da6820 2024-05-08T06:01:04.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.190+0000 7fb99be00700 10 client.25801 handle_command_reply: tid=0 2024-05-08T06:01:04.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.190+0000 7fb9cb975b80 1 client.25801 shutdown 2024-05-08T06:01:04.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.190+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2245492349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d159d0 msgr2=0x562b00d1a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:04.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.190+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/2245492349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d159d0 0x562b00d1a390 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40046b0 tx=0x7fb9b4009380 comp rx=0 tx=0).stop 2024-05-08T06:01:04.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.190+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2245492349 shutdown_connections 2024-05-08T06:01:04.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.190+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/2245492349 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x562b00da6820 0x562b00da8c40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.190+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/2245492349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b00d159d0 0x562b00d1a390 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.190+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/2245492349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b00d11060 0x562b00d19910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.190+0000 7fb9cb975b80 1 --2- 172.21.15.112:0/2245492349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b00d11980 0x562b00d19e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.190+0000 7fb99be00700 0 client.25801 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T06:01:04.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.190+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2245492349 >> 172.21.15.112:0/2245492349 conn(0x562b00c3fb60 msgr2=0x562b00d0fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:04.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.190+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2245492349 shutdown_connections 2024-05-08T06:01:04.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.190+0000 7fb9cb975b80 1 -- 172.21.15.112:0/2245492349 wait complete. 2024-05-08T06:01:04.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.190+0000 7fb9cb975b80 20 client.25801 trim_cache size 0 max 16384 2024-05-08T06:01:04.192 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T06:01:04.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.191+0000 7fb9c1000700 1 -- 172.21.15.112:0/841438216 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb9a00b2900 msgr2=0x7fb9a00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:04.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.191+0000 7fb9c1000700 1 --2- 172.21.15.112:0/841438216 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb9a00b2900 0x7fb9a00b4dc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac015000 tx=0x7fb9ac005fd0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.191+0000 7fb9c1000700 1 -- 172.21.15.112:0/841438216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9bc105ee0 msgr2=0x7fb9bc1a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:04.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.191+0000 7fb9c1000700 1 --2- 172.21.15.112:0/841438216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9bc105ee0 0x7fb9bc1a0730 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800e9a0 tx=0x7fb9a800ecb0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.192+0000 7fb9c1000700 1 -- 172.21.15.112:0/841438216 shutdown_connections 2024-05-08T06:01:04.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.192+0000 7fb9c1000700 1 --2- 172.21.15.112:0/841438216 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb9a00b2900 0x7fb9a00b4dc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.192+0000 7fb9c1000700 1 --2- 172.21.15.112:0/841438216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9bc1083d0 0x7fb9bc1a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.192+0000 7fb9c1000700 1 --2- 172.21.15.112:0/841438216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc06b6f0 0x7fb9bc1a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.192+0000 7fb9c1000700 1 --2- 172.21.15.112:0/841438216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9bc105ee0 0x7fb9bc1a0730 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:04.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.192+0000 7fb9c1000700 1 -- 172.21.15.112:0/841438216 >> 172.21.15.112:0/841438216 conn(0x7fb9bc079350 msgr2=0x7fb9bc101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:04.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.193+0000 7fb9c1000700 1 -- 172.21.15.112:0/841438216 shutdown_connections 2024-05-08T06:01:04.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:04.193+0000 7fb9c1000700 1 -- 172.21.15.112:0/841438216 wait complete. 2024-05-08T06:01:05.457 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-08T06:01:05.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:05 smithi150 ceph-mon[34315]: pgmap v614: 97 pgs: 97 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2024-05-08T06:01:05.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:05 smithi112 ceph-mon[30003]: pgmap v614: 97 pgs: 97 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2024-05-08T06:01:05.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:05 smithi162 ceph-mon[34800]: pgmap v614: 97 pgs: 97 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2024-05-08T06:01:05.942 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:01:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:07 smithi150 ceph-mon[34315]: pgmap v615: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.1 MiB/s wr, 419 op/s 2024-05-08T06:01:07.794 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:07 smithi112 ceph-mon[30003]: pgmap v615: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.1 MiB/s wr, 419 op/s 2024-05-08T06:01:07.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:07 smithi162 ceph-mon[34800]: pgmap v615: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.1 MiB/s wr, 419 op/s 2024-05-08T06:01:08.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.420+0000 7f4a6e000700 1 -- 172.21.15.112:0/1117175428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a68108850 msgr2=0x7f4a681103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:08.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.420+0000 7f4a6e000700 1 --2- 172.21.15.112:0/1117175428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a68108850 0x7f4a681103d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009fc0 tx=0x7f4a6000b330 comp rx=0 tx=0).stop 2024-05-08T06:01:08.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.420+0000 7f4a6e000700 1 -- 172.21.15.112:0/1117175428 shutdown_connections 2024-05-08T06:01:08.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.420+0000 7f4a6e000700 1 --2- 172.21.15.112:0/1117175428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a68108850 0x7f4a681103d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.420+0000 7f4a6e000700 1 --2- 172.21.15.112:0/1117175428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a68103750 0x7f4a68103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.420+0000 7f4a6e000700 1 --2- 172.21.15.112:0/1117175428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a68104100 0x7f4a68108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.420+0000 7f4a6e000700 1 -- 172.21.15.112:0/1117175428 >> 172.21.15.112:0/1117175428 conn(0x7f4a6807a600 msgr2=0x7f4a6807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:08.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.421+0000 7f4a6e000700 1 -- 172.21.15.112:0/1117175428 shutdown_connections 2024-05-08T06:01:08.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.421+0000 7f4a6e000700 1 -- 172.21.15.112:0/1117175428 wait complete. 2024-05-08T06:01:08.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.423+0000 7f4a6e000700 1 Processor -- start 2024-05-08T06:01:08.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.423+0000 7f4a6e000700 1 -- start start 2024-05-08T06:01:08.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.423+0000 7f4a6e000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a68103750 0x7f4a6810f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.424+0000 7f4a6e000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a68104100 0x7f4a6810fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.424+0000 7f4a6e000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a68108850 0x7f4a681a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.424+0000 7f4a6e000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68113670 con 0x7f4a68108850 2024-05-08T06:01:08.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.424+0000 7f4a6e000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681134f0 con 0x7f4a68104100 2024-05-08T06:01:08.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.424+0000 7f4a6e000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681137f0 con 0x7f4a68103750 2024-05-08T06:01:08.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.424+0000 7f4a67400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a68103750 0x7f4a6810f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.424+0000 7f4a67400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a68103750 0x7f4a6810f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38544/0 (socket says 172.21.15.112:38544) 2024-05-08T06:01:08.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.424+0000 7f4a67400700 1 -- 172.21.15.112:0/890740327 learned_addr learned my addr 172.21.15.112:0/890740327 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:08.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.424+0000 7f4a66a00700 1 --2- 172.21.15.112:0/890740327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a68104100 0x7f4a6810fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.424+0000 7f4a66a00700 1 -- 172.21.15.112:0/890740327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a68103750 msgr2=0x7f4a6810f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:08.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.424+0000 7f4a66a00700 1 --2- 172.21.15.112:0/890740327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a68103750 0x7f4a6810f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.424+0000 7f4a66a00700 1 -- 172.21.15.112:0/890740327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a68108850 msgr2=0x7f4a681a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:08.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.424+0000 7f4a66a00700 1 --2- 172.21.15.112:0/890740327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a68108850 0x7f4a681a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.424+0000 7f4a66a00700 1 -- 172.21.15.112:0/890740327 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a681a5040 con 0x7f4a68104100 2024-05-08T06:01:08.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.425+0000 7f4a66a00700 1 --2- 172.21.15.112:0/890740327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a68104100 0x7f4a6810fb00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400e9c0 tx=0x7f4a5400ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:08.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.425+0000 7f4a5fe00700 1 -- 172.21.15.112:0/890740327 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400cbd0 con 0x7f4a68104100 2024-05-08T06:01:08.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.425+0000 7f4a5fe00700 1 -- 172.21.15.112:0/890740327 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a54009e50 con 0x7f4a68104100 2024-05-08T06:01:08.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.425+0000 7f4a6e000700 1 -- 172.21.15.112:0/890740327 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a681a5330 con 0x7f4a68104100 2024-05-08T06:01:08.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.425+0000 7f4a5fe00700 1 -- 172.21.15.112:0/890740327 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54018680 con 0x7f4a68104100 2024-05-08T06:01:08.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.425+0000 7f4a6e000700 1 -- 172.21.15.112:0/890740327 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a681acbb0 con 0x7f4a68104100 2024-05-08T06:01:08.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.429+0000 7f4a5fe00700 1 -- 172.21.15.112:0/890740327 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4a54010430 con 0x7f4a68104100 2024-05-08T06:01:08.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.430+0000 7f4a7889eb80 1 Processor -- start 2024-05-08T06:01:08.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.430+0000 7f4a7889eb80 1 -- start start 2024-05-08T06:01:08.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.431+0000 7f4a7889eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345060 0x564d0b345440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.431+0000 7f4a7889eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345980 0x564d0b349490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.431+0000 7f4a7889eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b3499d0 0x564d0b349e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.431+0000 7f4a7889eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564d0b297a20 con 0x564d0b345980 2024-05-08T06:01:08.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.431+0000 7f4a7889eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564d0b235760 con 0x564d0b345060 2024-05-08T06:01:08.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.431+0000 7f4a7889eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x564d0b2939e0 con 0x564d0b3499d0 2024-05-08T06:01:08.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.431+0000 7f4a66a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b3499d0 0x564d0b349e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.431+0000 7f4a66a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b3499d0 0x564d0b349e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38546/0 (socket says 172.21.15.112:38546) 2024-05-08T06:01:08.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.431+0000 7f4a66a00700 1 -- 172.21.15.112:0/2414582773 learned_addr learned my addr 172.21.15.112:0/2414582773 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:08.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.431+0000 7f4a5fe00700 1 --2- 172.21.15.112:0/890740327 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4a4c0b2900 0x7f4a4c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.432+0000 7f4a67e00700 1 --2- 172.21.15.112:0/2414582773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345060 0x564d0b345440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.432+0000 7f4a66a00700 1 -- 172.21.15.112:0/2414582773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345060 msgr2=0x564d0b345440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.432+0000 7f4a66a00700 1 --2- 172.21.15.112:0/2414582773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345060 0x564d0b345440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.432+0000 7f4a66a00700 1 -- 172.21.15.112:0/2414582773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345980 msgr2=0x564d0b349490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.432+0000 7f4a66a00700 1 --2- 172.21.15.112:0/2414582773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345980 0x564d0b349490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.432+0000 7f4a66a00700 1 -- 172.21.15.112:0/2414582773 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0b1e48d0 con 0x564d0b3499d0 2024-05-08T06:01:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.432+0000 7f4a67e00700 1 --2- 172.21.15.112:0/2414582773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345060 0x564d0b345440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:01:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.432+0000 7f4a66a00700 1 --2- 172.21.15.112:0/2414582773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b3499d0 0x564d0b349e30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4a540ea560 tx=0x7f4a540ea870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd4795201804305d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.432+0000 7f4a5cc00700 1 -- 172.21.15.112:0/2414582773 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540e45e0 con 0x564d0b3499d0 2024-05-08T06:01:08.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.433+0000 7f4a5cc00700 1 -- 172.21.15.112:0/2414582773 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a540e2ca0 con 0x564d0b3499d0 2024-05-08T06:01:08.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.433+0000 7f4a5cc00700 1 -- 172.21.15.112:0/2414582773 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540f1c90 con 0x564d0b3499d0 2024-05-08T06:01:08.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.433+0000 7f4a67e00700 1 --2- 172.21.15.112:0/890740327 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4a4c0b2900 0x7f4a4c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.434+0000 7f4a7889eb80 1 -- 172.21.15.112:0/2414582773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b3499d0 msgr2=0x564d0b349e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:08.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.434+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/2414582773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b3499d0 0x564d0b349e30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4a540ea560 tx=0x7f4a540ea870 comp rx=0 tx=0).stop 2024-05-08T06:01:08.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.434+0000 7f4a5fe00700 1 -- 172.21.15.112:0/890740327 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f4a540e2400 con 0x7f4a68104100 2024-05-08T06:01:08.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.435+0000 7f4a7889eb80 1 -- 172.21.15.112:0/2414582773 shutdown_connections 2024-05-08T06:01:08.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.435+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/2414582773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b3499d0 0x564d0b349e30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.435+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/2414582773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345060 0x564d0b345440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.435+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/2414582773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345980 0x564d0b349490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.435+0000 7f4a7889eb80 1 -- 172.21.15.112:0/2414582773 >> 172.21.15.112:0/2414582773 conn(0x564d0b273b60 msgr2=0x564d0b33f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:08.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.435+0000 7f4a67e00700 1 --2- 172.21.15.112:0/890740327 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4a4c0b2900 0x7f4a4c0b4dc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4a60006070 tx=0x7f4a60009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:08.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.435+0000 7f4a7889eb80 1 -- 172.21.15.112:0/2414582773 shutdown_connections 2024-05-08T06:01:08.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.436+0000 7f4a7889eb80 1 -- 172.21.15.112:0/2414582773 wait complete. 2024-05-08T06:01:08.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.436+0000 7f4a7889eb80 1 -- start start 2024-05-08T06:01:08.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.437+0000 7f4a7889eb80 1 Processor -- start 2024-05-08T06:01:08.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.437+0000 7f4a7889eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b345060 0x564d0b350f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.437+0000 7f4a7889eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345980 0x564d0b34e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.437+0000 7f4a7889eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b3499d0 0x564d0b34e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.437+0000 7f4a7889eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564d0b351be0 con 0x564d0b345980 2024-05-08T06:01:08.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.437+0000 7f4a7889eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564d0b1fdb20 con 0x564d0b3499d0 2024-05-08T06:01:08.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.437+0000 7f4a7889eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x564d0b351d60 con 0x564d0b345060 2024-05-08T06:01:08.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.437+0000 7f4a7889eb80 10 client.0 _ll_register_callbacks cb 0x564d0b23ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:01:08.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.438+0000 7f4a67400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b345060 0x564d0b350f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.438+0000 7f4a67400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b345060 0x564d0b350f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38558/0 (socket says 172.21.15.112:38558) 2024-05-08T06:01:08.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.438+0000 7f4a67400700 1 -- 172.21.15.112:0/4015804133 learned_addr learned my addr 172.21.15.112:0/4015804133 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:08.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.438+0000 7f4a67e00700 1 --2- 172.21.15.112:0/4015804133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b3499d0 0x564d0b34e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.438+0000 7f4a66a00700 1 --2- 172.21.15.112:0/4015804133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345980 0x564d0b34e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.438+0000 7f4a67400700 1 -- 172.21.15.112:0/4015804133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b3499d0 msgr2=0x564d0b34e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:08.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.438+0000 7f4a67400700 1 --2- 172.21.15.112:0/4015804133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b3499d0 0x564d0b34e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.438+0000 7f4a67400700 1 -- 172.21.15.112:0/4015804133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345980 msgr2=0x564d0b34e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:08.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.438+0000 7f4a67400700 1 --2- 172.21.15.112:0/4015804133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345980 0x564d0b34e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.438+0000 7f4a67400700 1 -- 172.21.15.112:0/4015804133 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0b25d420 con 0x564d0b345060 2024-05-08T06:01:08.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.438+0000 7f4a67e00700 1 --2- 172.21.15.112:0/4015804133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b3499d0 0x564d0b34e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:01:08.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.439+0000 7f4a66a00700 1 --2- 172.21.15.112:0/4015804133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345980 0x564d0b34e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:01:08.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.439+0000 7f4a67400700 1 --2- 172.21.15.112:0/4015804133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b345060 0x564d0b350f00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4a58011690 tx=0x7f4a580119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47f78e8bfb8228e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:08.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.439+0000 7f4a51600700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:01:08.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.439+0000 7f4a51600700 1 -- 172.21.15.112:0/4015804133 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58012860 con 0x564d0b345060 2024-05-08T06:01:08.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.439+0000 7f4a51600700 1 -- 172.21.15.112:0/4015804133 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f4a58012a00 con 0x564d0b345060 2024-05-08T06:01:08.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.439+0000 7f4a7889eb80 1 -- 172.21.15.112:0/4015804133 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a540e2ca0 con 0x564d0b345060 2024-05-08T06:01:08.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.440+0000 7f4a51600700 1 -- 172.21.15.112:0/4015804133 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4a58012c00 con 0x564d0b345060 2024-05-08T06:01:08.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.440+0000 7f4a7889eb80 10 client.25813 fetch_fsmap learned FSMap version 32 2024-05-08T06:01:08.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.440+0000 7f4a7889eb80 1 -- 172.21.15.112:0/4015804133 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d0b35c4f0 con 0x564d0b345060 2024-05-08T06:01:08.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.440+0000 7f4a51600700 1 -- 172.21.15.112:0/4015804133 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a540e2ca0 con 0x564d0b345060 2024-05-08T06:01:08.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.440+0000 7f4a51600700 1 -- 172.21.15.112:0/4015804133 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 32) ==== 8342+0+0 (secure 0 0 0) 0x7f4a5801a3f0 con 0x564d0b345060 2024-05-08T06:01:08.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.440+0000 7f4a7889eb80 10 client.25813 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T06:01:08.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.440+0000 7f4a7889eb80 10 client.25813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:01:08.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.440+0000 7f4a7889eb80 20 client.25813 populate_metadata read hostname 'smithi112' 2024-05-08T06:01:08.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.440+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/4015804133 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x564d0b35eb30 0x564d0b360f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.441+0000 7f4a7889eb80 4 client.25813 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:01:08.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.441+0000 7f4a7889eb80 1 -- 172.21.15.112:0/4015804133 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d0b3120a0 con 0x564d0b35eb30 2024-05-08T06:01:08.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.441+0000 7f4a66a00700 1 --2- 172.21.15.112:0/4015804133 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x564d0b35eb30 0x564d0b360f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.442+0000 7f4a66a00700 1 --2- 172.21.15.112:0/4015804133 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x564d0b35eb30 0x564d0b360f40 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee455c585e8f4d4 server_cookie=511aa4ac93f93927 in_seq=0 out_seq=0 2024-05-08T06:01:08.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.442+0000 7f4a51600700 10 client.25813 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:08.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.443+0000 7f4a51600700 1 -- 172.21.15.112:0/4015804133 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d0b3120a0 con 0x564d0b35eb30 2024-05-08T06:01:08.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.443+0000 7f4a51600700 10 client.25813 handle_command_reply: tid=0 2024-05-08T06:01:08.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.443+0000 7f4a7889eb80 1 client.25813 shutdown 2024-05-08T06:01:08.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.443+0000 7f4a7889eb80 1 -- 172.21.15.112:0/4015804133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b345060 msgr2=0x564d0b350f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:08.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.443+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/4015804133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b345060 0x564d0b350f00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4a58011690 tx=0x7f4a580119a0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.443+0000 7f4a7889eb80 1 -- 172.21.15.112:0/4015804133 shutdown_connections 2024-05-08T06:01:08.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.443+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/4015804133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b3499d0 0x564d0b34e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.443+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/4015804133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b345060 0x564d0b350f00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.443+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/4015804133 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x564d0b35eb30 0x564d0b360f40 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.443+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/4015804133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345980 0x564d0b34e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.443+0000 7f4a7889eb80 1 -- 172.21.15.112:0/4015804133 >> 172.21.15.112:0/4015804133 conn(0x564d0b273b60 msgr2=0x564d0b33a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:08.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.443+0000 7f4a51600700 0 client.25813 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:08.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.444+0000 7f4a7889eb80 1 -- 172.21.15.112:0/4015804133 shutdown_connections 2024-05-08T06:01:08.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.444+0000 7f4a7889eb80 1 -- 172.21.15.112:0/4015804133 wait complete. 2024-05-08T06:01:08.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.444+0000 7f4a7889eb80 20 client.25813 trim_cache size 0 max 16384 2024-05-08T06:01:08.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.462+0000 7f4a7889eb80 1 Processor -- start 2024-05-08T06:01:08.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.462+0000 7f4a7889eb80 1 -- start start 2024-05-08T06:01:08.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.462+0000 7f4a7889eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b273b60 0x564d0b34d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.462+0000 7f4a7889eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345060 0x564d0b34be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.462+0000 7f4a7889eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345980 0x564d0b35aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.462+0000 7f4a7889eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681b6b00 con 0x564d0b345060 2024-05-08T06:01:08.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.462+0000 7f4a7889eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681b4390 con 0x564d0b345980 2024-05-08T06:01:08.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.462+0000 7f4a7889eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681b21e0 con 0x564d0b273b60 2024-05-08T06:01:08.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.463+0000 7f4a67400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b273b60 0x564d0b34d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.463+0000 7f4a67400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b273b60 0x564d0b34d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38560/0 (socket says 172.21.15.112:38560) 2024-05-08T06:01:08.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.463+0000 7f4a67400700 1 -- 172.21.15.112:0/3790716192 learned_addr learned my addr 172.21.15.112:0/3790716192 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:08.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.463+0000 7f4a66a00700 1 --2- 172.21.15.112:0/3790716192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345060 0x564d0b34be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.463+0000 7f4a67e00700 1 --2- 172.21.15.112:0/3790716192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345980 0x564d0b35aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.463+0000 7f4a67400700 1 -- 172.21.15.112:0/3790716192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345980 msgr2=0x564d0b35aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:08.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.463+0000 7f4a67400700 1 --2- 172.21.15.112:0/3790716192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345980 0x564d0b35aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.463+0000 7f4a67400700 1 -- 172.21.15.112:0/3790716192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345060 msgr2=0x564d0b34be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:08.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.463+0000 7f4a67400700 1 --2- 172.21.15.112:0/3790716192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345060 0x564d0b34be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.463+0000 7f4a67400700 1 -- 172.21.15.112:0/3790716192 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0b3615c0 con 0x564d0b273b60 2024-05-08T06:01:08.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.463+0000 7f4a66a00700 1 --2- 172.21.15.112:0/3790716192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345060 0x564d0b34be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:01:08.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.464+0000 7f4a67400700 1 --2- 172.21.15.112:0/3790716192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b273b60 0x564d0b34d200 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4a58011660 tx=0x7f4a58009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7e24cb474ae5a14 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:08.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.464+0000 7f4a53e00700 1 -- 172.21.15.112:0/3790716192 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58029040 con 0x564d0b273b60 2024-05-08T06:01:08.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.464+0000 7f4a53e00700 1 -- 172.21.15.112:0/3790716192 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a5801acd0 con 0x564d0b273b60 2024-05-08T06:01:08.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.464+0000 7f4a53e00700 1 -- 172.21.15.112:0/3790716192 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58003680 con 0x564d0b273b60 2024-05-08T06:01:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.465+0000 7f4a7889eb80 1 -- 172.21.15.112:0/3790716192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b273b60 msgr2=0x564d0b34d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.465+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/3790716192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b273b60 0x564d0b34d200 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4a58011660 tx=0x7f4a58009260 comp rx=0 tx=0).stop 2024-05-08T06:01:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.465+0000 7f4a7889eb80 1 -- 172.21.15.112:0/3790716192 shutdown_connections 2024-05-08T06:01:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.465+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/3790716192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345980 0x564d0b35aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.465+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/3790716192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b273b60 0x564d0b34d200 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.465+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/3790716192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b345060 0x564d0b34be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.465+0000 7f4a7889eb80 1 -- 172.21.15.112:0/3790716192 >> 172.21.15.112:0/3790716192 conn(0x564d0b339610 msgr2=0x564d0b339a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.465+0000 7f4a7889eb80 1 -- 172.21.15.112:0/3790716192 shutdown_connections 2024-05-08T06:01:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.465+0000 7f4a7889eb80 1 -- 172.21.15.112:0/3790716192 wait complete. 2024-05-08T06:01:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.466+0000 7f4a7889eb80 1 -- start start 2024-05-08T06:01:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.466+0000 7f4a7889eb80 1 Processor -- start 2024-05-08T06:01:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.467+0000 7f4a7889eb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b273b60 0x564d0b357640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.467+0000 7f4a7889eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345060 0x564d0b3529c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.467+0000 7f4a7889eb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b345980 0x564d0b352fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.467+0000 7f4a7889eb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681b2850 con 0x564d0b273b60 2024-05-08T06:01:08.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.467+0000 7f4a7889eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681b3cc0 con 0x564d0b345060 2024-05-08T06:01:08.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.467+0000 7f4a7889eb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681b2f20 con 0x564d0b345980 2024-05-08T06:01:08.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.467+0000 7f4a7889eb80 10 client.0 _ll_register_callbacks cb 0x564d0b23ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:01:08.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.467+0000 7f4a67e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b345980 0x564d0b352fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.467+0000 7f4a67e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b345980 0x564d0b352fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38570/0 (socket says 172.21.15.112:38570) 2024-05-08T06:01:08.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.467+0000 7f4a67e00700 1 -- 172.21.15.112:0/69173021 learned_addr learned my addr 172.21.15.112:0/69173021 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:08.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.467+0000 7f4a67400700 1 --2- 172.21.15.112:0/69173021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b273b60 0x564d0b357640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.467+0000 7f4a66a00700 1 --2- 172.21.15.112:0/69173021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345060 0x564d0b3529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.468+0000 7f4a67e00700 1 -- 172.21.15.112:0/69173021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345060 msgr2=0x564d0b3529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.469+0000 7f4a67e00700 1 --2- 172.21.15.112:0/69173021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345060 0x564d0b3529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.469+0000 7f4a67e00700 1 -- 172.21.15.112:0/69173021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b273b60 msgr2=0x564d0b357640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.469+0000 7f4a67e00700 1 --2- 172.21.15.112:0/69173021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b273b60 0x564d0b357640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.469+0000 7f4a67e00700 1 -- 172.21.15.112:0/69173021 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0b3537d0 con 0x564d0b345980 2024-05-08T06:01:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.469+0000 7f4a67e00700 1 --2- 172.21.15.112:0/69173021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b345980 0x564d0b352fd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4a60007730 tx=0x7f4a6000b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27bdc5b61907f4ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.469+0000 7f4a51600700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:01:08.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.470+0000 7f4a51600700 1 -- 172.21.15.112:0/69173021 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60033430 con 0x564d0b345980 2024-05-08T06:01:08.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.470+0000 7f4a51600700 1 -- 172.21.15.112:0/69173021 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f4a6001b430 con 0x564d0b345980 2024-05-08T06:01:08.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.470+0000 7f4a7889eb80 1 -- 172.21.15.112:0/69173021 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a5801acd0 con 0x564d0b345980 2024-05-08T06:01:08.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.470+0000 7f4a51600700 1 -- 172.21.15.112:0/69173021 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4a6001b630 con 0x564d0b345980 2024-05-08T06:01:08.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.470+0000 7f4a7889eb80 10 client.25819 fetch_fsmap learned FSMap version 32 2024-05-08T06:01:08.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.470+0000 7f4a7889eb80 1 -- 172.21.15.112:0/69173021 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d0b3c94c0 con 0x564d0b345980 2024-05-08T06:01:08.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.470+0000 7f4a51600700 1 -- 172.21.15.112:0/69173021 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a5801acd0 con 0x564d0b345980 2024-05-08T06:01:08.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.470+0000 7f4a51600700 1 -- 172.21.15.112:0/69173021 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 32) ==== 8342+0+0 (secure 0 0 0) 0x7f4a60011070 con 0x564d0b345980 2024-05-08T06:01:08.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.470+0000 7f4a7889eb80 10 client.25819 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T06:01:08.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.470+0000 7f4a7889eb80 10 client.25819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:01:08.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.470+0000 7f4a7889eb80 20 client.25819 populate_metadata read hostname 'smithi112' 2024-05-08T06:01:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.470+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/69173021 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x564d0b3499d0 0x564d0b3cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.470+0000 7f4a7889eb80 4 client.25819 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:01:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.470+0000 7f4a7889eb80 1 -- 172.21.15.112:0/69173021 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d0b2a5400 con 0x564d0b3499d0 2024-05-08T06:01:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.471+0000 7f4a67400700 1 --2- 172.21.15.112:0/69173021 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x564d0b3499d0 0x564d0b3cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.472+0000 7f4a67400700 1 --2- 172.21.15.112:0/69173021 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x564d0b3499d0 0x564d0b3cbcc0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38c51c63c6821e70 server_cookie=c5dc860d0a5ac234 in_seq=0 out_seq=0 2024-05-08T06:01:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.472+0000 7f4a51600700 10 client.25819 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.473+0000 7f4a51600700 1 -- 172.21.15.112:0/69173021 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d0b2a5400 con 0x564d0b3499d0 2024-05-08T06:01:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.473+0000 7f4a51600700 10 client.25819 handle_command_reply: tid=0 2024-05-08T06:01:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.473+0000 7f4a7889eb80 1 client.25819 shutdown 2024-05-08T06:01:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.473+0000 7f4a7889eb80 1 -- 172.21.15.112:0/69173021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b345980 msgr2=0x564d0b352fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.473+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/69173021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b345980 0x564d0b352fd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4a60007730 tx=0x7f4a6000b680 comp rx=0 tx=0).stop 2024-05-08T06:01:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.473+0000 7f4a7889eb80 1 -- 172.21.15.112:0/69173021 shutdown_connections 2024-05-08T06:01:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.473+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/69173021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d0b345980 0x564d0b352fd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.473+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/69173021 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x564d0b3499d0 0x564d0b3cbcc0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.473+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/69173021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d0b273b60 0x564d0b357640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.473+0000 7f4a7889eb80 1 --2- 172.21.15.112:0/69173021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d0b345060 0x564d0b3529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.473+0000 7f4a7889eb80 1 -- 172.21.15.112:0/69173021 >> 172.21.15.112:0/69173021 conn(0x564d0b339610 msgr2=0x564d0b34bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.473+0000 7f4a51600700 0 client.25819 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.473+0000 7f4a7889eb80 1 -- 172.21.15.112:0/69173021 shutdown_connections 2024-05-08T06:01:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.474+0000 7f4a7889eb80 1 -- 172.21.15.112:0/69173021 wait complete. 2024-05-08T06:01:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.474+0000 7f4a7889eb80 20 client.25819 trim_cache size 0 max 16384 2024-05-08T06:01:08.475 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T06:01:08.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.476+0000 7f4a6e000700 1 -- 172.21.15.112:0/890740327 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4a4c0b2900 msgr2=0x7f4a4c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:08.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.476+0000 7f4a6e000700 1 --2- 172.21.15.112:0/890740327 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4a4c0b2900 0x7f4a4c0b4dc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4a60006070 tx=0x7f4a60009920 comp rx=0 tx=0).stop 2024-05-08T06:01:08.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.476+0000 7f4a6e000700 1 -- 172.21.15.112:0/890740327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a68104100 msgr2=0x7f4a6810fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:08.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.476+0000 7f4a6e000700 1 --2- 172.21.15.112:0/890740327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a68104100 0x7f4a6810fb00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400e9c0 tx=0x7f4a5400ed80 comp rx=0 tx=0).stop 2024-05-08T06:01:08.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.476+0000 7f4a6e000700 1 -- 172.21.15.112:0/890740327 shutdown_connections 2024-05-08T06:01:08.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.476+0000 7f4a6e000700 1 --2- 172.21.15.112:0/890740327 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f4a4c0b2900 0x7f4a4c0b4dc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.476+0000 7f4a6e000700 1 --2- 172.21.15.112:0/890740327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a68108850 0x7f4a681a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.476+0000 7f4a6e000700 1 --2- 172.21.15.112:0/890740327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a68103750 0x7f4a6810f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.476+0000 7f4a6e000700 1 --2- 172.21.15.112:0/890740327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a68104100 0x7f4a6810fb00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:08.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.476+0000 7f4a6e000700 1 -- 172.21.15.112:0/890740327 >> 172.21.15.112:0/890740327 conn(0x7f4a6807a600 msgr2=0x7f4a681017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:08.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.477+0000 7f4a6e000700 1 -- 172.21.15.112:0/890740327 shutdown_connections 2024-05-08T06:01:08.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:08.477+0000 7f4a6e000700 1 -- 172.21.15.112:0/890740327 wait complete. 2024-05-08T06:01:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:09 smithi150 ceph-mon[34315]: pgmap v616: 97 pgs: 97 active+clean; 114 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 306 op/s 2024-05-08T06:01:09.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:09 smithi112 ceph-mon[30003]: pgmap v616: 97 pgs: 97 active+clean; 114 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 306 op/s 2024-05-08T06:01:09.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:09 smithi162 ceph-mon[34800]: pgmap v616: 97 pgs: 97 active+clean; 114 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 306 op/s 2024-05-08T06:01:10.517 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T06:01:10.518 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T06:01:10.998 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:01:11.416 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:11 smithi112 ceph-mon[30003]: pgmap v617: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.5 MiB/s wr, 403 op/s 2024-05-08T06:01:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:11 smithi150 ceph-mon[34315]: pgmap v617: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.5 MiB/s wr, 403 op/s 2024-05-08T06:01:11.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:11 smithi162 ceph-mon[34800]: pgmap v617: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.5 MiB/s wr, 403 op/s 2024-05-08T06:01:13.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:13 smithi112 ceph-mon[30003]: pgmap v618: 97 pgs: 97 active+clean; 114 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 930 KiB/s wr, 352 op/s 2024-05-08T06:01:13.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.471+0000 7fc625600700 1 -- 172.21.15.112:0/1417192461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc620104100 msgr2=0x7fc620108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:13.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.471+0000 7fc625600700 1 --2- 172.21.15.112:0/1417192461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc620104100 0x7fc620108120 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009990 tx=0x7fc60c009ca0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.471+0000 7fc625600700 1 -- 172.21.15.112:0/1417192461 shutdown_connections 2024-05-08T06:01:13.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.471+0000 7fc625600700 1 --2- 172.21.15.112:0/1417192461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc620108850 0x7fc6201103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.471+0000 7fc625600700 1 --2- 172.21.15.112:0/1417192461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc620103750 0x7fc620103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.471+0000 7fc625600700 1 --2- 172.21.15.112:0/1417192461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc620104100 0x7fc620108120 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.471+0000 7fc625600700 1 -- 172.21.15.112:0/1417192461 >> 172.21.15.112:0/1417192461 conn(0x7fc62007a600 msgr2=0x7fc62007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:13.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.472+0000 7fc625600700 1 -- 172.21.15.112:0/1417192461 shutdown_connections 2024-05-08T06:01:13.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.472+0000 7fc625600700 1 -- 172.21.15.112:0/1417192461 wait complete. 2024-05-08T06:01:13.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.473+0000 7fc625600700 1 Processor -- start 2024-05-08T06:01:13.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.473+0000 7fc625600700 1 -- start start 2024-05-08T06:01:13.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.474+0000 7fc625600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc620103750 0x7fc62019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.474+0000 7fc625600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc620104100 0x7fc62019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.474+0000 7fc625600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc620108850 0x7fc620199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.474+0000 7fc625600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc620113660 con 0x7fc620108850 2024-05-08T06:01:13.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.474+0000 7fc625600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201134e0 con 0x7fc620103750 2024-05-08T06:01:13.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.474+0000 7fc625600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201137e0 con 0x7fc620104100 2024-05-08T06:01:13.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.474+0000 7fc61ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc620103750 0x7fc62019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.474+0000 7fc61ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc620103750 0x7fc62019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:49430/0 (socket says 172.21.15.112:49430) 2024-05-08T06:01:13.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.474+0000 7fc61ea00700 1 -- 172.21.15.112:0/3329080350 learned_addr learned my addr 172.21.15.112:0/3329080350 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:13.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.474+0000 7fc61e000700 1 --2- 172.21.15.112:0/3329080350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc620104100 0x7fc62019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.474+0000 7fc61ea00700 1 -- 172.21.15.112:0/3329080350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc620104100 msgr2=0x7fc62019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:13.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.475+0000 7fc61ea00700 1 --2- 172.21.15.112:0/3329080350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc620104100 0x7fc62019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.475+0000 7fc61ea00700 1 -- 172.21.15.112:0/3329080350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc620108850 msgr2=0x7fc620199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:13.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.475+0000 7fc61ea00700 1 --2- 172.21.15.112:0/3329080350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc620108850 0x7fc620199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.475+0000 7fc61ea00700 1 -- 172.21.15.112:0/3329080350 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62019a080 con 0x7fc620103750 2024-05-08T06:01:13.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.475+0000 7fc61ea00700 1 --2- 172.21.15.112:0/3329080350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc620103750 0x7fc62019f760 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc61000e9a0 tx=0x7fc61000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:13.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.475+0000 7fc617400700 1 -- 172.21.15.112:0/3329080350 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61000cb50 con 0x7fc620103750 2024-05-08T06:01:13.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.475+0000 7fc617400700 1 -- 172.21.15.112:0/3329080350 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc61000ccf0 con 0x7fc620103750 2024-05-08T06:01:13.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.475+0000 7fc625600700 1 -- 172.21.15.112:0/3329080350 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62019a370 con 0x7fc620103750 2024-05-08T06:01:13.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.475+0000 7fc617400700 1 -- 172.21.15.112:0/3329080350 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610018680 con 0x7fc620103750 2024-05-08T06:01:13.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.475+0000 7fc625600700 1 -- 172.21.15.112:0/3329080350 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6201a6630 con 0x7fc620103750 2024-05-08T06:01:13.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.479+0000 7fc617400700 1 -- 172.21.15.112:0/3329080350 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc610010430 con 0x7fc620103750 2024-05-08T06:01:13.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.480+0000 7fc62fe9ab80 1 Processor -- start 2024-05-08T06:01:13.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.480+0000 7fc62fe9ab80 1 -- start start 2024-05-08T06:01:13.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.481+0000 7fc62fe9ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bcdc40 0x562991bce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.481+0000 7fc62fe9ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bce560 0x562991bc8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.481+0000 7fc62fe9ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bc8970 0x562991bc8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.481+0000 7fc62fe9ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562991b1aaa0 con 0x562991bcdc40 2024-05-08T06:01:13.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.481+0000 7fc62fe9ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562991ab87e0 con 0x562991bc8970 2024-05-08T06:01:13.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.481+0000 7fc62fe9ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562991b16a60 con 0x562991bce560 2024-05-08T06:01:13.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.481+0000 7fc61ea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bce560 0x562991bc8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.481+0000 7fc61f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bcdc40 0x562991bce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.481+0000 7fc61ea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bce560 0x562991bc8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38602/0 (socket says 172.21.15.112:38602) 2024-05-08T06:01:13.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.481+0000 7fc61ea00700 1 -- 172.21.15.112:0/1335879329 learned_addr learned my addr 172.21.15.112:0/1335879329 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:13.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.482+0000 7fc61e000700 1 --2- 172.21.15.112:0/1335879329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bc8970 0x562991bc8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.482+0000 7fc617400700 1 --2- 172.21.15.112:0/3329080350 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc6040b2880 0x7fc6040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.482+0000 7fc617400700 1 -- 172.21.15.112:0/3329080350 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fc6100e24f0 con 0x7fc620103750 2024-05-08T06:01:13.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.482+0000 7fc61ea00700 1 -- 172.21.15.112:0/1335879329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bc8970 msgr2=0x562991bc8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:13.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.482+0000 7fc61ea00700 1 --2- 172.21.15.112:0/1335879329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bc8970 0x562991bc8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.482+0000 7fc61ea00700 1 -- 172.21.15.112:0/1335879329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bcdc40 msgr2=0x562991bce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:13.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.482+0000 7fc61ea00700 1 --2- 172.21.15.112:0/1335879329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bcdc40 0x562991bce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.482+0000 7fc61ea00700 1 -- 172.21.15.112:0/1335879329 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562991a67950 con 0x562991bce560 2024-05-08T06:01:13.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.482+0000 7fc61f400700 1 --2- 172.21.15.112:0/1335879329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bcdc40 0x562991bce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:01:13.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.482+0000 7fc61f400700 1 --2- 172.21.15.112:0/3329080350 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc6040b2880 0x7fc6040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.483+0000 7fc61ea00700 1 --2- 172.21.15.112:0/1335879329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bce560 0x562991bc8350 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc6100a6cf0 tx=0x7fc610009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d66b20fbccdbe946 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:13.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.483+0000 7fc60be00700 1 -- 172.21.15.112:0/1335879329 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6100a93d0 con 0x562991bce560 2024-05-08T06:01:13.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.483+0000 7fc60be00700 1 -- 172.21.15.112:0/1335879329 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6100b2220 con 0x562991bce560 2024-05-08T06:01:13.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.483+0000 7fc60be00700 1 -- 172.21.15.112:0/1335879329 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6100a91f0 con 0x562991bce560 2024-05-08T06:01:13.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.484+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/1335879329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bce560 msgr2=0x562991bc8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:13.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.484+0000 7fc61f400700 1 --2- 172.21.15.112:0/3329080350 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc6040b2880 0x7fc6040b4d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc618009450 tx=0x7fc618009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:13.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.484+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/1335879329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bce560 0x562991bc8350 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc6100a6cf0 tx=0x7fc610009790 comp rx=0 tx=0).stop 2024-05-08T06:01:13.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.484+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/1335879329 shutdown_connections 2024-05-08T06:01:13.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.484+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/1335879329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bc8970 0x562991bc8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.484+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/1335879329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bcdc40 0x562991bce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.484+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/1335879329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bce560 0x562991bc8350 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.484+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/1335879329 >> 172.21.15.112:0/1335879329 conn(0x562991b40750 msgr2=0x562991bc2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:13.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.484+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/1335879329 shutdown_connections 2024-05-08T06:01:13.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.484+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/1335879329 wait complete. 2024-05-08T06:01:13.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.485+0000 7fc62fe9ab80 1 -- start start 2024-05-08T06:01:13.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.485+0000 7fc62fe9ab80 1 Processor -- start 2024-05-08T06:01:13.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.485+0000 7fc62fe9ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bc8970 0x562991bd41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.485+0000 7fc62fe9ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bcdc40 0x562991bd1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.485+0000 7fc62fe9ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bce560 0x562991bd1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.485+0000 7fc62fe9ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562991bd4f40 con 0x562991bc8970 2024-05-08T06:01:13.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.485+0000 7fc62fe9ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562991a80ba0 con 0x562991bce560 2024-05-08T06:01:13.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.485+0000 7fc62fe9ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562991bd50c0 con 0x562991bcdc40 2024-05-08T06:01:13.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.486+0000 7fc62fe9ab80 10 client.0 _ll_register_callbacks cb 0x562991ac1cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:01:13.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.486+0000 7fc61ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bc8970 0x562991bd41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.486+0000 7fc61ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bc8970 0x562991bd41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36790/0 (socket says 172.21.15.112:36790) 2024-05-08T06:01:13.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.486+0000 7fc61ea00700 1 -- 172.21.15.112:0/4251718286 learned_addr learned my addr 172.21.15.112:0/4251718286 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:13.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.486+0000 7fc61f400700 1 --2- 172.21.15.112:0/4251718286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bce560 0x562991bd1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.486+0000 7fc61e000700 1 --2- 172.21.15.112:0/4251718286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bcdc40 0x562991bd1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.486+0000 7fc61f400700 1 -- 172.21.15.112:0/4251718286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bcdc40 msgr2=0x562991bd1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:13.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.486+0000 7fc61f400700 1 --2- 172.21.15.112:0/4251718286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bcdc40 0x562991bd1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.486+0000 7fc61f400700 1 -- 172.21.15.112:0/4251718286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bc8970 msgr2=0x562991bd41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:13.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.486+0000 7fc61f400700 1 --2- 172.21.15.112:0/4251718286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bc8970 0x562991bd41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.486+0000 7fc61f400700 1 -- 172.21.15.112:0/4251718286 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562991ae04a0 con 0x562991bce560 2024-05-08T06:01:13.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.487+0000 7fc61f400700 1 --2- 172.21.15.112:0/4251718286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bce560 0x562991bd1ac0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc618007c20 tx=0x7fc61800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81c1614a76b14e07 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:13.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.487+0000 7fc608c00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:01:13.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.487+0000 7fc608c00700 1 -- 172.21.15.112:0/4251718286 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6180091f0 con 0x562991bce560 2024-05-08T06:01:13.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.487+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/4251718286 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562991b951c0 con 0x562991bce560 2024-05-08T06:01:13.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.487+0000 7fc608c00700 1 -- 172.21.15.112:0/4251718286 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fc61801eb60 con 0x562991bce560 2024-05-08T06:01:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.487+0000 7fc608c00700 1 -- 172.21.15.112:0/4251718286 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc61801ed60 con 0x562991bce560 2024-05-08T06:01:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.487+0000 7fc62fe9ab80 10 client.25787 fetch_fsmap learned FSMap version 32 2024-05-08T06:01:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.487+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/4251718286 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562991bdf820 con 0x562991bce560 2024-05-08T06:01:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.488+0000 7fc608c00700 1 -- 172.21.15.112:0/4251718286 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562991b951c0 con 0x562991bce560 2024-05-08T06:01:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.488+0000 7fc608c00700 1 -- 172.21.15.112:0/4251718286 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 32) ==== 8342+0+0 (secure 0 0 0) 0x7fc61801b450 con 0x562991bce560 2024-05-08T06:01:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.488+0000 7fc62fe9ab80 10 client.25787 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T06:01:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.488+0000 7fc62fe9ab80 10 client.25787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:01:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.488+0000 7fc62fe9ab80 20 client.25787 populate_metadata read hostname 'smithi112' 2024-05-08T06:01:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.488+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/4251718286 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562991be1db0 0x562991be41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.488+0000 7fc62fe9ab80 4 client.25787 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:01:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.488+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/4251718286 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562991b28480 con 0x562991be1db0 2024-05-08T06:01:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.488+0000 7fc61ea00700 1 --2- 172.21.15.112:0/4251718286 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562991be1db0 0x562991be41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.489+0000 7fc61ea00700 1 --2- 172.21.15.112:0/4251718286 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562991be1db0 0x562991be41a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe86e9049c49bb6 server_cookie=e610e47525d0792 in_seq=0 out_seq=0 2024-05-08T06:01:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.489+0000 7fc608c00700 10 client.25787 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.490+0000 7fc608c00700 1 -- 172.21.15.112:0/4251718286 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562991b28480 con 0x562991be1db0 2024-05-08T06:01:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.490+0000 7fc608c00700 10 client.25787 handle_command_reply: tid=0 2024-05-08T06:01:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.490+0000 7fc62fe9ab80 1 client.25787 shutdown 2024-05-08T06:01:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.490+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/4251718286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bce560 msgr2=0x562991bd1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.490+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/4251718286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bce560 0x562991bd1ac0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc618007c20 tx=0x7fc61800b040 comp rx=0 tx=0).stop 2024-05-08T06:01:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.490+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/4251718286 shutdown_connections 2024-05-08T06:01:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.490+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/4251718286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bce560 0x562991bd1ac0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.490+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/4251718286 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562991be1db0 0x562991be41a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.490+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/4251718286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bc8970 0x562991bd41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.490+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/4251718286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bcdc40 0x562991bd1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.490+0000 7fc608c00700 0 client.25787 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:13.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.490+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/4251718286 >> 172.21.15.112:0/4251718286 conn(0x562991b40750 msgr2=0x562991bc2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:13.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.490+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/4251718286 shutdown_connections 2024-05-08T06:01:13.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.491+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/4251718286 wait complete. 2024-05-08T06:01:13.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.491+0000 7fc62fe9ab80 20 client.25787 trim_cache size 0 max 16384 2024-05-08T06:01:13.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.516+0000 7fc62fe9ab80 1 Processor -- start 2024-05-08T06:01:13.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.516+0000 7fc62fe9ab80 1 -- start start 2024-05-08T06:01:13.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.516+0000 7fc62fe9ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bc8970 0x562991bcf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.516+0000 7fc62fe9ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bcdc40 0x562991bde000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.516+0000 7fc62fe9ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bce560 0x562991bd0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.516+0000 7fc62fe9ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201ae2e0 con 0x562991bc8970 2024-05-08T06:01:13.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.516+0000 7fc62fe9ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201abb70 con 0x562991bce560 2024-05-08T06:01:13.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.516+0000 7fc62fe9ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201a99c0 con 0x562991bcdc40 2024-05-08T06:01:13.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.517+0000 7fc61e000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bcdc40 0x562991bde000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.517+0000 7fc61e000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bcdc40 0x562991bde000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38620/0 (socket says 172.21.15.112:38620) 2024-05-08T06:01:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.517+0000 7fc61e000700 1 -- 172.21.15.112:0/1345578000 learned_addr learned my addr 172.21.15.112:0/1345578000 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.518+0000 7fc61ea00700 1 --2- 172.21.15.112:0/1345578000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bc8970 0x562991bcf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.518+0000 7fc61e000700 1 -- 172.21.15.112:0/1345578000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bce560 msgr2=0x562991bd0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.518+0000 7fc61f400700 1 --2- 172.21.15.112:0/1345578000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bce560 0x562991bd0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.518+0000 7fc61e000700 1 --2- 172.21.15.112:0/1345578000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bce560 0x562991bd0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.518+0000 7fc61e000700 1 -- 172.21.15.112:0/1345578000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bc8970 msgr2=0x562991bcf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.518+0000 7fc61e000700 1 --2- 172.21.15.112:0/1345578000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bc8970 0x562991bcf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.518+0000 7fc61e000700 1 -- 172.21.15.112:0/1345578000 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562991be4800 con 0x562991bcdc40 2024-05-08T06:01:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.518+0000 7fc61ea00700 1 --2- 172.21.15.112:0/1345578000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bc8970 0x562991bcf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:01:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.519+0000 7fc61e000700 1 --2- 172.21.15.112:0/1345578000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bcdc40 0x562991bde000 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc60c000c00 tx=0x7fc60c00b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28e56fb13a7a8969 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:13.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.519+0000 7fc60b400700 1 -- 172.21.15.112:0/1345578000 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c005690 con 0x562991bcdc40 2024-05-08T06:01:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.519+0000 7fc60b400700 1 -- 172.21.15.112:0/1345578000 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc60c00bca0 con 0x562991bcdc40 2024-05-08T06:01:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.519+0000 7fc60b400700 1 -- 172.21.15.112:0/1345578000 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c00f460 con 0x562991bcdc40 2024-05-08T06:01:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.520+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/1345578000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bcdc40 msgr2=0x562991bde000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.520+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/1345578000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bcdc40 0x562991bde000 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc60c000c00 tx=0x7fc60c00b260 comp rx=0 tx=0).stop 2024-05-08T06:01:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.520+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/1345578000 shutdown_connections 2024-05-08T06:01:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.520+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/1345578000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bce560 0x562991bd0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.520+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/1345578000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bc8970 0x562991bcf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.520+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/1345578000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bcdc40 0x562991bde000 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.520+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/1345578000 >> 172.21.15.112:0/1345578000 conn(0x562991b40750 msgr2=0x562991bcc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.520+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/1345578000 shutdown_connections 2024-05-08T06:01:13.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.520+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/1345578000 wait complete. 2024-05-08T06:01:13.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.521+0000 7fc62fe9ab80 1 -- start start 2024-05-08T06:01:13.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.521+0000 7fc62fe9ab80 1 Processor -- start 2024-05-08T06:01:13.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.521+0000 7fc62fe9ab80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bc8970 0x562991bda380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.521+0000 7fc62fe9ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bcdc40 0x562991bda8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.521+0000 7fc62fe9ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bce560 0x562991bd7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.521+0000 7fc62fe9ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201aa030 con 0x562991bce560 2024-05-08T06:01:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.521+0000 7fc62fe9ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201ab4a0 con 0x562991bcdc40 2024-05-08T06:01:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.521+0000 7fc62fe9ab80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201aa700 con 0x562991bc8970 2024-05-08T06:01:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.522+0000 7fc62fe9ab80 10 client.0 _ll_register_callbacks cb 0x562991ac1cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:01:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.522+0000 7fc61e000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bcdc40 0x562991bda8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.522+0000 7fc61ea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bc8970 0x562991bda380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.522+0000 7fc61e000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bcdc40 0x562991bda8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:49446/0 (socket says 172.21.15.112:49446) 2024-05-08T06:01:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.522+0000 7fc61e000700 1 -- 172.21.15.112:0/395760036 learned_addr learned my addr 172.21.15.112:0/395760036 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.522+0000 7fc61ea00700 1 --2- 172.21.15.112:0/395760036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bc8970 0x562991bda380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38630/0 (socket says 172.21.15.112:38630) 2024-05-08T06:01:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.522+0000 7fc61f400700 1 --2- 172.21.15.112:0/395760036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bce560 0x562991bd7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.522+0000 7fc61ea00700 1 -- 172.21.15.112:0/395760036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bcdc40 msgr2=0x562991bda8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.522+0000 7fc61ea00700 1 --2- 172.21.15.112:0/395760036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bcdc40 0x562991bda8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.522+0000 7fc61ea00700 1 -- 172.21.15.112:0/395760036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bce560 msgr2=0x562991bd7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.522+0000 7fc61ea00700 1 --2- 172.21.15.112:0/395760036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bce560 0x562991bd7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.522+0000 7fc61ea00700 1 -- 172.21.15.112:0/395760036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562991bd7a70 con 0x562991bc8970 2024-05-08T06:01:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.522+0000 7fc61e000700 1 --2- 172.21.15.112:0/395760036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bcdc40 0x562991bda8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:01:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.523+0000 7fc61ea00700 1 --2- 172.21.15.112:0/395760036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bc8970 0x562991bda380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc6100e27e0 tx=0x7fc6100a4070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7a4b19ebc822c77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.523+0000 7fc608c00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:01:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.523+0000 7fc608c00700 1 -- 172.21.15.112:0/395760036 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6100b2650 con 0x562991bc8970 2024-05-08T06:01:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.523+0000 7fc608c00700 1 -- 172.21.15.112:0/395760036 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fc6100cc0e0 con 0x562991bc8970 2024-05-08T06:01:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.523+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/395760036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc60c00bca0 con 0x562991bc8970 2024-05-08T06:01:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.523+0000 7fc608c00700 1 -- 172.21.15.112:0/395760036 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc6100e22e0 con 0x562991bc8970 2024-05-08T06:01:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.523+0000 7fc62fe9ab80 10 client.25837 fetch_fsmap learned FSMap version 32 2024-05-08T06:01:13.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.523+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/395760036 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562991c557f0 con 0x562991bc8970 2024-05-08T06:01:13.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.523+0000 7fc608c00700 1 -- 172.21.15.112:0/395760036 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc60c00bca0 con 0x562991bc8970 2024-05-08T06:01:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.523+0000 7fc608c00700 1 -- 172.21.15.112:0/395760036 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 32) ==== 8342+0+0 (secure 0 0 0) 0x7fc6100a8ce0 con 0x562991bc8970 2024-05-08T06:01:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.524+0000 7fc62fe9ab80 10 client.25837 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T06:01:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.524+0000 7fc62fe9ab80 10 client.25837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:01:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.524+0000 7fc62fe9ab80 20 client.25837 populate_metadata read hostname 'smithi112' 2024-05-08T06:01:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.524+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/395760036 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562991c55a80 0x562991c57e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.524+0000 7fc62fe9ab80 4 client.25837 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T06:01:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.524+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/395760036 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562991b22520 con 0x562991c55a80 2024-05-08T06:01:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.524+0000 7fc61e000700 1 --2- 172.21.15.112:0/395760036 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562991c55a80 0x562991c57e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.525+0000 7fc61e000700 1 --2- 172.21.15.112:0/395760036 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562991c55a80 0x562991c57e70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b68f694d2859e15b server_cookie=51efdde86de5795a in_seq=0 out_seq=0 2024-05-08T06:01:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.525+0000 7fc608c00700 10 client.25837 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.526+0000 7fc608c00700 1 -- 172.21.15.112:0/395760036 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562991b22520 con 0x562991c55a80 2024-05-08T06:01:13.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.526+0000 7fc608c00700 10 client.25837 handle_command_reply: tid=0 2024-05-08T06:01:13.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.526+0000 7fc62fe9ab80 1 client.25837 shutdown 2024-05-08T06:01:13.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.526+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/395760036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bc8970 msgr2=0x562991bda380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:13.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.526+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/395760036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bc8970 0x562991bda380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc6100e27e0 tx=0x7fc6100a4070 comp rx=0 tx=0).stop 2024-05-08T06:01:13.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.526+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/395760036 shutdown_connections 2024-05-08T06:01:13.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.526+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/395760036 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x562991c55a80 0x562991c57e70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.526+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/395760036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562991bce560 0x562991bd7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.526+0000 7fc608c00700 0 client.25837 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:13.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.526+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/395760036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562991bc8970 0x562991bda380 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.526+0000 7fc62fe9ab80 1 --2- 172.21.15.112:0/395760036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562991bcdc40 0x562991bda8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.526+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/395760036 >> 172.21.15.112:0/395760036 conn(0x562991b40750 msgr2=0x562991bd05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:13.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.527+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/395760036 shutdown_connections 2024-05-08T06:01:13.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.527+0000 7fc62fe9ab80 1 -- 172.21.15.112:0/395760036 wait complete. 2024-05-08T06:01:13.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.527+0000 7fc62fe9ab80 20 client.25837 trim_cache size 0 max 16384 2024-05-08T06:01:13.530 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:01:13.530 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-08T06:01:13.530 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "5e0780d5-c341-4a03-b461-d5f801c396c5", 2024-05-08T06:01:13.530 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-08T06:01:13.530 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:01:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.528+0000 7fc625600700 1 -- 172.21.15.112:0/3329080350 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc6040b2880 msgr2=0x7fc6040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.528+0000 7fc625600700 1 --2- 172.21.15.112:0/3329080350 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc6040b2880 0x7fc6040b4d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc618009450 tx=0x7fc618009040 comp rx=0 tx=0).stop 2024-05-08T06:01:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.528+0000 7fc625600700 1 -- 172.21.15.112:0/3329080350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc620103750 msgr2=0x7fc62019f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.528+0000 7fc625600700 1 --2- 172.21.15.112:0/3329080350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc620103750 0x7fc62019f760 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc61000e9a0 tx=0x7fc61000ecb0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.528+0000 7fc625600700 1 -- 172.21.15.112:0/3329080350 shutdown_connections 2024-05-08T06:01:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.528+0000 7fc625600700 1 --2- 172.21.15.112:0/3329080350 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fc6040b2880 0x7fc6040b4d40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.528+0000 7fc625600700 1 --2- 172.21.15.112:0/3329080350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc620108850 0x7fc620199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.528+0000 7fc625600700 1 --2- 172.21.15.112:0/3329080350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc620103750 0x7fc62019f760 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.528+0000 7fc625600700 1 --2- 172.21.15.112:0/3329080350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc620104100 0x7fc62019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.528+0000 7fc625600700 1 -- 172.21.15.112:0/3329080350 >> 172.21.15.112:0/3329080350 conn(0x7fc62007a600 msgr2=0x7fc620101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.529+0000 7fc625600700 1 -- 172.21.15.112:0/3329080350 shutdown_connections 2024-05-08T06:01:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:13.529+0000 7fc625600700 1 -- 172.21.15.112:0/3329080350 wait complete. 2024-05-08T06:01:13.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:13 smithi150 ceph-mon[34315]: pgmap v618: 97 pgs: 97 active+clean; 114 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 930 KiB/s wr, 352 op/s 2024-05-08T06:01:13.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:13 smithi162 ceph-mon[34800]: pgmap v618: 97 pgs: 97 active+clean; 114 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 930 KiB/s wr, 352 op/s 2024-05-08T06:01:14.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:14 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T06:01:14.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:14 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:01:14.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:14 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T06:01:14.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:14 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:01:14.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:14 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T06:01:14.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:14 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:01:14.882 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T06:01:15.343 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:01:15.634 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:15 smithi112 ceph-mon[30003]: scrub queued for path: / 2024-05-08T06:01:15.634 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:15 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:01:15.635 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:15 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T06:01:15.635 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:15 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:01:15.635 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:15 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T06:01:15.635 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:15 smithi112 ceph-mon[30003]: pgmap v619: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 918 KiB/s wr, 360 op/s 2024-05-08T06:01:15.635 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:15 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:01:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:15 smithi150 ceph-mon[34315]: scrub queued for path: / 2024-05-08T06:01:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:15 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:01:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:15 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T06:01:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:15 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:01:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:15 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T06:01:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:15 smithi150 ceph-mon[34315]: pgmap v619: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 918 KiB/s wr, 360 op/s 2024-05-08T06:01:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:15 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:01:15.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:15 smithi162 ceph-mon[34800]: scrub queued for path: / 2024-05-08T06:01:15.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:15 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:01:15.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:15 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T06:01:15.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:15 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:01:15.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:15 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T06:01:15.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:15 smithi162 ceph-mon[34800]: pgmap v619: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 918 KiB/s wr, 360 op/s 2024-05-08T06:01:15.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:15 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:01:17.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.656+0000 7fa491400700 1 -- 172.21.15.112:0/3316013783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa48c10b6e0 msgr2=0x7fa48c10bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:17.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.656+0000 7fa491400700 1 --2- 172.21.15.112:0/3316013783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa48c10b6e0 0x7fa48c10bac0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0098f0 tx=0x7fa47c009c00 comp rx=0 tx=0).stop 2024-05-08T06:01:17.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.656+0000 7fa491400700 1 -- 172.21.15.112:0/3316013783 shutdown_connections 2024-05-08T06:01:17.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.656+0000 7fa491400700 1 --2- 172.21.15.112:0/3316013783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c1060a0 0x7fa48c1125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.656+0000 7fa491400700 1 --2- 172.21.15.112:0/3316013783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa48c10b6e0 0x7fa48c10bac0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.656+0000 7fa491400700 1 --2- 172.21.15.112:0/3316013783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa48c1056e0 0x7fa48c105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.656+0000 7fa491400700 1 -- 172.21.15.112:0/3316013783 >> 172.21.15.112:0/3316013783 conn(0x7fa48c1011d0 msgr2=0x7fa48c1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:17.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.656+0000 7fa491400700 1 -- 172.21.15.112:0/3316013783 shutdown_connections 2024-05-08T06:01:17.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.656+0000 7fa491400700 1 -- 172.21.15.112:0/3316013783 wait complete. 2024-05-08T06:01:17.658 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:17 smithi112 ceph-mon[30003]: pgmap v620: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.1 MiB/s wr, 481 op/s 2024-05-08T06:01:17.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.658+0000 7fa491400700 1 Processor -- start 2024-05-08T06:01:17.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.658+0000 7fa491400700 1 -- start start 2024-05-08T06:01:17.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa491400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c1056e0 0x7fa48c19bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa491400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa48c1060a0 0x7fa48c19c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa491400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa48c10b6e0 0x7fa48c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa491400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c115920 con 0x7fa48c1056e0 2024-05-08T06:01:17.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa491400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1157a0 con 0x7fa48c10b6e0 2024-05-08T06:01:17.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa491400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c115aa0 con 0x7fa48c1060a0 2024-05-08T06:01:17.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa48a800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c1056e0 0x7fa48c19bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa48a800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c1056e0 0x7fa48c19bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36220/0 (socket says 172.21.15.112:36220) 2024-05-08T06:01:17.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa48a800700 1 -- 172.21.15.112:0/1278241219 learned_addr learned my addr 172.21.15.112:0/1278241219 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:17.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa489e00700 1 --2- 172.21.15.112:0/1278241219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa48c1060a0 0x7fa48c19c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa48b200700 1 --2- 172.21.15.112:0/1278241219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa48c10b6e0 0x7fa48c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa48a800700 1 -- 172.21.15.112:0/1278241219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa48c1060a0 msgr2=0x7fa48c19c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:17.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa48a800700 1 --2- 172.21.15.112:0/1278241219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa48c1060a0 0x7fa48c19c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa48a800700 1 -- 172.21.15.112:0/1278241219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa48c10b6e0 msgr2=0x7fa48c1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:17.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa48a800700 1 --2- 172.21.15.112:0/1278241219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa48c10b6e0 0x7fa48c1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.659+0000 7fa48a800700 1 -- 172.21.15.112:0/1278241219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c1a0e00 con 0x7fa48c1056e0 2024-05-08T06:01:17.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.660+0000 7fa48a800700 1 --2- 172.21.15.112:0/1278241219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c1056e0 0x7fa48c19bdf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009230 tx=0x7fa47c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:17.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.661+0000 7fa483400700 1 -- 172.21.15.112:0/1278241219 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c005140 con 0x7fa48c1056e0 2024-05-08T06:01:17.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.661+0000 7fa483400700 1 -- 172.21.15.112:0/1278241219 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa47c00b560 con 0x7fa48c1056e0 2024-05-08T06:01:17.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.661+0000 7fa491400700 1 -- 172.21.15.112:0/1278241219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48c1a1090 con 0x7fa48c1056e0 2024-05-08T06:01:17.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.661+0000 7fa483400700 1 -- 172.21.15.112:0/1278241219 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c003970 con 0x7fa48c1056e0 2024-05-08T06:01:17.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.661+0000 7fa491400700 1 -- 172.21.15.112:0/1278241219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48c1a8970 con 0x7fa48c1056e0 2024-05-08T06:01:17.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.665+0000 7fa483400700 1 -- 172.21.15.112:0/1278241219 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa47c00ba50 con 0x7fa48c1056e0 2024-05-08T06:01:17.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.666+0000 7fa49bcc0b80 1 Processor -- start 2024-05-08T06:01:17.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.667+0000 7fa49bcc0b80 1 -- start start 2024-05-08T06:01:17.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.667+0000 7fa49bcc0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1e4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.667+0000 7fa49bcc0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 0x55f2ce1e8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.667+0000 7fa49bcc0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e89d0 0x55f2ce1e8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.667+0000 7fa49bcc0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f2ce136a20 con 0x55f2ce1e4980 2024-05-08T06:01:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.667+0000 7fa49bcc0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f2ce0d4760 con 0x55f2ce1e89d0 2024-05-08T06:01:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.667+0000 7fa49bcc0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55f2ce1329e0 con 0x55f2ce1e4060 2024-05-08T06:01:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.667+0000 7fa48b200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.667+0000 7fa48a800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 0x55f2ce1e8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.667+0000 7fa489e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e89d0 0x55f2ce1e8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.667+0000 7fa48b200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1e4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38508/0 (socket says 172.21.15.112:38508) 2024-05-08T06:01:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.667+0000 7fa48b200700 1 -- 172.21.15.112:0/2205441907 learned_addr learned my addr 172.21.15.112:0/2205441907 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.667+0000 7fa483400700 1 --2- 172.21.15.112:0/1278241219 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa4700b2900 0x7fa4700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.668+0000 7fa489e00700 1 -- 172.21.15.112:0/2205441907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 msgr2=0x55f2ce1e4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.668+0000 7fa489e00700 1 --2- 172.21.15.112:0/2205441907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1e4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.668+0000 7fa489e00700 1 -- 172.21.15.112:0/2205441907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 msgr2=0x55f2ce1e8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.668+0000 7fa489e00700 1 --2- 172.21.15.112:0/2205441907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 0x55f2ce1e8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.668+0000 7fa489e00700 1 -- 172.21.15.112:0/2205441907 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2ce0838d0 con 0x55f2ce1e89d0 2024-05-08T06:01:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.668+0000 7fa483400700 1 -- 172.21.15.112:0/1278241219 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fa47c0e31d0 con 0x7fa48c1056e0 2024-05-08T06:01:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.668+0000 7fa48b200700 1 --2- 172.21.15.112:0/1278241219 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa4700b2900 0x7fa4700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.668+0000 7fa489e00700 1 --2- 172.21.15.112:0/2205441907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e89d0 0x55f2ce1e8e30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa478005fd0 tx=0x7fa478011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2268ef45a4c7263 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:17.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.669+0000 7fa48b200700 1 --2- 172.21.15.112:0/1278241219 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa4700b2900 0x7fa4700b4dc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa4840076c0 tx=0x7fa484009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:17.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.669+0000 7fa477e00700 1 -- 172.21.15.112:0/2205441907 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478011d70 con 0x55f2ce1e89d0 2024-05-08T06:01:17.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.669+0000 7fa477e00700 1 -- 172.21.15.112:0/2205441907 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa478004500 con 0x55f2ce1e89d0 2024-05-08T06:01:17.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.669+0000 7fa477e00700 1 -- 172.21.15.112:0/2205441907 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478005310 con 0x55f2ce1e89d0 2024-05-08T06:01:17.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.670+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/2205441907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e89d0 msgr2=0x55f2ce1e8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:17.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.670+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/2205441907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e89d0 0x55f2ce1e8e30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa478005fd0 tx=0x7fa478011440 comp rx=0 tx=0).stop 2024-05-08T06:01:17.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.670+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/2205441907 shutdown_connections 2024-05-08T06:01:17.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.670+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/2205441907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e89d0 0x55f2ce1e8e30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.670+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/2205441907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1e4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.670+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/2205441907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 0x55f2ce1e8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.670+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/2205441907 >> 172.21.15.112:0/2205441907 conn(0x55f2ce112b60 msgr2=0x55f2ce1de7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:17.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.670+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/2205441907 shutdown_connections 2024-05-08T06:01:17.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.670+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/2205441907 wait complete. 2024-05-08T06:01:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.671+0000 7fa49bcc0b80 1 -- start start 2024-05-08T06:01:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.671+0000 7fa49bcc0b80 1 Processor -- start 2024-05-08T06:01:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.671+0000 7fa49bcc0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1efe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.671+0000 7fa49bcc0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 0x55f2ce1ed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.671+0000 7fa49bcc0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e89d0 0x55f2ce1ed6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.671+0000 7fa49bcc0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f2ce1f0be0 con 0x55f2ce1e4980 2024-05-08T06:01:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.671+0000 7fa49bcc0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f2ce09cb20 con 0x55f2ce1e89d0 2024-05-08T06:01:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.671+0000 7fa49bcc0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55f2ce1f0d60 con 0x55f2ce1e4060 2024-05-08T06:01:17.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.672+0000 7fa49bcc0b80 10 client.0 _ll_register_callbacks cb 0x55f2ce0ddc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:01:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.672+0000 7fa48a800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.672+0000 7fa48a800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1efe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38518/0 (socket says 172.21.15.112:38518) 2024-05-08T06:01:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.672+0000 7fa48a800700 1 -- 172.21.15.112:0/2598353992 learned_addr learned my addr 172.21.15.112:0/2598353992 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.672+0000 7fa489e00700 1 --2- 172.21.15.112:0/2598353992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 0x55f2ce1ed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.672+0000 7fa48b200700 1 --2- 172.21.15.112:0/2598353992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e89d0 0x55f2ce1ed6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.672+0000 7fa48a800700 1 -- 172.21.15.112:0/2598353992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e89d0 msgr2=0x55f2ce1ed6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.672+0000 7fa48a800700 1 --2- 172.21.15.112:0/2598353992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e89d0 0x55f2ce1ed6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.672+0000 7fa48a800700 1 -- 172.21.15.112:0/2598353992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 msgr2=0x55f2ce1ed1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.672+0000 7fa48a800700 1 --2- 172.21.15.112:0/2598353992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 0x55f2ce1ed1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.672+0000 7fa48a800700 1 -- 172.21.15.112:0/2598353992 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2ce0fc420 con 0x55f2ce1e4060 2024-05-08T06:01:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.672+0000 7fa48b200700 1 --2- 172.21.15.112:0/2598353992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e89d0 0x55f2ce1ed6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:01:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.672+0000 7fa489e00700 1 --2- 172.21.15.112:0/2598353992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 0x55f2ce1ed1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:01:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.673+0000 7fa48a800700 1 --2- 172.21.15.112:0/2598353992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1efe00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009fd0 tx=0x7fa47c0aa140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8abc2a95e4316b36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:17.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.673+0000 7fa474c00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:01:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.673+0000 7fa474c00700 1 -- 172.21.15.112:0/2598353992 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0aa300 con 0x55f2ce1e4060 2024-05-08T06:01:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.673+0000 7fa474c00700 1 -- 172.21.15.112:0/2598353992 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fa47c0a6a80 con 0x55f2ce1e4060 2024-05-08T06:01:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.673+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/2598353992 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa478004500 con 0x55f2ce1e4060 2024-05-08T06:01:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.673+0000 7fa474c00700 1 -- 172.21.15.112:0/2598353992 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa47c0aaef0 con 0x55f2ce1e4060 2024-05-08T06:01:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.673+0000 7fa49bcc0b80 10 client.25849 fetch_fsmap learned FSMap version 32 2024-05-08T06:01:17.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.673+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/2598353992 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f2ce1f0c30 con 0x55f2ce1e4060 2024-05-08T06:01:17.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.673+0000 7fa474c00700 1 -- 172.21.15.112:0/2598353992 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa478004500 con 0x55f2ce1e4060 2024-05-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.673+0000 7fa474c00700 1 -- 172.21.15.112:0/2598353992 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 32) ==== 8342+0+0 (secure 0 0 0) 0x7fa47c003c10 con 0x55f2ce1e4060 2024-05-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.674+0000 7fa49bcc0b80 10 client.25849 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.674+0000 7fa49bcc0b80 10 client.25849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.674+0000 7fa49bcc0b80 20 client.25849 populate_metadata read hostname 'smithi112' 2024-05-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.674+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/2598353992 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f2ce1fd930 0x55f2ce1ffd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.674+0000 7fa49bcc0b80 4 client.25849 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.674+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/2598353992 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2ce1b10a0 con 0x55f2ce1fd930 2024-05-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.674+0000 7fa489e00700 1 --2- 172.21.15.112:0/2598353992 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f2ce1fd930 0x55f2ce1ffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.675+0000 7fa489e00700 1 --2- 172.21.15.112:0/2598353992 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f2ce1fd930 0x55f2ce1ffd90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d22f47ecf380940 server_cookie=edf7a3c4026668e3 in_seq=0 out_seq=0 2024-05-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.675+0000 7fa474c00700 10 client.25849 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.675+0000 7fa474c00700 1 -- 172.21.15.112:0/2598353992 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f2ce1b10a0 con 0x55f2ce1fd930 2024-05-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.675+0000 7fa474c00700 10 client.25849 handle_command_reply: tid=0 2024-05-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.675+0000 7fa49bcc0b80 1 client.25849 shutdown 2024-05-08T06:01:17.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.676+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/2598353992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 msgr2=0x55f2ce1efe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.676+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/2598353992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1efe00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009fd0 tx=0x7fa47c0aa140 comp rx=0 tx=0).stop 2024-05-08T06:01:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.676+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/2598353992 shutdown_connections 2024-05-08T06:01:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.676+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/2598353992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e89d0 0x55f2ce1ed6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.676+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/2598353992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1efe00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.676+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/2598353992 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f2ce1fd930 0x55f2ce1ffd90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.676+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/2598353992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 0x55f2ce1ed1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.676+0000 7fa474c00700 0 client.25849 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.676+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/2598353992 >> 172.21.15.112:0/2598353992 conn(0x55f2ce112b60 msgr2=0x55f2ce1d92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.676+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/2598353992 shutdown_connections 2024-05-08T06:01:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.676+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/2598353992 wait complete. 2024-05-08T06:01:17.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.676+0000 7fa49bcc0b80 20 client.25849 trim_cache size 0 max 16384 2024-05-08T06:01:17.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:17 smithi150 ceph-mon[34315]: pgmap v620: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.1 MiB/s wr, 481 op/s 2024-05-08T06:01:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.700+0000 7fa49bcc0b80 1 Processor -- start 2024-05-08T06:01:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.700+0000 7fa49bcc0b80 1 -- start start 2024-05-08T06:01:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.700+0000 7fa49bcc0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1f9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.700+0000 7fa49bcc0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e4980 0x55f2ce1eef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.701+0000 7fa49bcc0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e89d0 0x55f2ce2056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.701+0000 7fa49bcc0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1b25c0 con 0x55f2ce1e89d0 2024-05-08T06:01:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.701+0000 7fa49bcc0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1afe50 con 0x55f2ce1e4980 2024-05-08T06:01:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.701+0000 7fa49bcc0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1adca0 con 0x55f2ce1e4060 2024-05-08T06:01:17.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.701+0000 7fa48a800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1f9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.701+0000 7fa48b200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e89d0 0x55f2ce2056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.701+0000 7fa48a800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1f9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:38526/0 (socket says 172.21.15.112:38526) 2024-05-08T06:01:17.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.701+0000 7fa48a800700 1 -- 172.21.15.112:0/27216206 learned_addr learned my addr 172.21.15.112:0/27216206 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.701+0000 7fa489e00700 1 --2- 172.21.15.112:0/27216206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e4980 0x55f2ce1eef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.701+0000 7fa48a800700 1 -- 172.21.15.112:0/27216206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e4980 msgr2=0x55f2ce1eef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.701+0000 7fa48a800700 1 --2- 172.21.15.112:0/27216206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e4980 0x55f2ce1eef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.701+0000 7fa48a800700 1 -- 172.21.15.112:0/27216206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e89d0 msgr2=0x55f2ce2056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.701+0000 7fa48a800700 1 --2- 172.21.15.112:0/27216206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e89d0 0x55f2ce2056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.701+0000 7fa48a800700 1 -- 172.21.15.112:0/27216206 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2ce1db3f0 con 0x55f2ce1e4060 2024-05-08T06:01:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.702+0000 7fa48a800700 1 --2- 172.21.15.112:0/27216206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1f9c70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009680 tx=0x7fa47c00ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c741962aa8d084a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.702+0000 7fa477400700 1 -- 172.21.15.112:0/27216206 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0a9f60 con 0x55f2ce1e4060 2024-05-08T06:01:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.702+0000 7fa477400700 1 -- 172.21.15.112:0/27216206 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa47c0aa300 con 0x55f2ce1e4060 2024-05-08T06:01:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.702+0000 7fa477400700 1 -- 172.21.15.112:0/27216206 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0aeba0 con 0x55f2ce1e4060 2024-05-08T06:01:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.703+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/27216206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 msgr2=0x55f2ce1f9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.703+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/27216206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1f9c70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009680 tx=0x7fa47c00ec30 comp rx=0 tx=0).stop 2024-05-08T06:01:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.703+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/27216206 shutdown_connections 2024-05-08T06:01:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.703+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/27216206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e89d0 0x55f2ce2056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.703+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/27216206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e4060 0x55f2ce1f9c70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.703+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/27216206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e4980 0x55f2ce1eef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.703+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/27216206 >> 172.21.15.112:0/27216206 conn(0x55f2ce112b60 msgr2=0x55f2ce1ee850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.703+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/27216206 shutdown_connections 2024-05-08T06:01:17.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.703+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/27216206 wait complete. 2024-05-08T06:01:17.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.704+0000 7fa49bcc0b80 1 -- start start 2024-05-08T06:01:17.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.704+0000 7fa49bcc0b80 1 Processor -- start 2024-05-08T06:01:17.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa49bcc0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e4060 0x55f2ce1f6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa49bcc0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 0x55f2ce1f6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa49bcc0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e89d0 0x55f2ce1f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa49bcc0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1ae310 con 0x55f2ce1e4980 2024-05-08T06:01:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa49bcc0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1af780 con 0x55f2ce1e4060 2024-05-08T06:01:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa49bcc0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1ae9e0 con 0x55f2ce1e89d0 2024-05-08T06:01:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa49bcc0b80 10 client.0 _ll_register_callbacks cb 0x55f2ce0ddc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:01:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa48a800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e4060 0x55f2ce1f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa48a800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e4060 0x55f2ce1f6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:35754/0 (socket says 172.21.15.112:35754) 2024-05-08T06:01:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa48a800700 1 -- 172.21.15.112:0/747219257 learned_addr learned my addr 172.21.15.112:0/747219257 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa489e00700 1 --2- 172.21.15.112:0/747219257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 0x55f2ce1f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa48b200700 1 --2- 172.21.15.112:0/747219257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e89d0 0x55f2ce1f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa48a800700 1 -- 172.21.15.112:0/747219257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e89d0 msgr2=0x55f2ce1f0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:17.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa48a800700 1 --2- 172.21.15.112:0/747219257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e89d0 0x55f2ce1f0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa48a800700 1 -- 172.21.15.112:0/747219257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 msgr2=0x55f2ce1f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:17.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa48a800700 1 --2- 172.21.15.112:0/747219257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 0x55f2ce1f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.705+0000 7fa48a800700 1 -- 172.21.15.112:0/747219257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2ce1f17b0 con 0x55f2ce1e4060 2024-05-08T06:01:17.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.706+0000 7fa489e00700 1 --2- 172.21.15.112:0/747219257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 0x55f2ce1f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:01:17.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.706+0000 7fa48b200700 1 --2- 172.21.15.112:0/747219257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e89d0 0x55f2ce1f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:01:17.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.706+0000 7fa48a800700 1 --2- 172.21.15.112:0/747219257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e4060 0x55f2ce1f6000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005950 tx=0x7fa47c00b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67dfcc998c6563aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:17.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.706+0000 7fa474c00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:01:17.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.706+0000 7fa474c00700 1 -- 172.21.15.112:0/747219257 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0a9f60 con 0x55f2ce1e4060 2024-05-08T06:01:17.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.706+0000 7fa474c00700 1 -- 172.21.15.112:0/747219257 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fa47c0a6de0 con 0x55f2ce1e4060 2024-05-08T06:01:17.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.706+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/747219257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa47c0aa300 con 0x55f2ce1e4060 2024-05-08T06:01:17.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.706+0000 7fa474c00700 1 -- 172.21.15.112:0/747219257 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa47c0aeba0 con 0x55f2ce1e4060 2024-05-08T06:01:17.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.706+0000 7fa49bcc0b80 10 client.25808 fetch_fsmap learned FSMap version 32 2024-05-08T06:01:17.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.706+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/747219257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f2ce1f17b0 con 0x55f2ce1e4060 2024-05-08T06:01:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.706+0000 7fa474c00700 1 -- 172.21.15.112:0/747219257 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa47c0aa300 con 0x55f2ce1e4060 2024-05-08T06:01:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.707+0000 7fa474c00700 1 -- 172.21.15.112:0/747219257 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 32) ==== 8342+0+0 (secure 0 0 0) 0x7fa47c0bfc40 con 0x55f2ce1e4060 2024-05-08T06:01:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.707+0000 7fa49bcc0b80 10 client.25808 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T06:01:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.707+0000 7fa49bcc0b80 10 client.25808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:01:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.707+0000 7fa49bcc0b80 20 client.25808 populate_metadata read hostname 'smithi112' 2024-05-08T06:01:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.707+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/747219257 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f2ce268ff0 0x55f2ce26b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.707+0000 7fa49bcc0b80 4 client.25808 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T06:01:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.707+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/747219257 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f2ce144400 con 0x55f2ce268ff0 2024-05-08T06:01:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.707+0000 7fa489e00700 1 --2- 172.21.15.112:0/747219257 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f2ce268ff0 0x55f2ce26b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.708+0000 7fa489e00700 1 --2- 172.21.15.112:0/747219257 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f2ce268ff0 0x55f2ce26b3e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39cd5dfe4c7725a8 server_cookie=31ceec08c8998bfe in_seq=0 out_seq=0 2024-05-08T06:01:17.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.708+0000 7fa474c00700 10 client.25808 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:17.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.708+0000 7fa474c00700 1 -- 172.21.15.112:0/747219257 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f2ce144400 con 0x55f2ce268ff0 2024-05-08T06:01:17.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.708+0000 7fa474c00700 10 client.25808 handle_command_reply: tid=0 2024-05-08T06:01:17.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.708+0000 7fa49bcc0b80 1 client.25808 shutdown 2024-05-08T06:01:17.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.708+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/747219257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e4060 msgr2=0x55f2ce1f6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:17.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.708+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/747219257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e4060 0x55f2ce1f6000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005950 tx=0x7fa47c00b990 comp rx=0 tx=0).stop 2024-05-08T06:01:17.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.708+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/747219257 shutdown_connections 2024-05-08T06:01:17.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.708+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/747219257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f2ce1e89d0 0x55f2ce1f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.708+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/747219257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f2ce1e4060 0x55f2ce1f6000 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.708+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/747219257 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x55f2ce268ff0 0x55f2ce26b3e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.708+0000 7fa49bcc0b80 1 --2- 172.21.15.112:0/747219257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2ce1e4980 0x55f2ce1f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.708+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/747219257 >> 172.21.15.112:0/747219257 conn(0x55f2ce112b60 msgr2=0x55f2ce1ec230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:17.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.708+0000 7fa474c00700 0 client.25808 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:17.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.709+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/747219257 shutdown_connections 2024-05-08T06:01:17.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.709+0000 7fa49bcc0b80 1 -- 172.21.15.112:0/747219257 wait complete. 2024-05-08T06:01:17.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.709+0000 7fa49bcc0b80 20 client.25808 trim_cache size 0 max 16384 2024-05-08T06:01:17.713 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:01:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T06:01:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-05-08T06:01:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: "5e0780d5-c341-4a03-b461-d5f801c396c5": { 2024-05-08T06:01:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-05-08T06:01:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "5e0780d5-c341-4a03-b461-d5f801c396c5", 2024-05-08T06:01:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-05-08T06:01:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:01:17.713 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:01:17.713 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:01:17.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.710+0000 7fa491400700 1 -- 172.21.15.112:0/1278241219 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa4700b2900 msgr2=0x7fa4700b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:17.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.710+0000 7fa491400700 1 --2- 172.21.15.112:0/1278241219 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa4700b2900 0x7fa4700b4dc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa4840076c0 tx=0x7fa484009040 comp rx=0 tx=0).stop 2024-05-08T06:01:17.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.710+0000 7fa491400700 1 -- 172.21.15.112:0/1278241219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c1056e0 msgr2=0x7fa48c19bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:17.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.710+0000 7fa491400700 1 --2- 172.21.15.112:0/1278241219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c1056e0 0x7fa48c19bdf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009230 tx=0x7fa47c00fa70 comp rx=0 tx=0).stop 2024-05-08T06:01:17.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.710+0000 7fa491400700 1 -- 172.21.15.112:0/1278241219 shutdown_connections 2024-05-08T06:01:17.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.710+0000 7fa491400700 1 --2- 172.21.15.112:0/1278241219 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa4700b2900 0x7fa4700b4dc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.710+0000 7fa491400700 1 --2- 172.21.15.112:0/1278241219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa48c10b6e0 0x7fa48c1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.710+0000 7fa491400700 1 --2- 172.21.15.112:0/1278241219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa48c1056e0 0x7fa48c19bdf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.710+0000 7fa491400700 1 --2- 172.21.15.112:0/1278241219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa48c1060a0 0x7fa48c19c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.710+0000 7fa491400700 1 -- 172.21.15.112:0/1278241219 >> 172.21.15.112:0/1278241219 conn(0x7fa48c1011d0 msgr2=0x7fa48c101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:17.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.710+0000 7fa491400700 1 -- 172.21.15.112:0/1278241219 shutdown_connections 2024-05-08T06:01:17.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:17.710+0000 7fa491400700 1 -- 172.21.15.112:0/1278241219 wait complete. 2024-05-08T06:01:17.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:17 smithi162 ceph-mon[34800]: pgmap v620: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.1 MiB/s wr, 481 op/s 2024-05-08T06:01:18.581 INFO:tasks.cephfs.filesystem:scrub status for tag:5e0780d5-c341-4a03-b461-d5f801c396c5 - {'path': '/', 'tag': '5e0780d5-c341-4a03-b461-d5f801c396c5', 'options': 'recursive,force'} 2024-05-08T06:01:19.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:19 smithi112 ceph-mon[30003]: pgmap v621: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1008 KiB/s wr, 433 op/s 2024-05-08T06:01:19.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:19 smithi150 ceph-mon[34315]: pgmap v621: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1008 KiB/s wr, 433 op/s 2024-05-08T06:01:19.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:19 smithi162 ceph-mon[34800]: pgmap v621: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1008 KiB/s wr, 433 op/s 2024-05-08T06:01:21.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:21 smithi150 ceph-mon[34315]: pgmap v622: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 486 op/s 2024-05-08T06:01:21.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:21 smithi112 ceph-mon[30003]: pgmap v622: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 486 op/s 2024-05-08T06:01:21.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:21 smithi162 ceph-mon[34800]: pgmap v622: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 486 op/s 2024-05-08T06:01:23.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:23 smithi150 ceph-mon[34315]: pgmap v623: 97 pgs: 97 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 470 op/s 2024-05-08T06:01:23.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:23 smithi112 ceph-mon[30003]: pgmap v623: 97 pgs: 97 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 470 op/s 2024-05-08T06:01:23.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:23 smithi162 ceph-mon[34800]: pgmap v623: 97 pgs: 97 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 470 op/s 2024-05-08T06:01:25.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:25 smithi150 ceph-mon[34315]: pgmap v624: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1000 KiB/s wr, 388 op/s 2024-05-08T06:01:25.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:25 smithi112 ceph-mon[30003]: pgmap v624: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1000 KiB/s wr, 388 op/s 2024-05-08T06:01:25.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:25 smithi162 ceph-mon[34800]: pgmap v624: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1000 KiB/s wr, 388 op/s 2024-05-08T06:01:26.710 INFO:tasks.workunit.client.0.smithi112.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-05-08T06:01:26.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:26 smithi112 ceph-mon[30003]: pgmap v625: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.3 MiB/s wr, 477 op/s 2024-05-08T06:01:26.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:26 smithi162 ceph-mon[34800]: pgmap v625: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.3 MiB/s wr, 477 op/s 2024-05-08T06:01:26.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:26 smithi150 ceph-mon[34315]: pgmap v625: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.3 MiB/s wr, 477 op/s 2024-05-08T06:01:29.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:28 smithi112 ceph-mon[30003]: pgmap v626: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.1 MiB/s wr, 396 op/s 2024-05-08T06:01:29.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:28 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T06:01:29.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:28 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T06:01:29.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:28 smithi162 ceph-mon[34800]: pgmap v626: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.1 MiB/s wr, 396 op/s 2024-05-08T06:01:29.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:28 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T06:01:29.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:28 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T06:01:29.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:28 smithi150 ceph-mon[34315]: pgmap v626: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.1 MiB/s wr, 396 op/s 2024-05-08T06:01:29.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:28 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T06:01:29.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:28 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T06:01:30.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:29 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T06:01:30.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:29 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:01:30.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:29 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T06:01:30.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:29 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:01:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:29 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T06:01:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:29 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:01:31.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:30 smithi112 ceph-mon[30003]: pgmap v627: 97 pgs: 97 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.2 MiB/s wr, 423 op/s 2024-05-08T06:01:31.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:30 smithi162 ceph-mon[34800]: pgmap v627: 97 pgs: 97 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.2 MiB/s wr, 423 op/s 2024-05-08T06:01:31.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:30 smithi150 ceph-mon[34315]: pgmap v627: 97 pgs: 97 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.2 MiB/s wr, 423 op/s 2024-05-08T06:01:33.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:32 smithi150 ceph-mon[34315]: pgmap v628: 97 pgs: 97 active+clean; 130 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.1 MiB/s wr, 399 op/s 2024-05-08T06:01:33.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:32 smithi112 ceph-mon[30003]: pgmap v628: 97 pgs: 97 active+clean; 130 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.1 MiB/s wr, 399 op/s 2024-05-08T06:01:33.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:32 smithi162 ceph-mon[34800]: pgmap v628: 97 pgs: 97 active+clean; 130 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.1 MiB/s wr, 399 op/s 2024-05-08T06:01:35.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:34 smithi150 ceph-mon[34315]: pgmap v629: 97 pgs: 97 active+clean; 126 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 975 KiB/s wr, 331 op/s 2024-05-08T06:01:35.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:34 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:35.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:34 smithi162 ceph-mon[34800]: pgmap v629: 97 pgs: 97 active+clean; 126 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 975 KiB/s wr, 331 op/s 2024-05-08T06:01:35.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:34 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:35.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:34 smithi112 ceph-mon[30003]: pgmap v629: 97 pgs: 97 active+clean; 126 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 975 KiB/s wr, 331 op/s 2024-05-08T06:01:35.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:34 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:37.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:36 smithi150 ceph-mon[34315]: pgmap v630: 97 pgs: 97 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.2 MiB/s wr, 429 op/s 2024-05-08T06:01:37.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:36 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:37.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:36 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:37.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:36 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:01:37.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:36 smithi112 ceph-mon[30003]: pgmap v630: 97 pgs: 97 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.2 MiB/s wr, 429 op/s 2024-05-08T06:01:37.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:36 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:37.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:36 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:37.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:36 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:01:37.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:36 smithi162 ceph-mon[34800]: pgmap v630: 97 pgs: 97 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.2 MiB/s wr, 429 op/s 2024-05-08T06:01:37.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:36 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:37.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:36 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:37.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:36 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:01:39.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:38 smithi150 ceph-mon[34315]: pgmap v631: 97 pgs: 97 active+clean; 130 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1003 KiB/s wr, 374 op/s 2024-05-08T06:01:39.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:38 smithi112 ceph-mon[30003]: pgmap v631: 97 pgs: 97 active+clean; 130 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1003 KiB/s wr, 374 op/s 2024-05-08T06:01:39.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:38 smithi162 ceph-mon[34800]: pgmap v631: 97 pgs: 97 active+clean; 130 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1003 KiB/s wr, 374 op/s 2024-05-08T06:01:41.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:40 smithi150 ceph-mon[34315]: pgmap v632: 97 pgs: 97 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.0 MiB/s wr, 406 op/s 2024-05-08T06:01:41.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:40 smithi112 ceph-mon[30003]: pgmap v632: 97 pgs: 97 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.0 MiB/s wr, 406 op/s 2024-05-08T06:01:41.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:40 smithi162 ceph-mon[34800]: pgmap v632: 97 pgs: 97 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.0 MiB/s wr, 406 op/s 2024-05-08T06:01:43.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:42 smithi150 ceph-mon[34315]: pgmap v633: 97 pgs: 97 active+clean; 126 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1016 KiB/s wr, 397 op/s 2024-05-08T06:01:43.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:42 smithi112 ceph-mon[30003]: pgmap v633: 97 pgs: 97 active+clean; 126 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1016 KiB/s wr, 397 op/s 2024-05-08T06:01:43.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:42 smithi162 ceph-mon[34800]: pgmap v633: 97 pgs: 97 active+clean; 126 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1016 KiB/s wr, 397 op/s 2024-05-08T06:01:45.278 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x63e5d2a 2024-05-08T06:01:45.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:45 smithi112 ceph-mon[30003]: pgmap v634: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 833 KiB/s wr, 333 op/s 2024-05-08T06:01:45.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:45 smithi162 ceph-mon[34800]: pgmap v634: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 833 KiB/s wr, 333 op/s 2024-05-08T06:01:45.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:45 smithi150 ceph-mon[34315]: pgmap v634: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 833 KiB/s wr, 333 op/s 2024-05-08T06:01:47.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:47 smithi112 ceph-mon[30003]: pgmap v635: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.0 MiB/s wr, 410 op/s 2024-05-08T06:01:47.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:47 smithi162 ceph-mon[34800]: pgmap v635: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.0 MiB/s wr, 410 op/s 2024-05-08T06:01:47.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:47 smithi150 ceph-mon[34315]: pgmap v635: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.0 MiB/s wr, 410 op/s 2024-05-08T06:01:48.582 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T06:01:49.064 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:01:49.426 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:49 smithi112 ceph-mon[30003]: pgmap v636: 97 pgs: 97 active+clean; 120 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 863 KiB/s wr, 335 op/s 2024-05-08T06:01:49.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:49 smithi162 ceph-mon[34800]: pgmap v636: 97 pgs: 97 active+clean; 120 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 863 KiB/s wr, 335 op/s 2024-05-08T06:01:49.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:49 smithi150 ceph-mon[34315]: pgmap v636: 97 pgs: 97 active+clean; 120 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 863 KiB/s wr, 335 op/s 2024-05-08T06:01:51.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:51 smithi112 ceph-mon[30003]: pgmap v637: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 940 KiB/s wr, 365 op/s 2024-05-08T06:01:51.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:51 smithi162 ceph-mon[34800]: pgmap v637: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 940 KiB/s wr, 365 op/s 2024-05-08T06:01:51.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:51 smithi150 ceph-mon[34315]: pgmap v637: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 940 KiB/s wr, 365 op/s 2024-05-08T06:01:51.807 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x63e9b52 2024-05-08T06:01:52.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.165+0000 7fd450e00700 1 -- 172.21.15.112:0/4267451648 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd44c10ff40 msgr2=0x7fd44c112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:52.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.165+0000 7fd450e00700 1 --2- 172.21.15.112:0/4267451648 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd44c10ff40 0x7fd44c112330 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd44400b0a0 tx=0x7fd44400b3b0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.165+0000 7fd450e00700 1 -- 172.21.15.112:0/4267451648 shutdown_connections 2024-05-08T06:01:52.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.165+0000 7fd450e00700 1 --2- 172.21.15.112:0/4267451648 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd44c10ff40 0x7fd44c112330 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.165+0000 7fd450e00700 1 --2- 172.21.15.112:0/4267451648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd44c06ba40 0x7fd44c06be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.165+0000 7fd450e00700 1 --2- 172.21.15.112:0/4267451648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44c06c360 0x7fd44c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.165+0000 7fd450e00700 1 -- 172.21.15.112:0/4267451648 >> 172.21.15.112:0/4267451648 conn(0x7fd44c079310 msgr2=0x7fd44c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:52.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.165+0000 7fd450e00700 1 -- 172.21.15.112:0/4267451648 shutdown_connections 2024-05-08T06:01:52.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.165+0000 7fd450e00700 1 -- 172.21.15.112:0/4267451648 wait complete. 2024-05-08T06:01:52.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.166+0000 7fd450e00700 1 Processor -- start 2024-05-08T06:01:52.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.167+0000 7fd450e00700 1 -- start start 2024-05-08T06:01:52.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.167+0000 7fd450e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44c06ba40 0x7fd44c19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.167+0000 7fd450e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd44c06c360 0x7fd44c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.167+0000 7fd450e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd44c10ff40 0x7fd44c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.167+0000 7fd450e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1158b0 con 0x7fd44c10ff40 2024-05-08T06:01:52.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.167+0000 7fd450e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c115730 con 0x7fd44c06ba40 2024-05-08T06:01:52.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.167+0000 7fd450e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c115a30 con 0x7fd44c06c360 2024-05-08T06:01:52.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.167+0000 7fd449e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44c06ba40 0x7fd44c19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.167+0000 7fd449e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44c06ba40 0x7fd44c19bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:59092/0 (socket says 172.21.15.112:59092) 2024-05-08T06:01:52.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.167+0000 7fd449e00700 1 -- 172.21.15.112:0/2420443820 learned_addr learned my addr 172.21.15.112:0/2420443820 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:52.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.168+0000 7fd449400700 1 --2- 172.21.15.112:0/2420443820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd44c06c360 0x7fd44c19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.168+0000 7fd44a800700 1 --2- 172.21.15.112:0/2420443820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd44c10ff40 0x7fd44c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.168+0000 7fd449e00700 1 -- 172.21.15.112:0/2420443820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd44c06c360 msgr2=0x7fd44c19c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:52.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.168+0000 7fd449e00700 1 --2- 172.21.15.112:0/2420443820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd44c06c360 0x7fd44c19c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.168+0000 7fd449e00700 1 -- 172.21.15.112:0/2420443820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd44c10ff40 msgr2=0x7fd44c1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:52.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.168+0000 7fd449e00700 1 --2- 172.21.15.112:0/2420443820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd44c10ff40 0x7fd44c1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.168+0000 7fd449e00700 1 -- 172.21.15.112:0/2420443820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c1a0d90 con 0x7fd44c06ba40 2024-05-08T06:01:52.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.168+0000 7fd449e00700 1 --2- 172.21.15.112:0/2420443820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44c06ba40 0x7fd44c19bd80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00e9a0 tx=0x7fd43c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:52.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.168+0000 7fd442a00700 1 -- 172.21.15.112:0/2420443820 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c00cbd0 con 0x7fd44c06ba40 2024-05-08T06:01:52.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.169+0000 7fd442a00700 1 -- 172.21.15.112:0/2420443820 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd43c00cd70 con 0x7fd44c06ba40 2024-05-08T06:01:52.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.169+0000 7fd450e00700 1 -- 172.21.15.112:0/2420443820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44c1a1080 con 0x7fd44c06ba40 2024-05-08T06:01:52.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.169+0000 7fd442a00700 1 -- 172.21.15.112:0/2420443820 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c018680 con 0x7fd44c06ba40 2024-05-08T06:01:52.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.169+0000 7fd450e00700 1 -- 172.21.15.112:0/2420443820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd44c1a8900 con 0x7fd44c06ba40 2024-05-08T06:01:52.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.172+0000 7fd442a00700 1 -- 172.21.15.112:0/2420443820 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd43c010430 con 0x7fd44c06ba40 2024-05-08T06:01:52.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.174+0000 7fd442a00700 1 --2- 172.21.15.112:0/2420443820 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4300b2900 0x7fd4300b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.174+0000 7fd45b719b80 1 Processor -- start 2024-05-08T06:01:52.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.174+0000 7fd442a00700 1 -- 172.21.15.112:0/2420443820 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd43c0e24b0 con 0x7fd44c06ba40 2024-05-08T06:01:52.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.174+0000 7fd449400700 1 --2- 172.21.15.112:0/2420443820 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4300b2900 0x7fd4300b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.174+0000 7fd45b719b80 1 -- start start 2024-05-08T06:01:52.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd45b719b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe3bc0 0x5622b0fe3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd45b719b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe44e0 0x5622b0fde2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd45b719b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fde8f0 0x5622b0fded70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd45b719b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5622b0f30a20 con 0x5622b0fe3bc0 2024-05-08T06:01:52.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd45b719b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5622b0ece760 con 0x5622b0fde8f0 2024-05-08T06:01:52.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd45b719b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5622b0f2c9e0 con 0x5622b0fe44e0 2024-05-08T06:01:52.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd449400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe44e0 0x5622b0fde2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd449e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe3bc0 0x5622b0fe3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd44a800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fde8f0 0x5622b0fded70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd449e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe3bc0 0x5622b0fe3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33762/0 (socket says 172.21.15.112:33762) 2024-05-08T06:01:52.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd449400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe44e0 0x5622b0fde2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:55096/0 (socket says 172.21.15.112:55096) 2024-05-08T06:01:52.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd449e00700 1 -- 172.21.15.112:0/2072584491 learned_addr learned my addr 172.21.15.112:0/2072584491 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:52.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd449400700 1 --2- 172.21.15.112:0/2420443820 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4300b2900 0x7fd4300b4dc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd44c103880 tx=0x7fd438009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:52.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd44a800700 1 -- 172.21.15.112:0/2072584491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe44e0 msgr2=0x5622b0fde2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:52.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd44a800700 1 --2- 172.21.15.112:0/2072584491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe44e0 0x5622b0fde2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd44a800700 1 -- 172.21.15.112:0/2072584491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe3bc0 msgr2=0x5622b0fe3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:52.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.175+0000 7fd44a800700 1 --2- 172.21.15.112:0/2072584491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe3bc0 0x5622b0fe3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.176+0000 7fd44a800700 1 -- 172.21.15.112:0/2072584491 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622b0e7d8d0 con 0x5622b0fde8f0 2024-05-08T06:01:52.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.176+0000 7fd44a800700 1 --2- 172.21.15.112:0/2072584491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fde8f0 0x5622b0fded70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd44400b070 tx=0x7fd4440092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2b027c1eb46da9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:52.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.176+0000 7fd437400700 1 -- 172.21.15.112:0/2072584491 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44400bcb0 con 0x5622b0fde8f0 2024-05-08T06:01:52.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.176+0000 7fd437400700 1 -- 172.21.15.112:0/2072584491 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4440074e0 con 0x5622b0fde8f0 2024-05-08T06:01:52.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.176+0000 7fd437400700 1 -- 172.21.15.112:0/2072584491 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444005260 con 0x5622b0fde8f0 2024-05-08T06:01:52.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.177+0000 7fd45b719b80 1 -- 172.21.15.112:0/2072584491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fde8f0 msgr2=0x5622b0fded70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:52.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.177+0000 7fd45b719b80 1 --2- 172.21.15.112:0/2072584491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fde8f0 0x5622b0fded70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd44400b070 tx=0x7fd4440092e0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.177+0000 7fd45b719b80 1 -- 172.21.15.112:0/2072584491 shutdown_connections 2024-05-08T06:01:52.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.177+0000 7fd45b719b80 1 --2- 172.21.15.112:0/2072584491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fde8f0 0x5622b0fded70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.177+0000 7fd45b719b80 1 --2- 172.21.15.112:0/2072584491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe3bc0 0x5622b0fe3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.177+0000 7fd45b719b80 1 --2- 172.21.15.112:0/2072584491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe44e0 0x5622b0fde2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.177+0000 7fd45b719b80 1 -- 172.21.15.112:0/2072584491 >> 172.21.15.112:0/2072584491 conn(0x5622b0f566d0 msgr2=0x5622b0fd8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:52.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.177+0000 7fd45b719b80 1 -- 172.21.15.112:0/2072584491 shutdown_connections 2024-05-08T06:01:52.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.178+0000 7fd45b719b80 1 -- 172.21.15.112:0/2072584491 wait complete. 2024-05-08T06:01:52.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.178+0000 7fd45b719b80 1 -- start start 2024-05-08T06:01:52.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.178+0000 7fd45b719b80 1 Processor -- start 2024-05-08T06:01:52.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.179+0000 7fd45b719b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fde8f0 0x5622b0fea1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.179+0000 7fd45b719b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe3bc0 0x5622b0fe7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.179+0000 7fd45b719b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe44e0 0x5622b0fe7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.179+0000 7fd45b719b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5622b0feaec0 con 0x5622b0fe44e0 2024-05-08T06:01:52.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.179+0000 7fd45b719b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5622b0e96b20 con 0x5622b0fde8f0 2024-05-08T06:01:52.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.179+0000 7fd45b719b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5622b0feb040 con 0x5622b0fe3bc0 2024-05-08T06:01:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.179+0000 7fd45b719b80 10 client.0 _ll_register_callbacks cb 0x5622b0ed7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:01:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.179+0000 7fd449e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fde8f0 0x5622b0fea1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.179+0000 7fd449e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fde8f0 0x5622b0fea1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:59108/0 (socket says 172.21.15.112:59108) 2024-05-08T06:01:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.179+0000 7fd449e00700 1 -- 172.21.15.112:0/1116711755 learned_addr learned my addr 172.21.15.112:0/1116711755 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.179+0000 7fd44a800700 1 --2- 172.21.15.112:0/1116711755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe44e0 0x5622b0fe7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.179+0000 7fd449e00700 1 -- 172.21.15.112:0/1116711755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe3bc0 msgr2=0x5622b0fe7570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T06:01:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.179+0000 7fd449e00700 1 --2- 172.21.15.112:0/1116711755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe3bc0 0x5622b0fe7570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.179+0000 7fd449e00700 1 -- 172.21.15.112:0/1116711755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe44e0 msgr2=0x5622b0fe7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.180+0000 7fd449e00700 1 --2- 172.21.15.112:0/1116711755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe44e0 0x5622b0fe7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.180+0000 7fd449e00700 1 -- 172.21.15.112:0/1116711755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622b0ef6420 con 0x5622b0fde8f0 2024-05-08T06:01:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.180+0000 7fd44a800700 1 --2- 172.21.15.112:0/1116711755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe44e0 0x5622b0fe7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:01:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.180+0000 7fd449e00700 1 --2- 172.21.15.112:0/1116711755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fde8f0 0x5622b0fea1d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00c5b0 tx=0x7fd43c0b26f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1320d4af54539e81 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.180+0000 7fd42be00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:01:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.180+0000 7fd42be00700 1 -- 172.21.15.112:0/1116711755 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0c3060 con 0x5622b0fde8f0 2024-05-08T06:01:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.180+0000 7fd42be00700 1 -- 172.21.15.112:0/1116711755 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd43c0cc280 con 0x5622b0fde8f0 2024-05-08T06:01:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.180+0000 7fd45b719b80 1 -- 172.21.15.112:0/1116711755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4440074e0 con 0x5622b0fde8f0 2024-05-08T06:01:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.181+0000 7fd42be00700 1 -- 172.21.15.112:0/1116711755 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd43c0b2040 con 0x5622b0fde8f0 2024-05-08T06:01:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.181+0000 7fd45b719b80 10 client.25820 fetch_fsmap learned FSMap version 33 2024-05-08T06:01:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.181+0000 7fd45b719b80 1 -- 172.21.15.112:0/1116711755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5622b0ff57e0 con 0x5622b0fde8f0 2024-05-08T06:01:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.181+0000 7fd42be00700 1 -- 172.21.15.112:0/1116711755 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4440074e0 con 0x5622b0fde8f0 2024-05-08T06:01:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.181+0000 7fd42be00700 1 -- 172.21.15.112:0/1116711755 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7fd43c0a53b0 con 0x5622b0fde8f0 2024-05-08T06:01:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.181+0000 7fd45b719b80 10 client.25820 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:01:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.181+0000 7fd45b719b80 10 client.25820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:01:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.181+0000 7fd45b719b80 20 client.25820 populate_metadata read hostname 'smithi112' 2024-05-08T06:01:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.181+0000 7fd45b719b80 1 --2- 172.21.15.112:0/1116711755 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622b0ff7e20 0x5622b0ffa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.181+0000 7fd45b719b80 4 client.25820 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:01:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.181+0000 7fd45b719b80 1 -- 172.21.15.112:0/1116711755 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622b0fab140 con 0x5622b0ff7e20 2024-05-08T06:01:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.182+0000 7fd449400700 1 --2- 172.21.15.112:0/1116711755 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622b0ff7e20 0x5622b0ffa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.183+0000 7fd42be00700 10 client.25820 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.183+0000 7fd449400700 1 --2- 172.21.15.112:0/1116711755 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622b0ff7e20 0x5622b0ffa230 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aa5ecc13c1de1d5 server_cookie=a6424912fa909039 in_seq=0 out_seq=0 2024-05-08T06:01:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.184+0000 7fd42be00700 1 -- 172.21.15.112:0/1116711755 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5622b0fab140 con 0x5622b0ff7e20 2024-05-08T06:01:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.184+0000 7fd42be00700 10 client.25820 handle_command_reply: tid=0 2024-05-08T06:01:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.184+0000 7fd45b719b80 1 client.25820 shutdown 2024-05-08T06:01:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.184+0000 7fd45b719b80 1 -- 172.21.15.112:0/1116711755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fde8f0 msgr2=0x5622b0fea1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.184+0000 7fd45b719b80 1 --2- 172.21.15.112:0/1116711755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fde8f0 0x5622b0fea1d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00c5b0 tx=0x7fd43c0b26f0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.185+0000 7fd45b719b80 1 -- 172.21.15.112:0/1116711755 shutdown_connections 2024-05-08T06:01:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.185+0000 7fd45b719b80 1 --2- 172.21.15.112:0/1116711755 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622b0ff7e20 0x5622b0ffa230 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.185+0000 7fd45b719b80 1 --2- 172.21.15.112:0/1116711755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe44e0 0x5622b0fe7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.185+0000 7fd45b719b80 1 --2- 172.21.15.112:0/1116711755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fde8f0 0x5622b0fea1d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.185+0000 7fd45b719b80 1 --2- 172.21.15.112:0/1116711755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe3bc0 0x5622b0fe7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.185+0000 7fd45b719b80 1 -- 172.21.15.112:0/1116711755 >> 172.21.15.112:0/1116711755 conn(0x5622b0f566d0 msgr2=0x5622b0fd8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.185+0000 7fd42be00700 0 client.25820 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.185+0000 7fd45b719b80 1 -- 172.21.15.112:0/1116711755 shutdown_connections 2024-05-08T06:01:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.186+0000 7fd45b719b80 1 -- 172.21.15.112:0/1116711755 wait complete. 2024-05-08T06:01:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.186+0000 7fd45b719b80 20 client.25820 trim_cache size 0 max 16384 2024-05-08T06:01:52.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.211+0000 7fd45b719b80 1 Processor -- start 2024-05-08T06:01:52.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.211+0000 7fd45b719b80 1 -- start start 2024-05-08T06:01:52.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.212+0000 7fd45b719b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fde8f0 0x5622b0fde2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.212+0000 7fd45b719b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe3bc0 0x5622b0ff3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.212+0000 7fd45b719b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fe44e0 0x5622b0fd8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.212+0000 7fd45b719b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1b2610 con 0x5622b0fde8f0 2024-05-08T06:01:52.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.212+0000 7fd45b719b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1afea0 con 0x5622b0fe44e0 2024-05-08T06:01:52.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.212+0000 7fd45b719b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1adcf0 con 0x5622b0fe3bc0 2024-05-08T06:01:52.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.212+0000 7fd449400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe3bc0 0x5622b0ff3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.212+0000 7fd449e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fde8f0 0x5622b0fde2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.212+0000 7fd449400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe3bc0 0x5622b0ff3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:55114/0 (socket says 172.21.15.112:55114) 2024-05-08T06:01:52.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.212+0000 7fd449400700 1 -- 172.21.15.112:0/3608096123 learned_addr learned my addr 172.21.15.112:0/3608096123 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:52.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.212+0000 7fd44a800700 1 --2- 172.21.15.112:0/3608096123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fe44e0 0x5622b0fd8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.213+0000 7fd449400700 1 -- 172.21.15.112:0/3608096123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fe44e0 msgr2=0x5622b0fd8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:52.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.213+0000 7fd449400700 1 --2- 172.21.15.112:0/3608096123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fe44e0 0x5622b0fd8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.213+0000 7fd449400700 1 -- 172.21.15.112:0/3608096123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fde8f0 msgr2=0x5622b0fde2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:52.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.213+0000 7fd449400700 1 --2- 172.21.15.112:0/3608096123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fde8f0 0x5622b0fde2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.213+0000 7fd449400700 1 -- 172.21.15.112:0/3608096123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622b0fe41f0 con 0x5622b0fe3bc0 2024-05-08T06:01:52.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.213+0000 7fd44a800700 1 --2- 172.21.15.112:0/3608096123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fe44e0 0x5622b0fd8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:01:52.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.213+0000 7fd449400700 1 --2- 172.21.15.112:0/3608096123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe3bc0 0x5622b0ff3f50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd43800a010 tx=0x7fd438013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f87185b4e840164c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:52.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.213+0000 7fd436a00700 1 -- 172.21.15.112:0/3608096123 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43802d070 con 0x5622b0fe3bc0 2024-05-08T06:01:52.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.213+0000 7fd436a00700 1 -- 172.21.15.112:0/3608096123 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd438020110 con 0x5622b0fe3bc0 2024-05-08T06:01:52.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.213+0000 7fd436a00700 1 -- 172.21.15.112:0/3608096123 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438027740 con 0x5622b0fe3bc0 2024-05-08T06:01:52.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.214+0000 7fd45b719b80 1 -- 172.21.15.112:0/3608096123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe3bc0 msgr2=0x5622b0ff3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:52.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.214+0000 7fd45b719b80 1 --2- 172.21.15.112:0/3608096123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe3bc0 0x5622b0ff3f50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd43800a010 tx=0x7fd438013000 comp rx=0 tx=0).stop 2024-05-08T06:01:52.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.214+0000 7fd45b719b80 1 -- 172.21.15.112:0/3608096123 shutdown_connections 2024-05-08T06:01:52.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.214+0000 7fd45b719b80 1 --2- 172.21.15.112:0/3608096123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fe44e0 0x5622b0fd8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.214+0000 7fd45b719b80 1 --2- 172.21.15.112:0/3608096123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fde8f0 0x5622b0fde2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.214+0000 7fd45b719b80 1 --2- 172.21.15.112:0/3608096123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fe3bc0 0x5622b0ff3f50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.214+0000 7fd45b719b80 1 -- 172.21.15.112:0/3608096123 >> 172.21.15.112:0/3608096123 conn(0x5622b0f566d0 msgr2=0x5622b0fe26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:52.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.214+0000 7fd45b719b80 1 -- 172.21.15.112:0/3608096123 shutdown_connections 2024-05-08T06:01:52.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.214+0000 7fd45b719b80 1 -- 172.21.15.112:0/3608096123 wait complete. 2024-05-08T06:01:52.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.215+0000 7fd45b719b80 1 -- start start 2024-05-08T06:01:52.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.215+0000 7fd45b719b80 1 Processor -- start 2024-05-08T06:01:52.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.215+0000 7fd45b719b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fde8f0 0x5622b0ff04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd45b719b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fe3bc0 0x5622b0ff09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd45b719b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe44e0 0x5622b0fed300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd45b719b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1ae360 con 0x5622b0fe44e0 2024-05-08T06:01:52.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd45b719b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1af7d0 con 0x5622b0fe3bc0 2024-05-08T06:01:52.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd45b719b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1aea30 con 0x5622b0fde8f0 2024-05-08T06:01:52.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd45b719b80 10 client.0 _ll_register_callbacks cb 0x5622b0ed7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:01:52.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd449e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fde8f0 0x5622b0ff04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd449400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fe3bc0 0x5622b0ff09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd449e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fde8f0 0x5622b0ff04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:55116/0 (socket says 172.21.15.112:55116) 2024-05-08T06:01:52.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd449e00700 1 -- 172.21.15.112:0/2187973578 learned_addr learned my addr 172.21.15.112:0/2187973578 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:52.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd44a800700 1 --2- 172.21.15.112:0/2187973578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe44e0 0x5622b0fed300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd449e00700 1 -- 172.21.15.112:0/2187973578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fe3bc0 msgr2=0x5622b0ff09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:52.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd449e00700 1 --2- 172.21.15.112:0/2187973578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fe3bc0 0x5622b0ff09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd449e00700 1 -- 172.21.15.112:0/2187973578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe44e0 msgr2=0x5622b0fed300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:01:52.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd449e00700 1 --2- 172.21.15.112:0/2187973578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe44e0 0x5622b0fed300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.216+0000 7fd449e00700 1 -- 172.21.15.112:0/2187973578 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622b0fedb40 con 0x5622b0fde8f0 2024-05-08T06:01:52.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.217+0000 7fd449e00700 1 --2- 172.21.15.112:0/2187973578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fde8f0 0x5622b0ff04b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0a5ac0 tx=0x7fd43c0a98c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a4cd166295cfbf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:52.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.217+0000 7fd42be00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:01:52.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.217+0000 7fd42be00700 1 -- 172.21.15.112:0/2187973578 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0a51d0 con 0x5622b0fde8f0 2024-05-08T06:01:52.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.217+0000 7fd45b719b80 1 -- 172.21.15.112:0/2187973578 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd438020110 con 0x5622b0fde8f0 2024-05-08T06:01:52.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.217+0000 7fd42be00700 1 -- 172.21.15.112:0/2187973578 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd43c0a8ca0 con 0x5622b0fde8f0 2024-05-08T06:01:52.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.217+0000 7fd42be00700 1 -- 172.21.15.112:0/2187973578 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd43c0a9550 con 0x5622b0fde8f0 2024-05-08T06:01:52.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.217+0000 7fd45b719b80 10 client.25870 fetch_fsmap learned FSMap version 33 2024-05-08T06:01:52.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.217+0000 7fd45b719b80 1 -- 172.21.15.112:0/2187973578 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5622b1063270 con 0x5622b0fde8f0 2024-05-08T06:01:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.218+0000 7fd42be00700 1 -- 172.21.15.112:0/2187973578 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd438020110 con 0x5622b0fde8f0 2024-05-08T06:01:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.218+0000 7fd42be00700 1 -- 172.21.15.112:0/2187973578 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7fd43c0bb070 con 0x5622b0fde8f0 2024-05-08T06:01:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.218+0000 7fd45b719b80 10 client.25870 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:01:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.218+0000 7fd45b719b80 10 client.25870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:01:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.218+0000 7fd45b719b80 20 client.25870 populate_metadata read hostname 'smithi112' 2024-05-08T06:01:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.218+0000 7fd45b719b80 1 --2- 172.21.15.112:0/2187973578 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622b1063860 0x5622b1065c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.218+0000 7fd45b719b80 4 client.25870 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T06:01:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.218+0000 7fd45b719b80 1 -- 172.21.15.112:0/2187973578 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5622b0f3e400 con 0x5622b1063860 2024-05-08T06:01:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.219+0000 7fd449400700 1 --2- 172.21.15.112:0/2187973578 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622b1063860 0x5622b1065c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.219+0000 7fd449400700 1 --2- 172.21.15.112:0/2187973578 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622b1063860 0x5622b1065c50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=add391de82cfc54f server_cookie=d45cb291d22fbe86 in_seq=0 out_seq=0 2024-05-08T06:01:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.219+0000 7fd42be00700 10 client.25870 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.220+0000 7fd42be00700 1 -- 172.21.15.112:0/2187973578 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5622b0f3e400 con 0x5622b1063860 2024-05-08T06:01:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.220+0000 7fd42be00700 10 client.25870 handle_command_reply: tid=0 2024-05-08T06:01:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.220+0000 7fd45b719b80 1 client.25870 shutdown 2024-05-08T06:01:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.220+0000 7fd45b719b80 1 -- 172.21.15.112:0/2187973578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fde8f0 msgr2=0x5622b0ff04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.220+0000 7fd45b719b80 1 --2- 172.21.15.112:0/2187973578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fde8f0 0x5622b0ff04b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0a5ac0 tx=0x7fd43c0a98c0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.220+0000 7fd45b719b80 1 -- 172.21.15.112:0/2187973578 shutdown_connections 2024-05-08T06:01:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.220+0000 7fd45b719b80 1 --2- 172.21.15.112:0/2187973578 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622b1063860 0x5622b1065c50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.220+0000 7fd45b719b80 1 --2- 172.21.15.112:0/2187973578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622b0fe44e0 0x5622b0fed300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.220+0000 7fd45b719b80 1 --2- 172.21.15.112:0/2187973578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622b0fde8f0 0x5622b0ff04b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.220+0000 7fd45b719b80 1 --2- 172.21.15.112:0/2187973578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622b0fe3bc0 0x5622b0ff09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.220+0000 7fd45b719b80 1 -- 172.21.15.112:0/2187973578 >> 172.21.15.112:0/2187973578 conn(0x5622b0f566d0 msgr2=0x5622b0fe2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.220+0000 7fd42be00700 0 client.25870 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:01:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.221+0000 7fd45b719b80 1 -- 172.21.15.112:0/2187973578 shutdown_connections 2024-05-08T06:01:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.221+0000 7fd45b719b80 1 -- 172.21.15.112:0/2187973578 wait complete. 2024-05-08T06:01:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.221+0000 7fd45b719b80 20 client.25870 trim_cache size 0 max 16384 2024-05-08T06:01:52.224 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:01:52.224 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-08T06:01:52.224 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-08T06:01:52.224 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:01:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.222+0000 7fd450e00700 1 -- 172.21.15.112:0/2420443820 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4300b2900 msgr2=0x7fd4300b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.222+0000 7fd450e00700 1 --2- 172.21.15.112:0/2420443820 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4300b2900 0x7fd4300b4dc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd44c103880 tx=0x7fd438009260 comp rx=0 tx=0).stop 2024-05-08T06:01:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.222+0000 7fd450e00700 1 -- 172.21.15.112:0/2420443820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44c06ba40 msgr2=0x7fd44c19bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.222+0000 7fd450e00700 1 --2- 172.21.15.112:0/2420443820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44c06ba40 0x7fd44c19bd80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00e9a0 tx=0x7fd43c00ecb0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.222+0000 7fd450e00700 1 -- 172.21.15.112:0/2420443820 shutdown_connections 2024-05-08T06:01:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.222+0000 7fd450e00700 1 --2- 172.21.15.112:0/2420443820 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd4300b2900 0x7fd4300b4dc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.222+0000 7fd450e00700 1 --2- 172.21.15.112:0/2420443820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd44c10ff40 0x7fd44c1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.222+0000 7fd450e00700 1 --2- 172.21.15.112:0/2420443820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44c06ba40 0x7fd44c19bd80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.222+0000 7fd450e00700 1 --2- 172.21.15.112:0/2420443820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd44c06c360 0x7fd44c19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.222+0000 7fd450e00700 1 -- 172.21.15.112:0/2420443820 >> 172.21.15.112:0/2420443820 conn(0x7fd44c079310 msgr2=0x7fd44c1113f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.222+0000 7fd450e00700 1 -- 172.21.15.112:0/2420443820 shutdown_connections 2024-05-08T06:01:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:52.222+0000 7fd450e00700 1 -- 172.21.15.112:0/2420443820 wait complete. 2024-05-08T06:01:53.367 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T06:01:53.367 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:01:53.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:53 smithi112 ceph-mon[30003]: pgmap v638: 97 pgs: 97 active+clean; 119 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 941 KiB/s wr, 363 op/s 2024-05-08T06:01:53.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:53 smithi162 ceph-mon[34800]: pgmap v638: 97 pgs: 97 active+clean; 119 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 941 KiB/s wr, 363 op/s 2024-05-08T06:01:53.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:53 smithi150 ceph-mon[34315]: pgmap v638: 97 pgs: 97 active+clean; 119 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 941 KiB/s wr, 363 op/s 2024-05-08T06:01:53.815 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:01:55.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:55 smithi112 ceph-mon[30003]: pgmap v639: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 842 KiB/s wr, 306 op/s 2024-05-08T06:01:55.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:55 smithi162 ceph-mon[34800]: pgmap v639: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 842 KiB/s wr, 306 op/s 2024-05-08T06:01:55.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:55 smithi150 ceph-mon[34315]: pgmap v639: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 842 KiB/s wr, 306 op/s 2024-05-08T06:01:56.653 INFO:tasks.workunit.client.0.smithi112.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-05-08T06:01:56.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.856+0000 7f78ddc00700 1 -- 172.21.15.112:0/776081344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78d810fea0 msgr2=0x7f78d8112290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:56.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.857+0000 7f78ddc00700 1 --2- 172.21.15.112:0/776081344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78d810fea0 0x7f78d8112290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009f90 tx=0x7f78d000b330 comp rx=0 tx=0).stop 2024-05-08T06:01:56.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.857+0000 7f78ddc00700 1 -- 172.21.15.112:0/776081344 shutdown_connections 2024-05-08T06:01:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.857+0000 7f78ddc00700 1 --2- 172.21.15.112:0/776081344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78d810fea0 0x7f78d8112290 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.857+0000 7f78ddc00700 1 --2- 172.21.15.112:0/776081344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78d80756d0 0x7f78d8075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.857+0000 7f78ddc00700 1 --2- 172.21.15.112:0/776081344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78d8075ff0 0x7f78d810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.857+0000 7f78ddc00700 1 -- 172.21.15.112:0/776081344 >> 172.21.15.112:0/776081344 conn(0x7f78d807a600 msgr2=0x7f78d807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.857+0000 7f78ddc00700 1 -- 172.21.15.112:0/776081344 shutdown_connections 2024-05-08T06:01:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.857+0000 7f78ddc00700 1 -- 172.21.15.112:0/776081344 wait complete. 2024-05-08T06:01:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.858+0000 7f78ddc00700 1 Processor -- start 2024-05-08T06:01:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.858+0000 7f78ddc00700 1 -- start start 2024-05-08T06:01:56.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.858+0000 7f78ddc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78d80756d0 0x7f78d819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:56.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.858+0000 7f78ddc00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78d8075ff0 0x7f78d819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:56.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.858+0000 7f78ddc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78d810fea0 0x7f78d81a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:56.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.859+0000 7f78ddc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f78d8115840 con 0x7f78d810fea0 2024-05-08T06:01:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.859+0000 7f78ddc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81156c0 con 0x7f78d80756d0 2024-05-08T06:01:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.859+0000 7f78ddc00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81159c0 con 0x7f78d8075ff0 2024-05-08T06:01:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.859+0000 7f78d6a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78d80756d0 0x7f78d819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.859+0000 7f78d6000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78d8075ff0 0x7f78d819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.859+0000 7f78d6a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78d80756d0 0x7f78d819bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:51512/0 (socket says 172.21.15.112:51512) 2024-05-08T06:01:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.859+0000 7f78d6a00700 1 -- 172.21.15.112:0/1440995625 learned_addr learned my addr 172.21.15.112:0/1440995625 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:01:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.859+0000 7f78d6a00700 1 -- 172.21.15.112:0/1440995625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78d8075ff0 msgr2=0x7f78d819c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.859+0000 7f78d6a00700 1 --2- 172.21.15.112:0/1440995625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78d8075ff0 0x7f78d819c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.859+0000 7f78d6a00700 1 -- 172.21.15.112:0/1440995625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78d810fea0 msgr2=0x7f78d81a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.859+0000 7f78d6a00700 1 --2- 172.21.15.112:0/1440995625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78d810fea0 0x7f78d81a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.859+0000 7f78d6a00700 1 -- 172.21.15.112:0/1440995625 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d81a0d20 con 0x7f78d80756d0 2024-05-08T06:01:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.859+0000 7f78d6000700 1 --2- 172.21.15.112:0/1440995625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78d8075ff0 0x7f78d819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:01:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.860+0000 7f78d6a00700 1 --2- 172.21.15.112:0/1440995625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78d80756d0 0x7f78d819bd10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f78c800e9c0 tx=0x7f78c800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.860+0000 7f78cf400700 1 -- 172.21.15.112:0/1440995625 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c800cbd0 con 0x7f78d80756d0 2024-05-08T06:01:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.860+0000 7f78ddc00700 1 -- 172.21.15.112:0/1440995625 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d81a1010 con 0x7f78d80756d0 2024-05-08T06:01:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.860+0000 7f78cf400700 1 -- 172.21.15.112:0/1440995625 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78c800cd70 con 0x7f78d80756d0 2024-05-08T06:01:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.860+0000 7f78cf400700 1 -- 172.21.15.112:0/1440995625 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8005550 con 0x7f78d80756d0 2024-05-08T06:01:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.860+0000 7f78ddc00700 1 -- 172.21.15.112:0/1440995625 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d81a8890 con 0x7f78d80756d0 2024-05-08T06:01:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.862+0000 7f78ddc00700 1 -- 172.21.15.112:0/1440995625 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d806a190 con 0x7f78d80756d0 2024-05-08T06:01:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.863+0000 7f78cf400700 1 -- 172.21.15.112:0/1440995625 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f78c8004750 con 0x7f78d80756d0 2024-05-08T06:01:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.865+0000 7f78cf400700 1 --2- 172.21.15.112:0/1440995625 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f78bc0b2900 0x7f78bc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.865+0000 7f78cf400700 1 -- 172.21.15.112:0/1440995625 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f78c80e2460 con 0x7f78d80756d0 2024-05-08T06:01:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.865+0000 7f78d6000700 1 --2- 172.21.15.112:0/1440995625 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f78bc0b2900 0x7f78bc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:56.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.866+0000 7f78d6000700 1 --2- 172.21.15.112:0/1440995625 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f78bc0b2900 0x7f78bc0b4dc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f78d8111bc0 tx=0x7f78c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:56.869+0000 7f78cf400700 1 -- 172.21.15.112:0/1440995625 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f78c80aa0f0 con 0x7f78d80756d0 2024-05-08T06:01:57.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:57.222+0000 7f78ddc00700 1 -- 172.21.15.112:0/1440995625 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78d819d070 con 0x7f78d80756d0 2024-05-08T06:01:57.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:57.224+0000 7f78cf400700 1 -- 172.21.15.112:0/1440995625 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11891 (secure 0 0 0) 0x7f78c80a99f0 con 0x7f78d80756d0 2024-05-08T06:01:57.224 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T06:01:57.224 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T06:01:28.310032+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24518,24511]},"id":1}]} 2024-05-08T06:01:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:57.227+0000 7f78ddc00700 1 -- 172.21.15.112:0/1440995625 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f78bc0b2900 msgr2=0x7f78bc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:57.227+0000 7f78ddc00700 1 --2- 172.21.15.112:0/1440995625 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f78bc0b2900 0x7f78bc0b4dc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f78d8111bc0 tx=0x7f78c4009260 comp rx=0 tx=0).stop 2024-05-08T06:01:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:57.227+0000 7f78ddc00700 1 -- 172.21.15.112:0/1440995625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78d80756d0 msgr2=0x7f78d819bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:57.227+0000 7f78ddc00700 1 --2- 172.21.15.112:0/1440995625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78d80756d0 0x7f78d819bd10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f78c800e9c0 tx=0x7f78c800ecd0 comp rx=0 tx=0).stop 2024-05-08T06:01:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:57.228+0000 7f78ddc00700 1 -- 172.21.15.112:0/1440995625 shutdown_connections 2024-05-08T06:01:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:57.228+0000 7f78ddc00700 1 --2- 172.21.15.112:0/1440995625 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f78bc0b2900 0x7f78bc0b4dc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:57.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:57.228+0000 7f78ddc00700 1 --2- 172.21.15.112:0/1440995625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78d810fea0 0x7f78d81a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:57.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:57.228+0000 7f78ddc00700 1 --2- 172.21.15.112:0/1440995625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78d80756d0 0x7f78d819bd10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:57.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:57.228+0000 7f78ddc00700 1 --2- 172.21.15.112:0/1440995625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78d8075ff0 0x7f78d819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:57.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:57.228+0000 7f78ddc00700 1 -- 172.21.15.112:0/1440995625 >> 172.21.15.112:0/1440995625 conn(0x7f78d807a600 msgr2=0x7f78d8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:57.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:57.228+0000 7f78ddc00700 1 -- 172.21.15.112:0/1440995625 shutdown_connections 2024-05-08T06:01:57.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:01:57.228+0000 7f78ddc00700 1 -- 172.21.15.112:0/1440995625 wait complete. 2024-05-08T06:01:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 33 2024-05-08T06:01:57.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:57 smithi112 ceph-mon[30003]: pgmap v640: 97 pgs: 97 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.1 MiB/s wr, 417 op/s 2024-05-08T06:01:57.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:57 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1440995625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:01:57.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:57 smithi162 ceph-mon[34800]: pgmap v640: 97 pgs: 97 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.1 MiB/s wr, 417 op/s 2024-05-08T06:01:57.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:57 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1440995625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:01:57.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:57 smithi150 ceph-mon[34315]: pgmap v640: 97 pgs: 97 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.1 MiB/s wr, 417 op/s 2024-05-08T06:01:57.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:57 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1440995625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:01:58.214 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-08T06:01:58.689 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:01:59.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:01:59 smithi112 ceph-mon[30003]: pgmap v641: 97 pgs: 97 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 993 KiB/s wr, 375 op/s 2024-05-08T06:01:59.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:01:59 smithi162 ceph-mon[34800]: pgmap v641: 97 pgs: 97 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 993 KiB/s wr, 375 op/s 2024-05-08T06:01:59.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:01:59 smithi150 ceph-mon[34315]: pgmap v641: 97 pgs: 97 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 993 KiB/s wr, 375 op/s 2024-05-08T06:02:01.004 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x63ed6b9 2024-05-08T06:02:01.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.056+0000 7fec54a00700 1 -- 172.21.15.112:0/3470046665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec50103550 msgr2=0x7fec501039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:01.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.056+0000 7fec54a00700 1 --2- 172.21.15.112:0/3470046665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec50103550 0x7fec501039b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0098f0 tx=0x7fec3c009c00 comp rx=0 tx=0).stop 2024-05-08T06:02:01.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.057+0000 7fec54a00700 1 -- 172.21.15.112:0/3470046665 shutdown_connections 2024-05-08T06:02:01.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.057+0000 7fec54a00700 1 --2- 172.21.15.112:0/3470046665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec50103ef0 0x7fec501103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.057+0000 7fec54a00700 1 --2- 172.21.15.112:0/3470046665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec50108460 0x7fec50108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.057+0000 7fec54a00700 1 --2- 172.21.15.112:0/3470046665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec50103550 0x7fec501039b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.057+0000 7fec54a00700 1 -- 172.21.15.112:0/3470046665 >> 172.21.15.112:0/3470046665 conn(0x7fec5007a600 msgr2=0x7fec5007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:01.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.057+0000 7fec54a00700 1 -- 172.21.15.112:0/3470046665 shutdown_connections 2024-05-08T06:02:01.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.057+0000 7fec54a00700 1 -- 172.21.15.112:0/3470046665 wait complete. 2024-05-08T06:02:01.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.058+0000 7fec54a00700 1 Processor -- start 2024-05-08T06:02:01.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.058+0000 7fec54a00700 1 -- start start 2024-05-08T06:02:01.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.058+0000 7fec54a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec50103550 0x7fec5019be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.058+0000 7fec54a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec50103ef0 0x7fec5019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.059+0000 7fec54a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec50108460 0x7fec501a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.059+0000 7fec54a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec501136e0 con 0x7fec50108460 2024-05-08T06:02:01.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.059+0000 7fec54a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fec50113560 con 0x7fec50103550 2024-05-08T06:02:01.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.059+0000 7fec54a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fec50113860 con 0x7fec50103ef0 2024-05-08T06:02:01.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.059+0000 7fec4de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec50103550 0x7fec5019be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.059+0000 7fec4de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec50103550 0x7fec5019be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:51526/0 (socket says 172.21.15.112:51526) 2024-05-08T06:02:01.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.059+0000 7fec4d400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec50103ef0 0x7fec5019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.059+0000 7fec4de00700 1 -- 172.21.15.112:0/155283609 learned_addr learned my addr 172.21.15.112:0/155283609 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:01.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.059+0000 7fec4e800700 1 --2- 172.21.15.112:0/155283609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec50108460 0x7fec501a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.059+0000 7fec4de00700 1 -- 172.21.15.112:0/155283609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec50103ef0 msgr2=0x7fec5019c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:01.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.059+0000 7fec4de00700 1 --2- 172.21.15.112:0/155283609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec50103ef0 0x7fec5019c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.059+0000 7fec4de00700 1 -- 172.21.15.112:0/155283609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec50108460 msgr2=0x7fec501a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:01.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.059+0000 7fec4de00700 1 --2- 172.21.15.112:0/155283609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec50108460 0x7fec501a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.059+0000 7fec4de00700 1 -- 172.21.15.112:0/155283609 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec501a0e20 con 0x7fec50103550 2024-05-08T06:02:01.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.060+0000 7fec4e800700 1 --2- 172.21.15.112:0/155283609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec50108460 0x7fec501a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:02:01.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.060+0000 7fec4d400700 1 --2- 172.21.15.112:0/155283609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec50103ef0 0x7fec5019c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:02:01.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.060+0000 7fec4de00700 1 --2- 172.21.15.112:0/155283609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec50103550 0x7fec5019be10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fec4000e970 tx=0x7fec4000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:01.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.060+0000 7fec46a00700 1 -- 172.21.15.112:0/155283609 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4000cbd0 con 0x7fec50103550 2024-05-08T06:02:01.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.060+0000 7fec46a00700 1 -- 172.21.15.112:0/155283609 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec400044d0 con 0x7fec50103550 2024-05-08T06:02:01.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.060+0000 7fec54a00700 1 -- 172.21.15.112:0/155283609 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec501a1110 con 0x7fec50103550 2024-05-08T06:02:01.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.060+0000 7fec46a00700 1 -- 172.21.15.112:0/155283609 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec40003df0 con 0x7fec50103550 2024-05-08T06:02:01.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.061+0000 7fec54a00700 1 -- 172.21.15.112:0/155283609 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec501a8990 con 0x7fec50103550 2024-05-08T06:02:01.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.066+0000 7fec46a00700 1 -- 172.21.15.112:0/155283609 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fec40003f90 con 0x7fec50103550 2024-05-08T06:02:01.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.068+0000 7fec5f2afb80 1 Processor -- start 2024-05-08T06:02:01.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.068+0000 7fec46a00700 1 --2- 172.21.15.112:0/155283609 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fec340b2900 0x7fec340b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.068+0000 7fec5f2afb80 1 -- start start 2024-05-08T06:02:01.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.068+0000 7fec46a00700 1 -- 172.21.15.112:0/155283609 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fec400e23a0 con 0x7fec50103550 2024-05-08T06:02:01.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.068+0000 7fec4e800700 1 --2- 172.21.15.112:0/155283609 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fec340b2900 0x7fec340b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.068+0000 7fec5f2afb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd7bc0 0x55d97edd7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.068+0000 7fec5f2afb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 0x55d97edd22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.068+0000 7fec5f2afb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 0x55d97edd2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.069+0000 7fec5f2afb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d97ed24a20 con 0x55d97edd28f0 2024-05-08T06:02:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.069+0000 7fec5f2afb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d97ecc2760 con 0x55d97edd84e0 2024-05-08T06:02:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.069+0000 7fec5f2afb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d97ed209e0 con 0x55d97edd7bc0 2024-05-08T06:02:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.069+0000 7fec4de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd7bc0 0x55d97edd7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.069+0000 7fec4e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 0x55d97edd2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.069+0000 7fec4de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd7bc0 0x55d97edd7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:43424/0 (socket says 172.21.15.112:43424) 2024-05-08T06:02:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.069+0000 7fec4de00700 1 -- 172.21.15.112:0/1553389536 learned_addr learned my addr 172.21.15.112:0/1553389536 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.069+0000 7fec4d400700 1 --2- 172.21.15.112:0/1553389536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 0x55d97edd22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.069+0000 7fec4de00700 1 -- 172.21.15.112:0/1553389536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 msgr2=0x55d97edd22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.069+0000 7fec4de00700 1 --2- 172.21.15.112:0/1553389536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 0x55d97edd22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.069+0000 7fec4de00700 1 -- 172.21.15.112:0/1553389536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 msgr2=0x55d97edd2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.069+0000 7fec4de00700 1 --2- 172.21.15.112:0/1553389536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 0x55d97edd2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.069+0000 7fec4de00700 1 -- 172.21.15.112:0/1553389536 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d97ec718d0 con 0x55d97edd7bc0 2024-05-08T06:02:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.069+0000 7fec4e800700 1 --2- 172.21.15.112:0/1553389536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 0x55d97edd2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.070+0000 7fec4de00700 1 --2- 172.21.15.112:0/1553389536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd7bc0 0x55d97edd7fa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec400a9690 tx=0x7fec400a9850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bc4b0862d303884 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.070+0000 7fec4e800700 1 --2- 172.21.15.112:0/155283609 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fec340b2900 0x7fec340b4dc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fec480077b0 tx=0x7fec48007ac0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.070+0000 7fec3b400700 1 -- 172.21.15.112:0/1553389536 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4000ce00 con 0x55d97edd7bc0 2024-05-08T06:02:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.070+0000 7fec3b400700 1 -- 172.21.15.112:0/1553389536 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec400a94a0 con 0x55d97edd7bc0 2024-05-08T06:02:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.070+0000 7fec3b400700 1 -- 172.21.15.112:0/1553389536 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec400b27f0 con 0x55d97edd7bc0 2024-05-08T06:02:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.071+0000 7fec5f2afb80 1 -- 172.21.15.112:0/1553389536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd7bc0 msgr2=0x55d97edd7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.071+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/1553389536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd7bc0 0x55d97edd7fa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec400a9690 tx=0x7fec400a9850 comp rx=0 tx=0).stop 2024-05-08T06:02:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.071+0000 7fec5f2afb80 1 -- 172.21.15.112:0/1553389536 shutdown_connections 2024-05-08T06:02:01.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.071+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/1553389536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 0x55d97edd2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.071+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/1553389536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd7bc0 0x55d97edd7fa0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.071+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/1553389536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 0x55d97edd22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.071+0000 7fec5f2afb80 1 -- 172.21.15.112:0/1553389536 >> 172.21.15.112:0/1553389536 conn(0x55d97ed4a6d0 msgr2=0x55d97edccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.071+0000 7fec5f2afb80 1 -- 172.21.15.112:0/1553389536 shutdown_connections 2024-05-08T06:02:01.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.071+0000 7fec5f2afb80 1 -- 172.21.15.112:0/1553389536 wait complete. 2024-05-08T06:02:01.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.072+0000 7fec5f2afb80 1 -- start start 2024-05-08T06:02:01.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.072+0000 7fec5f2afb80 1 Processor -- start 2024-05-08T06:02:01.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.072+0000 7fec5f2afb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 0x55d97edde150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.072+0000 7fec5f2afb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd7bc0 0x55d97eddb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.072+0000 7fec5f2afb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd84e0 0x55d97eddbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.072+0000 7fec5f2afb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d97eddef20 con 0x55d97edd28f0 2024-05-08T06:02:01.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.072+0000 7fec5f2afb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d97ec8ab20 con 0x55d97edd7bc0 2024-05-08T06:02:01.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.072+0000 7fec5f2afb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d97eddf0a0 con 0x55d97edd84e0 2024-05-08T06:02:01.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.073+0000 7fec5f2afb80 10 client.0 _ll_register_callbacks cb 0x55d97eccbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:02:01.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.073+0000 7fec4e800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd84e0 0x55d97eddbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.073+0000 7fec4d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd7bc0 0x55d97eddb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.073+0000 7fec4d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd7bc0 0x55d97eddb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:51556/0 (socket says 172.21.15.112:51556) 2024-05-08T06:02:01.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.073+0000 7fec4e800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd84e0 0x55d97eddbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:43426/0 (socket says 172.21.15.112:43426) 2024-05-08T06:02:01.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.073+0000 7fec4d400700 1 -- 172.21.15.112:0/3289976358 learned_addr learned my addr 172.21.15.112:0/3289976358 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:01.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.073+0000 7fec4de00700 1 --2- 172.21.15.112:0/3289976358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 0x55d97edde150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.073+0000 7fec4d400700 1 -- 172.21.15.112:0/3289976358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd84e0 msgr2=0x55d97eddbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:01.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.073+0000 7fec4d400700 1 --2- 172.21.15.112:0/3289976358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd84e0 0x55d97eddbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.073+0000 7fec4d400700 1 -- 172.21.15.112:0/3289976358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 msgr2=0x55d97edde150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:01.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.073+0000 7fec4d400700 1 --2- 172.21.15.112:0/3289976358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 0x55d97edde150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.073+0000 7fec4d400700 1 -- 172.21.15.112:0/3289976358 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d97ecea420 con 0x55d97edd7bc0 2024-05-08T06:02:01.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.073+0000 7fec4de00700 1 --2- 172.21.15.112:0/3289976358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 0x55d97edde150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:02:01.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.074+0000 7fec4d400700 1 --2- 172.21.15.112:0/3289976358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd7bc0 0x55d97eddb560 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009230 tx=0x7fec3c009440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a06362a9384ba50 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:01.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.074+0000 7fec2fe00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:02:01.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.074+0000 7fec2fe00700 1 -- 172.21.15.112:0/3289976358 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c005510 con 0x55d97edd7bc0 2024-05-08T06:02:01.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.074+0000 7fec2fe00700 1 -- 172.21.15.112:0/3289976358 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fec3c003680 con 0x55d97edd7bc0 2024-05-08T06:02:01.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.074+0000 7fec5f2afb80 1 -- 172.21.15.112:0/3289976358 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec400a94a0 con 0x55d97edd7bc0 2024-05-08T06:02:01.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.075+0000 7fec2fe00700 1 -- 172.21.15.112:0/3289976358 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fec3c0056b0 con 0x55d97edd7bc0 2024-05-08T06:02:01.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.075+0000 7fec5f2afb80 10 client.25844 fetch_fsmap learned FSMap version 33 2024-05-08T06:02:01.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.075+0000 7fec5f2afb80 1 -- 172.21.15.112:0/3289976358 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d97ede9790 con 0x55d97edd7bc0 2024-05-08T06:02:01.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.075+0000 7fec2fe00700 1 -- 172.21.15.112:0/3289976358 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec400a94a0 con 0x55d97edd7bc0 2024-05-08T06:02:01.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.075+0000 7fec2fe00700 1 -- 172.21.15.112:0/3289976358 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7fec3c00ba90 con 0x55d97edd7bc0 2024-05-08T06:02:01.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.075+0000 7fec5f2afb80 10 client.25844 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:02:01.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.075+0000 7fec5f2afb80 10 client.25844 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:02:01.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.075+0000 7fec5f2afb80 20 client.25844 populate_metadata read hostname 'smithi112' 2024-05-08T06:02:01.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.075+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/3289976358 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d97edebdd0 0x55d97edee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.075+0000 7fec5f2afb80 4 client.25844 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:02:01.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.075+0000 7fec5f2afb80 1 -- 172.21.15.112:0/3289976358 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d97ed9f140 con 0x55d97edebdd0 2024-05-08T06:02:01.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.076+0000 7fec4de00700 1 --2- 172.21.15.112:0/3289976358 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d97edebdd0 0x55d97edee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.076+0000 7fec2fe00700 10 client.25844 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T06:02:01.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.076+0000 7fec4de00700 1 --2- 172.21.15.112:0/3289976358 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d97edebdd0 0x55d97edee1e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5d140d502ac73d9 server_cookie=e69dda9ff084d0f1 in_seq=0 out_seq=0 2024-05-08T06:02:01.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.077+0000 7fec2fe00700 1 -- 172.21.15.112:0/3289976358 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d97ed9f140 con 0x55d97edebdd0 2024-05-08T06:02:01.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.077+0000 7fec2fe00700 10 client.25844 handle_command_reply: tid=0 2024-05-08T06:02:01.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.077+0000 7fec5f2afb80 1 client.25844 shutdown 2024-05-08T06:02:01.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.077+0000 7fec5f2afb80 1 -- 172.21.15.112:0/3289976358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd7bc0 msgr2=0x55d97eddb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:01.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.077+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/3289976358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd7bc0 0x55d97eddb560 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009230 tx=0x7fec3c009440 comp rx=0 tx=0).stop 2024-05-08T06:02:01.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.077+0000 7fec5f2afb80 1 -- 172.21.15.112:0/3289976358 shutdown_connections 2024-05-08T06:02:01.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.077+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/3289976358 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d97edebdd0 0x55d97edee1e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.077+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/3289976358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd84e0 0x55d97eddbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.077+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/3289976358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 0x55d97edde150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.077+0000 7fec2fe00700 0 client.25844 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T06:02:01.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.077+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/3289976358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd7bc0 0x55d97eddb560 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.077+0000 7fec5f2afb80 1 -- 172.21.15.112:0/3289976358 >> 172.21.15.112:0/3289976358 conn(0x55d97ed4a6d0 msgr2=0x55d97edcc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:01.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.078+0000 7fec5f2afb80 1 -- 172.21.15.112:0/3289976358 shutdown_connections 2024-05-08T06:02:01.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.078+0000 7fec5f2afb80 1 -- 172.21.15.112:0/3289976358 wait complete. 2024-05-08T06:02:01.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.078+0000 7fec5f2afb80 20 client.25844 trim_cache size 0 max 16384 2024-05-08T06:02:01.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.098+0000 7fec5f2afb80 1 Processor -- start 2024-05-08T06:02:01.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.098+0000 7fec5f2afb80 1 -- start start 2024-05-08T06:02:01.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.099+0000 7fec5f2afb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd28f0 0x55d97ede7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.099+0000 7fec5f2afb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd7bc0 0x55d97edcc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.099+0000 7fec5f2afb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 0x55d97edf3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.099+0000 7fec5f2afb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec501b2870 con 0x55d97edd7bc0 2024-05-08T06:02:01.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.099+0000 7fec5f2afb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fec501b0100 con 0x55d97edd84e0 2024-05-08T06:02:01.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.099+0000 7fec5f2afb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fec501adf50 con 0x55d97edd28f0 2024-05-08T06:02:01.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.099+0000 7fec4de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd28f0 0x55d97ede7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.099+0000 7fec4d400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd7bc0 0x55d97edcc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.099+0000 7fec4de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd28f0 0x55d97ede7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:43442/0 (socket says 172.21.15.112:43442) 2024-05-08T06:02:01.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.099+0000 7fec4de00700 1 -- 172.21.15.112:0/954725624 learned_addr learned my addr 172.21.15.112:0/954725624 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:01.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.099+0000 7fec4e800700 1 --2- 172.21.15.112:0/954725624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 0x55d97edf3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.100+0000 7fec4de00700 1 -- 172.21.15.112:0/954725624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 msgr2=0x55d97edf3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:01.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.100+0000 7fec4de00700 1 --2- 172.21.15.112:0/954725624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 0x55d97edf3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.100+0000 7fec4de00700 1 -- 172.21.15.112:0/954725624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd7bc0 msgr2=0x55d97edcc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:01.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.100+0000 7fec4de00700 1 --2- 172.21.15.112:0/954725624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd7bc0 0x55d97edcc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.100+0000 7fec4de00700 1 -- 172.21.15.112:0/954725624 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d97edc7ab0 con 0x55d97edd28f0 2024-05-08T06:02:01.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.100+0000 7fec4e800700 1 --2- 172.21.15.112:0/954725624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 0x55d97edf3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:01.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.100+0000 7fec4de00700 1 --2- 172.21.15.112:0/954725624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd28f0 0x55d97ede7fb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fec400046e0 tx=0x7fec40003890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2181aeb30c1c2fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:01.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.100+0000 7fec3aa00700 1 -- 172.21.15.112:0/954725624 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4008e070 con 0x55d97edd28f0 2024-05-08T06:02:01.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.100+0000 7fec3aa00700 1 -- 172.21.15.112:0/954725624 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec400c2040 con 0x55d97edd28f0 2024-05-08T06:02:01.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.101+0000 7fec3aa00700 1 -- 172.21.15.112:0/954725624 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec400add40 con 0x55d97edd28f0 2024-05-08T06:02:01.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.102+0000 7fec5f2afb80 1 -- 172.21.15.112:0/954725624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd28f0 msgr2=0x55d97ede7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:01.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.102+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/954725624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd28f0 0x55d97ede7fb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fec400046e0 tx=0x7fec40003890 comp rx=0 tx=0).stop 2024-05-08T06:02:01.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.102+0000 7fec5f2afb80 1 -- 172.21.15.112:0/954725624 shutdown_connections 2024-05-08T06:02:01.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.102+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/954725624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 0x55d97edf3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.102+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/954725624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd28f0 0x55d97ede7fb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.102+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/954725624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd7bc0 0x55d97edcc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.102+0000 7fec5f2afb80 1 -- 172.21.15.112:0/954725624 >> 172.21.15.112:0/954725624 conn(0x55d97ed4a6d0 msgr2=0x55d97edd6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:01.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.102+0000 7fec5f2afb80 1 -- 172.21.15.112:0/954725624 shutdown_connections 2024-05-08T06:02:01.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.102+0000 7fec5f2afb80 1 -- 172.21.15.112:0/954725624 wait complete. 2024-05-08T06:02:01.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.102+0000 7fec5f2afb80 1 -- start start 2024-05-08T06:02:01.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.103+0000 7fec5f2afb80 1 Processor -- start 2024-05-08T06:02:01.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.103+0000 7fec5f2afb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 0x55d97ede4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.103+0000 7fec5f2afb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd7bc0 0x55d97ede19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.103+0000 7fec5f2afb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 0x55d97ede1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.103+0000 7fec5f2afb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec501ae5c0 con 0x55d97edd28f0 2024-05-08T06:02:01.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.103+0000 7fec5f2afb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fec501afa30 con 0x55d97edd84e0 2024-05-08T06:02:01.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.103+0000 7fec5f2afb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fec501aec90 con 0x55d97edd7bc0 2024-05-08T06:02:01.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.103+0000 7fec5f2afb80 10 client.0 _ll_register_callbacks cb 0x55d97eccbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:02:01.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.103+0000 7fec4d400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd7bc0 0x55d97ede19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.103+0000 7fec4d400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd7bc0 0x55d97ede19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:43452/0 (socket says 172.21.15.112:43452) 2024-05-08T06:02:01.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.103+0000 7fec4d400700 1 -- 172.21.15.112:0/2569984350 learned_addr learned my addr 172.21.15.112:0/2569984350 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:01.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.104+0000 7fec4de00700 1 --2- 172.21.15.112:0/2569984350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 0x55d97ede4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.104+0000 7fec4e800700 1 --2- 172.21.15.112:0/2569984350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 0x55d97ede1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.104+0000 7fec4d400700 1 -- 172.21.15.112:0/2569984350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 msgr2=0x55d97ede1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:01.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.104+0000 7fec4d400700 1 --2- 172.21.15.112:0/2569984350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 0x55d97ede1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.104+0000 7fec4d400700 1 -- 172.21.15.112:0/2569984350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 msgr2=0x55d97ede4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:01.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.104+0000 7fec4d400700 1 --2- 172.21.15.112:0/2569984350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 0x55d97ede4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.104+0000 7fec4d400700 1 -- 172.21.15.112:0/2569984350 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d97ede2510 con 0x55d97edd7bc0 2024-05-08T06:02:01.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.104+0000 7fec4de00700 1 --2- 172.21.15.112:0/2569984350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 0x55d97ede4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:01.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.104+0000 7fec4d400700 1 --2- 172.21.15.112:0/2569984350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd7bc0 0x55d97ede19e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00c010 tx=0x7fec3c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1518a8aab77099b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:01.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.104+0000 7fec2fe00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:02:01.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.104+0000 7fec2fe00700 1 -- 172.21.15.112:0/2569984350 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c03d060 con 0x55d97edd7bc0 2024-05-08T06:02:01.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.104+0000 7fec5f2afb80 1 -- 172.21.15.112:0/2569984350 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec400c2040 con 0x55d97edd7bc0 2024-05-08T06:02:01.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.105+0000 7fec2fe00700 1 -- 172.21.15.112:0/2569984350 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fec3c00b560 con 0x55d97edd7bc0 2024-05-08T06:02:01.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.105+0000 7fec2fe00700 1 -- 172.21.15.112:0/2569984350 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fec3c023e20 con 0x55d97edd7bc0 2024-05-08T06:02:01.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.105+0000 7fec5f2afb80 10 client.25894 fetch_fsmap learned FSMap version 33 2024-05-08T06:02:01.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.105+0000 7fec5f2afb80 1 -- 172.21.15.112:0/2569984350 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d97ee56470 con 0x55d97edd7bc0 2024-05-08T06:02:01.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.105+0000 7fec2fe00700 1 -- 172.21.15.112:0/2569984350 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec400c2040 con 0x55d97edd7bc0 2024-05-08T06:02:01.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.105+0000 7fec2fe00700 1 -- 172.21.15.112:0/2569984350 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7fec3c003c50 con 0x55d97edd7bc0 2024-05-08T06:02:01.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.105+0000 7fec5f2afb80 10 client.25894 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:02:01.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.105+0000 7fec5f2afb80 10 client.25894 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:02:01.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.105+0000 7fec5f2afb80 20 client.25894 populate_metadata read hostname 'smithi112' 2024-05-08T06:02:01.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.105+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/2569984350 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d97ee56b20 0x55d97ee58f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:01.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.105+0000 7fec5f2afb80 4 client.25894 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:02:01.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.105+0000 7fec5f2afb80 1 -- 172.21.15.112:0/2569984350 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d97ed32400 con 0x55d97ee56b20 2024-05-08T06:02:01.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.106+0000 7fec4de00700 1 --2- 172.21.15.112:0/2569984350 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d97ee56b20 0x55d97ee58f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:01.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.106+0000 7fec2fe00700 10 client.25894 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T06:02:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.106+0000 7fec4de00700 1 --2- 172.21.15.112:0/2569984350 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d97ee56b20 0x55d97ee58f40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cdfcb9495de18f87 server_cookie=9545595c22c84d40 in_seq=0 out_seq=0 2024-05-08T06:02:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.106+0000 7fec2fe00700 1 -- 172.21.15.112:0/2569984350 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d97ed32400 con 0x55d97ee56b20 2024-05-08T06:02:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.106+0000 7fec2fe00700 10 client.25894 handle_command_reply: tid=0 2024-05-08T06:02:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.106+0000 7fec5f2afb80 1 client.25894 shutdown 2024-05-08T06:02:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.107+0000 7fec5f2afb80 1 -- 172.21.15.112:0/2569984350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd7bc0 msgr2=0x55d97ede19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.107+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/2569984350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd7bc0 0x55d97ede19e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00c010 tx=0x7fec3c009560 comp rx=0 tx=0).stop 2024-05-08T06:02:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.107+0000 7fec5f2afb80 1 -- 172.21.15.112:0/2569984350 shutdown_connections 2024-05-08T06:02:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.107+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/2569984350 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55d97ee56b20 0x55d97ee58f40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.107+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/2569984350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d97edd84e0 0x55d97ede1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.107+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/2569984350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d97edd28f0 0x55d97ede4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.107+0000 7fec5f2afb80 1 --2- 172.21.15.112:0/2569984350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d97edd7bc0 0x55d97ede19e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.107+0000 7fec5f2afb80 1 -- 172.21.15.112:0/2569984350 >> 172.21.15.112:0/2569984350 conn(0x55d97ed4a6d0 msgr2=0x55d97eddcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.107+0000 7fec2fe00700 0 client.25894 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T06:02:01.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.107+0000 7fec5f2afb80 1 -- 172.21.15.112:0/2569984350 shutdown_connections 2024-05-08T06:02:01.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.107+0000 7fec5f2afb80 1 -- 172.21.15.112:0/2569984350 wait complete. 2024-05-08T06:02:01.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.107+0000 7fec5f2afb80 20 client.25894 trim_cache size 0 max 16384 2024-05-08T06:02:01.111 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T06:02:01.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.108+0000 7fec54a00700 1 -- 172.21.15.112:0/155283609 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fec340b2900 msgr2=0x7fec340b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:01.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.108+0000 7fec54a00700 1 --2- 172.21.15.112:0/155283609 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fec340b2900 0x7fec340b4dc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fec480077b0 tx=0x7fec48007ac0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.108+0000 7fec54a00700 1 -- 172.21.15.112:0/155283609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec50103550 msgr2=0x7fec5019be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:01.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.108+0000 7fec54a00700 1 --2- 172.21.15.112:0/155283609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec50103550 0x7fec5019be10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fec4000e970 tx=0x7fec4000ec80 comp rx=0 tx=0).stop 2024-05-08T06:02:01.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.108+0000 7fec54a00700 1 -- 172.21.15.112:0/155283609 shutdown_connections 2024-05-08T06:02:01.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.108+0000 7fec54a00700 1 --2- 172.21.15.112:0/155283609 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fec340b2900 0x7fec340b4dc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.108+0000 7fec54a00700 1 --2- 172.21.15.112:0/155283609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec50108460 0x7fec501a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.108+0000 7fec54a00700 1 --2- 172.21.15.112:0/155283609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec50103550 0x7fec5019be10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.108+0000 7fec54a00700 1 --2- 172.21.15.112:0/155283609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec50103ef0 0x7fec5019c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:01.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.109+0000 7fec54a00700 1 -- 172.21.15.112:0/155283609 >> 172.21.15.112:0/155283609 conn(0x7fec5007a600 msgr2=0x7fec501013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:01.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.109+0000 7fec54a00700 1 -- 172.21.15.112:0/155283609 shutdown_connections 2024-05-08T06:02:01.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:01.109+0000 7fec54a00700 1 -- 172.21.15.112:0/155283609 wait complete. 2024-05-08T06:02:01.559 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:01 smithi112 ceph-mon[30003]: pgmap v642: 97 pgs: 97 active+clean; 121 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.1 MiB/s wr, 427 op/s 2024-05-08T06:02:01.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:01 smithi162 ceph-mon[34800]: pgmap v642: 97 pgs: 97 active+clean; 121 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.1 MiB/s wr, 427 op/s 2024-05-08T06:02:01.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:01 smithi150 ceph-mon[34315]: pgmap v642: 97 pgs: 97 active+clean; 121 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.1 MiB/s wr, 427 op/s 2024-05-08T06:02:02.088 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-08T06:02:02.532 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:02:03.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:03 smithi112 ceph-mon[30003]: pgmap v643: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.1 MiB/s wr, 434 op/s 2024-05-08T06:02:03.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:03 smithi162 ceph-mon[34800]: pgmap v643: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.1 MiB/s wr, 434 op/s 2024-05-08T06:02:03.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:03 smithi150 ceph-mon[34315]: pgmap v643: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.1 MiB/s wr, 434 op/s 2024-05-08T06:02:04.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.866+0000 7f131e600700 1 -- 172.21.15.112:0/1795066214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f131810fea0 msgr2=0x7f1318112290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:04.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.866+0000 7f131e600700 1 --2- 172.21.15.112:0/1795066214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f131810fea0 0x7f1318112290 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f131000b0a0 tx=0x7f131000b3b0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.867+0000 7f131e600700 1 -- 172.21.15.112:0/1795066214 shutdown_connections 2024-05-08T06:02:04.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.867+0000 7f131e600700 1 --2- 172.21.15.112:0/1795066214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f131810fea0 0x7f1318112290 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.867+0000 7f131e600700 1 --2- 172.21.15.112:0/1795066214 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13180756d0 0x7f1318075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.867+0000 7f131e600700 1 --2- 172.21.15.112:0/1795066214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1318075ff0 0x7f131810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.867+0000 7f131e600700 1 -- 172.21.15.112:0/1795066214 >> 172.21.15.112:0/1795066214 conn(0x7f131807a600 msgr2=0x7f131807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:04.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.868+0000 7f131e600700 1 -- 172.21.15.112:0/1795066214 shutdown_connections 2024-05-08T06:02:04.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.868+0000 7f131e600700 1 -- 172.21.15.112:0/1795066214 wait complete. 2024-05-08T06:02:04.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.869+0000 7f131e600700 1 Processor -- start 2024-05-08T06:02:04.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.869+0000 7f131e600700 1 -- start start 2024-05-08T06:02:04.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.869+0000 7f131e600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13180756d0 0x7f131819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.869+0000 7f131e600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1318075ff0 0x7f131819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.869+0000 7f131e600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f131810fea0 0x7f13181a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.869+0000 7f131e600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1318115830 con 0x7f131810fea0 2024-05-08T06:02:04.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.869+0000 7f131e600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f13181156b0 con 0x7f13180756d0 2024-05-08T06:02:04.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.869+0000 7f131e600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f13181159b0 con 0x7f1318075ff0 2024-05-08T06:02:04.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.870+0000 7f1317400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13180756d0 0x7f131819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.870+0000 7f1317e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f131810fea0 0x7f13181a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.870+0000 7f1316a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1318075ff0 0x7f131819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.870+0000 7f1317400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13180756d0 0x7f131819bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:57246/0 (socket says 172.21.15.112:57246) 2024-05-08T06:02:04.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.870+0000 7f1317400700 1 -- 172.21.15.112:0/3129434575 learned_addr learned my addr 172.21.15.112:0/3129434575 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:04.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.870+0000 7f1316a00700 1 -- 172.21.15.112:0/3129434575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13180756d0 msgr2=0x7f131819bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:04.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.870+0000 7f1316a00700 1 --2- 172.21.15.112:0/3129434575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13180756d0 0x7f131819bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.870+0000 7f1316a00700 1 -- 172.21.15.112:0/3129434575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f131810fea0 msgr2=0x7f13181a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:04.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.870+0000 7f1316a00700 1 --2- 172.21.15.112:0/3129434575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f131810fea0 0x7f13181a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.870+0000 7f1316a00700 1 -- 172.21.15.112:0/3129434575 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13181a0d40 con 0x7f1318075ff0 2024-05-08T06:02:04.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.870+0000 7f1316a00700 1 --2- 172.21.15.112:0/3129434575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1318075ff0 0x7f131819c2c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f130400e9c0 tx=0x7f130400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:04.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.871+0000 7f130fe00700 1 -- 172.21.15.112:0/3129434575 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304004220 con 0x7f1318075ff0 2024-05-08T06:02:04.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.871+0000 7f131e600700 1 -- 172.21.15.112:0/3129434575 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13181a1030 con 0x7f1318075ff0 2024-05-08T06:02:04.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.871+0000 7f130fe00700 1 -- 172.21.15.112:0/3129434575 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13040043c0 con 0x7f1318075ff0 2024-05-08T06:02:04.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.871+0000 7f130fe00700 1 -- 172.21.15.112:0/3129434575 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304003c50 con 0x7f1318075ff0 2024-05-08T06:02:04.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.871+0000 7f131e600700 1 -- 172.21.15.112:0/3129434575 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13181034f0 con 0x7f1318075ff0 2024-05-08T06:02:04.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.876+0000 7f130fe00700 1 -- 172.21.15.112:0/3129434575 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f130400c7c0 con 0x7f1318075ff0 2024-05-08T06:02:04.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.876+0000 7f1328f1db80 1 Processor -- start 2024-05-08T06:02:04.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.877+0000 7f1328f1db80 1 -- start start 2024-05-08T06:02:04.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.877+0000 7f1328f1db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81060 0x55b5cbd81440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.877+0000 7f1328f1db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbd85490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.877+0000 7f1328f1db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd859d0 0x55b5cbd85e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.877+0000 7f1328f1db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b5cbcd3a20 con 0x55b5cbd81060 2024-05-08T06:02:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.877+0000 7f1328f1db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b5cbc71760 con 0x55b5cbd81980 2024-05-08T06:02:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.877+0000 7f1328f1db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55b5cbccf9e0 con 0x55b5cbd859d0 2024-05-08T06:02:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.877+0000 7f1317400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbd85490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.877+0000 7f1317400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbd85490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:57256/0 (socket says 172.21.15.112:57256) 2024-05-08T06:02:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.877+0000 7f1317400700 1 -- 172.21.15.112:0/2929636264 learned_addr learned my addr 172.21.15.112:0/2929636264 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.877+0000 7f1317e00700 1 --2- 172.21.15.112:0/2929636264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81060 0x55b5cbd81440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.878+0000 7f1317400700 1 -- 172.21.15.112:0/2929636264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd859d0 msgr2=0x55b5cbd85e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T06:02:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.878+0000 7f1317400700 1 --2- 172.21.15.112:0/2929636264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd859d0 0x55b5cbd85e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.878+0000 7f1317400700 1 -- 172.21.15.112:0/2929636264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81060 msgr2=0x55b5cbd81440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:04.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.878+0000 7f1317400700 1 --2- 172.21.15.112:0/2929636264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81060 0x55b5cbd81440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.878+0000 7f1317400700 1 -- 172.21.15.112:0/2929636264 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5cbc208d0 con 0x55b5cbd81980 2024-05-08T06:02:04.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.878+0000 7f1317e00700 1 --2- 172.21.15.112:0/2929636264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81060 0x55b5cbd81440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:04.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.878+0000 7f130fe00700 1 --2- 172.21.15.112:0/3129434575 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f12fc0b2900 0x7f12fc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.878+0000 7f130fe00700 1 -- 172.21.15.112:0/3129434575 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f13040e23a0 con 0x7f1318075ff0 2024-05-08T06:02:04.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.878+0000 7f1317400700 1 --2- 172.21.15.112:0/2929636264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbd85490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1308009ea0 tx=0x7f1308011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87ca31e8b7febd89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:04.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.878+0000 7f1317e00700 1 --2- 172.21.15.112:0/3129434575 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f12fc0b2900 0x7f12fc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.879+0000 7f130cc00700 1 -- 172.21.15.112:0/2929636264 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130801e070 con 0x55b5cbd81980 2024-05-08T06:02:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.879+0000 7f130cc00700 1 -- 172.21.15.112:0/2929636264 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f130800fb30 con 0x55b5cbd81980 2024-05-08T06:02:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.879+0000 7f130cc00700 1 -- 172.21.15.112:0/2929636264 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308019c90 con 0x55b5cbd81980 2024-05-08T06:02:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.880+0000 7f1328f1db80 1 -- 172.21.15.112:0/2929636264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 msgr2=0x55b5cbd85490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.880+0000 7f1317e00700 1 --2- 172.21.15.112:0/3129434575 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f12fc0b2900 0x7f12fc0b4dc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1310009e60 tx=0x7f1310022040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.880+0000 7f1328f1db80 1 --2- 172.21.15.112:0/2929636264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbd85490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1308009ea0 tx=0x7f1308011720 comp rx=0 tx=0).stop 2024-05-08T06:02:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.880+0000 7f1328f1db80 1 -- 172.21.15.112:0/2929636264 shutdown_connections 2024-05-08T06:02:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.880+0000 7f1328f1db80 1 --2- 172.21.15.112:0/2929636264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd859d0 0x55b5cbd85e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.880+0000 7f1328f1db80 1 --2- 172.21.15.112:0/2929636264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81060 0x55b5cbd81440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.880+0000 7f1328f1db80 1 --2- 172.21.15.112:0/2929636264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbd85490 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.880+0000 7f1328f1db80 1 -- 172.21.15.112:0/2929636264 >> 172.21.15.112:0/2929636264 conn(0x55b5cbcafb60 msgr2=0x55b5cbd7b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:04.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.880+0000 7f1328f1db80 1 -- 172.21.15.112:0/2929636264 shutdown_connections 2024-05-08T06:02:04.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.880+0000 7f1328f1db80 1 -- 172.21.15.112:0/2929636264 wait complete. 2024-05-08T06:02:04.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.881+0000 7f1328f1db80 1 -- start start 2024-05-08T06:02:04.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.881+0000 7f1328f1db80 1 Processor -- start 2024-05-08T06:02:04.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1328f1db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81060 0x55b5cbd8ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1328f1db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81980 0x55b5cbd8a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1328f1db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd859d0 0x55b5cbd8a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1328f1db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b5cbd8dbe0 con 0x55b5cbd81980 2024-05-08T06:02:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1328f1db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b5cbc39b20 con 0x55b5cbd81060 2024-05-08T06:02:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1328f1db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55b5cbd8dd60 con 0x55b5cbd859d0 2024-05-08T06:02:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1328f1db80 10 client.0 _ll_register_callbacks cb 0x55b5cbc7ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:02:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1316a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81980 0x55b5cbd8a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1317e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd859d0 0x55b5cbd8a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1317e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd859d0 0x55b5cbd8a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:53680/0 (socket says 172.21.15.112:53680) 2024-05-08T06:02:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1317e00700 1 -- 172.21.15.112:0/1851807107 learned_addr learned my addr 172.21.15.112:0/1851807107 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1317e00700 1 -- 172.21.15.112:0/1851807107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81060 msgr2=0x55b5cbd8ced0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T06:02:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1317e00700 1 --2- 172.21.15.112:0/1851807107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81060 0x55b5cbd8ced0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1317e00700 1 -- 172.21.15.112:0/1851807107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81980 msgr2=0x55b5cbd8a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1317e00700 1 --2- 172.21.15.112:0/1851807107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81980 0x55b5cbd8a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.882+0000 7f1317e00700 1 -- 172.21.15.112:0/1851807107 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5cbc99420 con 0x55b5cbd859d0 2024-05-08T06:02:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.883+0000 7f1317e00700 1 --2- 172.21.15.112:0/1851807107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd859d0 0x55b5cbd8a820 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1310015000 tx=0x7f131000b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=545a5e66206ed89b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.883+0000 7f1301600700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:02:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.883+0000 7f1301600700 1 -- 172.21.15.112:0/1851807107 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131003c060 con 0x55b5cbd859d0 2024-05-08T06:02:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.883+0000 7f1328f1db80 1 -- 172.21.15.112:0/1851807107 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f130800fb30 con 0x55b5cbd859d0 2024-05-08T06:02:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.883+0000 7f1301600700 1 -- 172.21.15.112:0/1851807107 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f13100074e0 con 0x55b5cbd859d0 2024-05-08T06:02:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.884+0000 7f1301600700 1 -- 172.21.15.112:0/1851807107 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f13100076e0 con 0x55b5cbd859d0 2024-05-08T06:02:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.884+0000 7f1328f1db80 10 client.25903 fetch_fsmap learned FSMap version 33 2024-05-08T06:02:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.884+0000 7f1328f1db80 1 -- 172.21.15.112:0/1851807107 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b5cbd985a0 con 0x55b5cbd859d0 2024-05-08T06:02:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.884+0000 7f1301600700 1 -- 172.21.15.112:0/1851807107 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f130800fb30 con 0x55b5cbd859d0 2024-05-08T06:02:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.884+0000 7f1301600700 1 -- 172.21.15.112:0/1851807107 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f1310011070 con 0x55b5cbd859d0 2024-05-08T06:02:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.884+0000 7f1328f1db80 10 client.25903 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:02:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.884+0000 7f1328f1db80 10 client.25903 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T06:02:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.884+0000 7f1328f1db80 20 client.25903 populate_metadata read hostname 'smithi112' 2024-05-08T06:02:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.884+0000 7f1328f1db80 1 --2- 172.21.15.112:0/1851807107 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55b5cbd9ab30 0x55b5cbd9cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.884+0000 7f1328f1db80 4 client.25903 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:02:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.884+0000 7f1328f1db80 1 -- 172.21.15.112:0/1851807107 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5cbd4e0a0 con 0x55b5cbd9ab30 2024-05-08T06:02:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.885+0000 7f1317400700 1 --2- 172.21.15.112:0/1851807107 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55b5cbd9ab30 0x55b5cbd9cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.885+0000 7f1317400700 1 --2- 172.21.15.112:0/1851807107 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55b5cbd9ab30 0x55b5cbd9cf60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca3d60e85e585ca5 server_cookie=ecd3cb45d641594c in_seq=0 out_seq=0 2024-05-08T06:02:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.885+0000 7f1301600700 10 client.25903 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T06:02:04.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.886+0000 7f1301600700 1 -- 172.21.15.112:0/1851807107 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b5cbd4e0a0 con 0x55b5cbd9ab30 2024-05-08T06:02:04.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.886+0000 7f1301600700 10 client.25903 handle_command_reply: tid=0 2024-05-08T06:02:04.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.886+0000 7f1328f1db80 1 client.25903 shutdown 2024-05-08T06:02:04.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.886+0000 7f1328f1db80 1 -- 172.21.15.112:0/1851807107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd859d0 msgr2=0x55b5cbd8a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:04.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.886+0000 7f1328f1db80 1 --2- 172.21.15.112:0/1851807107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd859d0 0x55b5cbd8a820 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1310015000 tx=0x7f131000b3b0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.886+0000 7f1328f1db80 1 -- 172.21.15.112:0/1851807107 shutdown_connections 2024-05-08T06:02:04.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.886+0000 7f1328f1db80 1 --2- 172.21.15.112:0/1851807107 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55b5cbd9ab30 0x55b5cbd9cf60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.886+0000 7f1328f1db80 1 --2- 172.21.15.112:0/1851807107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd859d0 0x55b5cbd8a820 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.886+0000 7f1328f1db80 1 --2- 172.21.15.112:0/1851807107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81060 0x55b5cbd8ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.886+0000 7f1328f1db80 1 --2- 172.21.15.112:0/1851807107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81980 0x55b5cbd8a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.886+0000 7f1301600700 0 client.25903 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T06:02:04.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.886+0000 7f1328f1db80 1 -- 172.21.15.112:0/1851807107 >> 172.21.15.112:0/1851807107 conn(0x55b5cbcafb60 msgr2=0x55b5cbd762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:04.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.887+0000 7f1328f1db80 1 -- 172.21.15.112:0/1851807107 shutdown_connections 2024-05-08T06:02:04.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.887+0000 7f1328f1db80 1 -- 172.21.15.112:0/1851807107 wait complete. 2024-05-08T06:02:04.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.887+0000 7f1328f1db80 20 client.25903 trim_cache size 0 max 16384 2024-05-08T06:02:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.908+0000 7f1328f1db80 1 Processor -- start 2024-05-08T06:02:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.908+0000 7f1328f1db80 1 -- start start 2024-05-08T06:02:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.909+0000 7f1328f1db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbcafb60 0x55b5cbd89c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.909+0000 7f1328f1db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81060 0x55b5cbd96c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.909+0000 7f1328f1db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbd86bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.909+0000 7f1328f1db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f13181ae2e0 con 0x55b5cbd81060 2024-05-08T06:02:04.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.909+0000 7f1328f1db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f13181abb70 con 0x55b5cbd81980 2024-05-08T06:02:04.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.909+0000 7f1328f1db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f13181a9ac0 con 0x55b5cbcafb60 2024-05-08T06:02:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.909+0000 7f1316a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81060 0x55b5cbd96c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.909+0000 7f1317400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbcafb60 0x55b5cbd89c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.909+0000 7f1317400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbcafb60 0x55b5cbd89c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:53682/0 (socket says 172.21.15.112:53682) 2024-05-08T06:02:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.909+0000 7f1317400700 1 -- 172.21.15.112:0/2564429280 learned_addr learned my addr 172.21.15.112:0/2564429280 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.910+0000 7f1317e00700 1 --2- 172.21.15.112:0/2564429280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbd86bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.910+0000 7f1317400700 1 -- 172.21.15.112:0/2564429280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 msgr2=0x55b5cbd86bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.910+0000 7f1317400700 1 --2- 172.21.15.112:0/2564429280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbd86bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.910+0000 7f1317400700 1 -- 172.21.15.112:0/2564429280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81060 msgr2=0x55b5cbd96c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.910+0000 7f1317400700 1 --2- 172.21.15.112:0/2564429280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81060 0x55b5cbd96c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.910+0000 7f1317400700 1 -- 172.21.15.112:0/2564429280 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5cbd971b0 con 0x55b5cbcafb60 2024-05-08T06:02:04.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.910+0000 7f1317e00700 1 --2- 172.21.15.112:0/2564429280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbd86bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:04.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.910+0000 7f1317400700 1 --2- 172.21.15.112:0/2564429280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbcafb60 0x55b5cbd89c00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f130801c040 tx=0x7f1308019830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0573a481c0c72e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:04.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.910+0000 7f1303e00700 1 -- 172.21.15.112:0/2564429280 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130801e040 con 0x55b5cbcafb60 2024-05-08T06:02:04.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.910+0000 7f1303e00700 1 -- 172.21.15.112:0/2564429280 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1308019550 con 0x55b5cbcafb60 2024-05-08T06:02:04.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.910+0000 7f1303e00700 1 -- 172.21.15.112:0/2564429280 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308005650 con 0x55b5cbcafb60 2024-05-08T06:02:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.911+0000 7f1328f1db80 1 -- 172.21.15.112:0/2564429280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbcafb60 msgr2=0x55b5cbd89c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.911+0000 7f1328f1db80 1 --2- 172.21.15.112:0/2564429280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbcafb60 0x55b5cbd89c00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f130801c040 tx=0x7f1308019830 comp rx=0 tx=0).stop 2024-05-08T06:02:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.911+0000 7f1328f1db80 1 -- 172.21.15.112:0/2564429280 shutdown_connections 2024-05-08T06:02:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.911+0000 7f1328f1db80 1 --2- 172.21.15.112:0/2564429280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbd86bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.911+0000 7f1328f1db80 1 --2- 172.21.15.112:0/2564429280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbcafb60 0x55b5cbd89c00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.911+0000 7f1328f1db80 1 --2- 172.21.15.112:0/2564429280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbd81060 0x55b5cbd96c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.911+0000 7f1328f1db80 1 -- 172.21.15.112:0/2564429280 >> 172.21.15.112:0/2564429280 conn(0x55b5cbd8aa60 msgr2=0x55b5cbd762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.911+0000 7f1328f1db80 1 -- 172.21.15.112:0/2564429280 shutdown_connections 2024-05-08T06:02:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.912+0000 7f1328f1db80 1 -- 172.21.15.112:0/2564429280 wait complete. 2024-05-08T06:02:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.912+0000 7f1328f1db80 1 -- start start 2024-05-08T06:02:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.912+0000 7f1328f1db80 1 Processor -- start 2024-05-08T06:02:04.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.913+0000 7f1328f1db80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbcafb60 0x55b5cbd93320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.913+0000 7f1328f1db80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd81060 0x55b5cbda1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.913+0000 7f1328f1db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbda2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.913+0000 7f1328f1db80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f13181aa030 con 0x55b5cbcafb60 2024-05-08T06:02:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.913+0000 7f1328f1db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f13181ab4a0 con 0x55b5cbd81980 2024-05-08T06:02:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.913+0000 7f1328f1db80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f13181aa700 con 0x55b5cbd81060 2024-05-08T06:02:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.913+0000 7f1328f1db80 10 client.0 _ll_register_callbacks cb 0x55b5cbc7ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:02:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.913+0000 7f1317e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbda2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.913+0000 7f1316a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd81060 0x55b5cbda1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.913+0000 7f1317e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbda2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:57290/0 (socket says 172.21.15.112:57290) 2024-05-08T06:02:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.913+0000 7f1317e00700 1 -- 172.21.15.112:0/2301114250 learned_addr learned my addr 172.21.15.112:0/2301114250 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:04.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.913+0000 7f1317400700 1 --2- 172.21.15.112:0/2301114250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbcafb60 0x55b5cbd93320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.914+0000 7f1316a00700 1 -- 172.21.15.112:0/2301114250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 msgr2=0x55b5cbda2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.914+0000 7f1316a00700 1 --2- 172.21.15.112:0/2301114250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbda2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.914+0000 7f1316a00700 1 -- 172.21.15.112:0/2301114250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbcafb60 msgr2=0x55b5cbd93320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.914+0000 7f1316a00700 1 --2- 172.21.15.112:0/2301114250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbcafb60 0x55b5cbd93320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.914+0000 7f1316a00700 1 -- 172.21.15.112:0/2301114250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5cbda27a0 con 0x55b5cbd81060 2024-05-08T06:02:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.914+0000 7f1317e00700 1 --2- 172.21.15.112:0/2301114250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbda2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.914+0000 7f1317400700 1 --2- 172.21.15.112:0/2301114250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbcafb60 0x55b5cbd93320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.914+0000 7f1316a00700 1 --2- 172.21.15.112:0/2301114250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd81060 0x55b5cbda1c20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f13040a6ba0 tx=0x7f13040a91c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92b434fa1f35a04a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.914+0000 7f1301600700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:02:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.914+0000 7f1301600700 1 -- 172.21.15.112:0/2301114250 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13040a9440 con 0x55b5cbd81060 2024-05-08T06:02:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.914+0000 7f1328f1db80 1 -- 172.21.15.112:0/2301114250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1308019550 con 0x55b5cbd81060 2024-05-08T06:02:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.914+0000 7f1301600700 1 -- 172.21.15.112:0/2301114250 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f13040cadf0 con 0x55b5cbd81060 2024-05-08T06:02:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.915+0000 7f1301600700 1 -- 172.21.15.112:0/2301114250 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f13040aa030 con 0x55b5cbd81060 2024-05-08T06:02:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.915+0000 7f1328f1db80 10 client.25909 fetch_fsmap learned FSMap version 33 2024-05-08T06:02:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.915+0000 7f1328f1db80 1 -- 172.21.15.112:0/2301114250 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b5cbe05230 con 0x55b5cbd81060 2024-05-08T06:02:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.915+0000 7f1301600700 1 -- 172.21.15.112:0/2301114250 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1308019550 con 0x55b5cbd81060 2024-05-08T06:02:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.915+0000 7f1301600700 1 -- 172.21.15.112:0/2301114250 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f13040b2300 con 0x55b5cbd81060 2024-05-08T06:02:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.915+0000 7f1328f1db80 10 client.25909 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:02:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.915+0000 7f1328f1db80 10 client.25909 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T06:02:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.915+0000 7f1328f1db80 20 client.25909 populate_metadata read hostname 'smithi112' 2024-05-08T06:02:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.915+0000 7f1328f1db80 1 --2- 172.21.15.112:0/2301114250 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55b5cbd859d0 0x55b5cbe079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.915+0000 7f1328f1db80 4 client.25909 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:02:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.915+0000 7f1328f1db80 1 -- 172.21.15.112:0/2301114250 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5cbce1400 con 0x55b5cbd859d0 2024-05-08T06:02:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.916+0000 7f1317400700 1 --2- 172.21.15.112:0/2301114250 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55b5cbd859d0 0x55b5cbe079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.916+0000 7f1317400700 1 --2- 172.21.15.112:0/2301114250 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55b5cbd859d0 0x55b5cbe079d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=46adf0f8f0986213 server_cookie=1f80357236696c90 in_seq=0 out_seq=0 2024-05-08T06:02:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.916+0000 7f1301600700 10 client.25909 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T06:02:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.916+0000 7f1301600700 1 -- 172.21.15.112:0/2301114250 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5cbce1400 con 0x55b5cbd859d0 2024-05-08T06:02:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.916+0000 7f1301600700 10 client.25909 handle_command_reply: tid=0 2024-05-08T06:02:04.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.916+0000 7f1328f1db80 1 client.25909 shutdown 2024-05-08T06:02:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.917+0000 7f1328f1db80 1 -- 172.21.15.112:0/2301114250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd81060 msgr2=0x55b5cbda1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.917+0000 7f1328f1db80 1 --2- 172.21.15.112:0/2301114250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd81060 0x55b5cbda1c20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f13040a6ba0 tx=0x7f13040a91c0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.917+0000 7f1328f1db80 1 -- 172.21.15.112:0/2301114250 shutdown_connections 2024-05-08T06:02:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.917+0000 7f1328f1db80 1 --2- 172.21.15.112:0/2301114250 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55b5cbd859d0 0x55b5cbe079d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.917+0000 7f1328f1db80 1 --2- 172.21.15.112:0/2301114250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b5cbd81980 0x55b5cbda2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.917+0000 7f1328f1db80 1 --2- 172.21.15.112:0/2301114250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5cbcafb60 0x55b5cbd93320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.917+0000 7f1328f1db80 1 --2- 172.21.15.112:0/2301114250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b5cbd81060 0x55b5cbda1c20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.917+0000 7f1301600700 0 client.25909 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T06:02:04.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.917+0000 7f1328f1db80 1 -- 172.21.15.112:0/2301114250 >> 172.21.15.112:0/2301114250 conn(0x55b5cbd8aa60 msgr2=0x55b5cbd7fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:04.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.917+0000 7f1328f1db80 1 -- 172.21.15.112:0/2301114250 shutdown_connections 2024-05-08T06:02:04.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.917+0000 7f1328f1db80 1 -- 172.21.15.112:0/2301114250 wait complete. 2024-05-08T06:02:04.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.917+0000 7f1328f1db80 20 client.25909 trim_cache size 0 max 16384 2024-05-08T06:02:04.923 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T06:02:04.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.918+0000 7f131e600700 1 -- 172.21.15.112:0/3129434575 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f12fc0b2900 msgr2=0x7f12fc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:04.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.918+0000 7f131e600700 1 --2- 172.21.15.112:0/3129434575 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f12fc0b2900 0x7f12fc0b4dc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1310009e60 tx=0x7f1310022040 comp rx=0 tx=0).stop 2024-05-08T06:02:04.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.918+0000 7f131e600700 1 -- 172.21.15.112:0/3129434575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1318075ff0 msgr2=0x7f131819c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:04.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.918+0000 7f131e600700 1 --2- 172.21.15.112:0/3129434575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1318075ff0 0x7f131819c2c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f130400e9c0 tx=0x7f130400ecd0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.918+0000 7f131e600700 1 -- 172.21.15.112:0/3129434575 shutdown_connections 2024-05-08T06:02:04.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.918+0000 7f131e600700 1 --2- 172.21.15.112:0/3129434575 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f12fc0b2900 0x7f12fc0b4dc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.918+0000 7f131e600700 1 --2- 172.21.15.112:0/3129434575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f131810fea0 0x7f13181a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.918+0000 7f131e600700 1 --2- 172.21.15.112:0/3129434575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13180756d0 0x7f131819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.918+0000 7f131e600700 1 --2- 172.21.15.112:0/3129434575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1318075ff0 0x7f131819c2c0 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f130400e9c0 tx=0x7f130400ecd0 comp rx=0 tx=0).stop 2024-05-08T06:02:04.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.918+0000 7f131e600700 1 -- 172.21.15.112:0/3129434575 >> 172.21.15.112:0/3129434575 conn(0x7f131807a600 msgr2=0x7f1318101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:04.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.918+0000 7f131e600700 1 -- 172.21.15.112:0/3129434575 shutdown_connections 2024-05-08T06:02:04.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:04.918+0000 7f131e600700 1 -- 172.21.15.112:0/3129434575 wait complete. 2024-05-08T06:02:05.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:05 smithi112 ceph-mon[30003]: pgmap v644: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1003 KiB/s wr, 372 op/s 2024-05-08T06:02:05.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:05 smithi162 ceph-mon[34800]: pgmap v644: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1003 KiB/s wr, 372 op/s 2024-05-08T06:02:05.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:05 smithi150 ceph-mon[34315]: pgmap v644: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1003 KiB/s wr, 372 op/s 2024-05-08T06:02:05.902 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-08T06:02:06.388 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:02:07.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:07 smithi112 ceph-mon[30003]: pgmap v645: 97 pgs: 97 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.3 MiB/s wr, 477 op/s 2024-05-08T06:02:07.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:07 smithi162 ceph-mon[34800]: pgmap v645: 97 pgs: 97 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.3 MiB/s wr, 477 op/s 2024-05-08T06:02:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:07 smithi150 ceph-mon[34315]: pgmap v645: 97 pgs: 97 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.3 MiB/s wr, 477 op/s 2024-05-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.670+0000 7f1dd5a00700 1 -- 172.21.15.112:0/2200361001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dd00756d0 msgr2=0x7f1dd0075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.670+0000 7f1dd5a00700 1 --2- 172.21.15.112:0/2200361001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dd00756d0 0x7f1dd0075ab0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009990 tx=0x7f1dc0009ca0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.671+0000 7f1dd5a00700 1 -- 172.21.15.112:0/2200361001 shutdown_connections 2024-05-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.671+0000 7f1dd5a00700 1 --2- 172.21.15.112:0/2200361001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dd010fed0 0x7f1dd01122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.671+0000 7f1dd5a00700 1 --2- 172.21.15.112:0/2200361001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dd00756d0 0x7f1dd0075ab0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.671+0000 7f1dd5a00700 1 --2- 172.21.15.112:0/2200361001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dd0075ff0 0x7f1dd010f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.671+0000 7f1dd5a00700 1 -- 172.21.15.112:0/2200361001 >> 172.21.15.112:0/2200361001 conn(0x7f1dd007a570 msgr2=0x7f1dd007a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.671+0000 7f1dd5a00700 1 -- 172.21.15.112:0/2200361001 shutdown_connections 2024-05-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.671+0000 7f1dd5a00700 1 -- 172.21.15.112:0/2200361001 wait complete. 2024-05-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.672+0000 7f1dd5a00700 1 Processor -- start 2024-05-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.672+0000 7f1dd5a00700 1 -- start start 2024-05-08T06:02:08.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.672+0000 7f1dd5a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dd00756d0 0x7f1dd019bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.672+0000 7f1dd5a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dd0075ff0 0x7f1dd019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.672+0000 7f1dd5a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dd010fed0 0x7f1dd01a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.672+0000 7f1dd5a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd01158f0 con 0x7f1dd0075ff0 2024-05-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.672+0000 7f1dd5a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd0115770 con 0x7f1dd010fed0 2024-05-08T06:02:08.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.672+0000 7f1dd5a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd0115a70 con 0x7f1dd00756d0 2024-05-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.673+0000 7f1dce000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dd0075ff0 0x7f1dd019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.673+0000 7f1dcea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dd00756d0 0x7f1dd019bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.673+0000 7f1dcea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dd00756d0 0x7f1dd019bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:53706/0 (socket says 172.21.15.112:53706) 2024-05-08T06:02:08.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.673+0000 7f1dcea00700 1 -- 172.21.15.112:0/3280316322 learned_addr learned my addr 172.21.15.112:0/3280316322 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.673+0000 7f1dce000700 1 --2- 172.21.15.112:0/3280316322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dd0075ff0 0x7f1dd019c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48040/0 (socket says 172.21.15.112:48040) 2024-05-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.673+0000 7f1dcf400700 1 --2- 172.21.15.112:0/3280316322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dd010fed0 0x7f1dd01a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.673+0000 7f1dcea00700 1 -- 172.21.15.112:0/3280316322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dd010fed0 msgr2=0x7f1dd01a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.673+0000 7f1dcea00700 1 --2- 172.21.15.112:0/3280316322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dd010fed0 0x7f1dd01a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.673+0000 7f1dcea00700 1 -- 172.21.15.112:0/3280316322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dd0075ff0 msgr2=0x7f1dd019c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.673+0000 7f1dcea00700 1 --2- 172.21.15.112:0/3280316322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dd0075ff0 0x7f1dd019c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.673+0000 7f1dcea00700 1 -- 172.21.15.112:0/3280316322 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd01a0d40 con 0x7f1dd00756d0 2024-05-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.673+0000 7f1dcf400700 1 --2- 172.21.15.112:0/3280316322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dd010fed0 0x7f1dd01a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.673+0000 7f1dcea00700 1 --2- 172.21.15.112:0/3280316322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dd00756d0 0x7f1dd019bd80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0000c00 tx=0x7f1dc00057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.674+0000 7f1dc7400700 1 -- 172.21.15.112:0/3280316322 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0027070 con 0x7f1dd00756d0 2024-05-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.674+0000 7f1dc7400700 1 -- 172.21.15.112:0/3280316322 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc0022470 con 0x7f1dd00756d0 2024-05-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.674+0000 7f1dc7400700 1 -- 172.21.15.112:0/3280316322 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0017650 con 0x7f1dd00756d0 2024-05-08T06:02:08.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.675+0000 7f1dd5a00700 1 -- 172.21.15.112:0/3280316322 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd01a0fd0 con 0x7f1dd00756d0 2024-05-08T06:02:08.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.675+0000 7f1dd5a00700 1 -- 172.21.15.112:0/3280316322 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd01a1430 con 0x7f1dd00756d0 2024-05-08T06:02:08.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.678+0000 7f1dc7400700 1 -- 172.21.15.112:0/3280316322 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1dc0009d10 con 0x7f1dd00756d0 2024-05-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.680+0000 7f1de028fb80 1 Processor -- start 2024-05-08T06:02:08.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.680+0000 7f1de028fb80 1 -- start start 2024-05-08T06:02:08.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.681+0000 7f1dc7400700 1 --2- 172.21.15.112:0/3280316322 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1db00b2900 0x7f1db00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.681+0000 7f1de028fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5060 0x563ae61c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.681+0000 7f1de028fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5980 0x563ae61c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.681+0000 7f1de028fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 0x563ae61c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.681+0000 7f1de028fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563ae6117a20 con 0x563ae61c5980 2024-05-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.681+0000 7f1de028fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563ae60b5760 con 0x563ae61c99d0 2024-05-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.681+0000 7f1de028fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563ae61139e0 con 0x563ae61c5060 2024-05-08T06:02:08.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.681+0000 7f1dc7400700 1 -- 172.21.15.112:0/3280316322 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f1dc00e2c00 con 0x7f1dd00756d0 2024-05-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.681+0000 7f1dcf400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 0x563ae61c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.681+0000 7f1dcea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5060 0x563ae61c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.681+0000 7f1dcea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5060 0x563ae61c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:53722/0 (socket says 172.21.15.112:53722) 2024-05-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.682+0000 7f1dcea00700 1 -- 172.21.15.112:0/2363387897 learned_addr learned my addr 172.21.15.112:0/2363387897 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.681+0000 7f1dcf400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 0x563ae61c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:57316/0 (socket says 172.21.15.112:57316) 2024-05-08T06:02:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.682+0000 7f1dce000700 1 --2- 172.21.15.112:0/2363387897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5980 0x563ae61c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.682+0000 7f1dcf400700 1 --2- 172.21.15.112:0/3280316322 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1db00b2900 0x7f1db00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.682+0000 7f1dcea00700 1 -- 172.21.15.112:0/2363387897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 msgr2=0x563ae61c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.682+0000 7f1dcea00700 1 --2- 172.21.15.112:0/2363387897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 0x563ae61c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.682+0000 7f1dcea00700 1 -- 172.21.15.112:0/2363387897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5980 msgr2=0x563ae61c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.682+0000 7f1dcea00700 1 --2- 172.21.15.112:0/2363387897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5980 0x563ae61c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.682+0000 7f1dcea00700 1 -- 172.21.15.112:0/2363387897 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae60648d0 con 0x563ae61c5060 2024-05-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.682+0000 7f1dce000700 1 --2- 172.21.15.112:0/2363387897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5980 0x563ae61c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.682+0000 7f1dcea00700 1 --2- 172.21.15.112:0/2363387897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5060 0x563ae61c5440 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00aa1b0 tx=0x7f1dc0022cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=182f5380b56980c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.682+0000 7f1dcf400700 1 --2- 172.21.15.112:0/3280316322 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1db00b2900 0x7f1db00b4dc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1dd006ac70 tx=0x7f1dc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.683+0000 7f1dbfe00700 1 -- 172.21.15.112:0/2363387897 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc00a9d30 con 0x563ae61c5060 2024-05-08T06:02:08.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.683+0000 7f1dbfe00700 1 -- 172.21.15.112:0/2363387897 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc00aa920 con 0x563ae61c5060 2024-05-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.683+0000 7f1dbfe00700 1 -- 172.21.15.112:0/2363387897 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc00aa480 con 0x563ae61c5060 2024-05-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.683+0000 7f1de028fb80 1 -- 172.21.15.112:0/2363387897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5060 msgr2=0x563ae61c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.683+0000 7f1de028fb80 1 --2- 172.21.15.112:0/2363387897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5060 0x563ae61c5440 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00aa1b0 tx=0x7f1dc0022cc0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.684+0000 7f1de028fb80 1 -- 172.21.15.112:0/2363387897 shutdown_connections 2024-05-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.684+0000 7f1de028fb80 1 --2- 172.21.15.112:0/2363387897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 0x563ae61c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.684+0000 7f1de028fb80 1 --2- 172.21.15.112:0/2363387897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5060 0x563ae61c5440 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.684+0000 7f1de028fb80 1 --2- 172.21.15.112:0/2363387897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5980 0x563ae61c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.684+0000 7f1de028fb80 1 -- 172.21.15.112:0/2363387897 >> 172.21.15.112:0/2363387897 conn(0x563ae60f3b60 msgr2=0x563ae61bf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.684+0000 7f1de028fb80 1 -- 172.21.15.112:0/2363387897 shutdown_connections 2024-05-08T06:02:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.684+0000 7f1de028fb80 1 -- 172.21.15.112:0/2363387897 wait complete. 2024-05-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.684+0000 7f1de028fb80 1 -- start start 2024-05-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.685+0000 7f1de028fb80 1 Processor -- start 2024-05-08T06:02:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.685+0000 7f1de028fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5060 0x563ae61d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.685+0000 7f1de028fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5980 0x563ae61ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.685+0000 7f1de028fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 0x563ae61ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.685+0000 7f1de028fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563ae61d19a0 con 0x563ae61c5060 2024-05-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.685+0000 7f1de028fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563ae607db20 con 0x563ae61c99d0 2024-05-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.685+0000 7f1de028fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563ae61d1b20 con 0x563ae61c5980 2024-05-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.685+0000 7f1de028fb80 10 client.0 _ll_register_callbacks cb 0x563ae60bec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.686+0000 7f1dce000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5980 0x563ae61ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.686+0000 7f1dce000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5980 0x563ae61ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:53728/0 (socket says 172.21.15.112:53728) 2024-05-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.686+0000 7f1dce000700 1 -- 172.21.15.112:0/2619381641 learned_addr learned my addr 172.21.15.112:0/2619381641 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.686+0000 7f1dcea00700 1 --2- 172.21.15.112:0/2619381641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5060 0x563ae61d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.686+0000 7f1dcf400700 1 --2- 172.21.15.112:0/2619381641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 0x563ae61ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.686+0000 7f1dce000700 1 -- 172.21.15.112:0/2619381641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 msgr2=0x563ae61ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.686+0000 7f1dce000700 1 --2- 172.21.15.112:0/2619381641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 0x563ae61ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.686+0000 7f1dce000700 1 -- 172.21.15.112:0/2619381641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5060 msgr2=0x563ae61d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.686+0000 7f1dce000700 1 --2- 172.21.15.112:0/2619381641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5060 0x563ae61d0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.686+0000 7f1dce000700 1 -- 172.21.15.112:0/2619381641 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae60dd420 con 0x563ae61c5980 2024-05-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.686+0000 7f1dcea00700 1 --2- 172.21.15.112:0/2619381641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5060 0x563ae61d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.686+0000 7f1dce000700 1 --2- 172.21.15.112:0/2619381641 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5980 0x563ae61ce250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1db8007ba0 tx=0x7f1db8007c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62348f24c872bb07 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.686+0000 7f1dbcc00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.687+0000 7f1dbcc00700 1 -- 172.21.15.112:0/2619381641 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db801a030 con 0x563ae61c5980 2024-05-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.687+0000 7f1dbcc00700 1 -- 172.21.15.112:0/2619381641 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f1db8007e00 con 0x563ae61c5980 2024-05-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.687+0000 7f1de028fb80 1 -- 172.21.15.112:0/2619381641 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dc00aa920 con 0x563ae61c5980 2024-05-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.687+0000 7f1dbcc00700 1 -- 172.21.15.112:0/2619381641 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1db80139b0 con 0x563ae61c5980 2024-05-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.687+0000 7f1de028fb80 10 client.25921 fetch_fsmap learned FSMap version 33 2024-05-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.687+0000 7f1de028fb80 1 -- 172.21.15.112:0/2619381641 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563ae61dc1b0 con 0x563ae61c5980 2024-05-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.688+0000 7f1dbcc00700 1 -- 172.21.15.112:0/2619381641 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc00aa920 con 0x563ae61c5980 2024-05-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.688+0000 7f1dbcc00700 1 -- 172.21.15.112:0/2619381641 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f1db8029420 con 0x563ae61c5980 2024-05-08T06:02:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.688+0000 7f1de028fb80 10 client.25921 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.688+0000 7f1de028fb80 10 client.25921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.688+0000 7f1de028fb80 20 client.25921 populate_metadata read hostname 'smithi112' 2024-05-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.688+0000 7f1de028fb80 1 --2- 172.21.15.112:0/2619381641 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563ae61de7f0 0x563ae61e0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.688+0000 7f1de028fb80 4 client.25921 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:02:08.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.688+0000 7f1de028fb80 1 -- 172.21.15.112:0/2619381641 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ae61920a0 con 0x563ae61de7f0 2024-05-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.688+0000 7f1dcea00700 1 --2- 172.21.15.112:0/2619381641 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563ae61de7f0 0x563ae61e0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.689+0000 7f1dbcc00700 10 client.25921 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.689+0000 7f1dcea00700 1 --2- 172.21.15.112:0/2619381641 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563ae61de7f0 0x563ae61e0c40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0117db1b863e1c8 server_cookie=34f8173bc39b24fa in_seq=0 out_seq=0 2024-05-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.690+0000 7f1dbcc00700 1 -- 172.21.15.112:0/2619381641 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563ae61920a0 con 0x563ae61de7f0 2024-05-08T06:02:08.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.690+0000 7f1dbcc00700 10 client.25921 handle_command_reply: tid=0 2024-05-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.690+0000 7f1de028fb80 1 client.25921 shutdown 2024-05-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.690+0000 7f1de028fb80 1 -- 172.21.15.112:0/2619381641 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5980 msgr2=0x563ae61ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.690+0000 7f1de028fb80 1 --2- 172.21.15.112:0/2619381641 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5980 0x563ae61ce250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1db8007ba0 tx=0x7f1db8007c80 comp rx=0 tx=0).stop 2024-05-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.690+0000 7f1de028fb80 1 -- 172.21.15.112:0/2619381641 shutdown_connections 2024-05-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.690+0000 7f1de028fb80 1 --2- 172.21.15.112:0/2619381641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 0x563ae61ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.690+0000 7f1de028fb80 1 --2- 172.21.15.112:0/2619381641 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563ae61de7f0 0x563ae61e0c40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.690+0000 7f1de028fb80 1 --2- 172.21.15.112:0/2619381641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5060 0x563ae61d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.690+0000 7f1de028fb80 1 --2- 172.21.15.112:0/2619381641 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5980 0x563ae61ce250 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.690+0000 7f1de028fb80 1 -- 172.21.15.112:0/2619381641 >> 172.21.15.112:0/2619381641 conn(0x563ae60f3b60 msgr2=0x563ae61ba2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.690+0000 7f1dbcc00700 0 client.25921 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.691+0000 7f1de028fb80 1 -- 172.21.15.112:0/2619381641 shutdown_connections 2024-05-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.691+0000 7f1de028fb80 1 -- 172.21.15.112:0/2619381641 wait complete. 2024-05-08T06:02:08.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.691+0000 7f1de028fb80 20 client.25921 trim_cache size 0 max 16384 2024-05-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.708+0000 7f1de028fb80 1 Processor -- start 2024-05-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.708+0000 7f1de028fb80 1 -- start start 2024-05-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.708+0000 7f1de028fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c5060 0x563ae61ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.708+0000 7f1de028fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5980 0x563ae61da9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.708+0000 7f1de028fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c99d0 0x563ae61ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.708+0000 7f1de028fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd01b2580 con 0x563ae61c5980 2024-05-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.708+0000 7f1de028fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd01afe10 con 0x563ae61c5060 2024-05-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.708+0000 7f1de028fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd01adc60 con 0x563ae61c99d0 2024-05-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.708+0000 7f1dcea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c5060 0x563ae61ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.708+0000 7f1dcea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c5060 0x563ae61ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:57334/0 (socket says 172.21.15.112:57334) 2024-05-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.708+0000 7f1dcea00700 1 -- 172.21.15.112:0/1877660374 learned_addr learned my addr 172.21.15.112:0/1877660374 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.709+0000 7f1dcf400700 1 --2- 172.21.15.112:0/1877660374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c99d0 0x563ae61ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.709+0000 7f1dce000700 1 --2- 172.21.15.112:0/1877660374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5980 0x563ae61da9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.709+0000 7f1dcf400700 1 -- 172.21.15.112:0/1877660374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c5060 msgr2=0x563ae61ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.709+0000 7f1dcf400700 1 --2- 172.21.15.112:0/1877660374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c5060 0x563ae61ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.709+0000 7f1dcf400700 1 -- 172.21.15.112:0/1877660374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5980 msgr2=0x563ae61da9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.709+0000 7f1dcf400700 1 --2- 172.21.15.112:0/1877660374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5980 0x563ae61da9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.709+0000 7f1dcf400700 1 -- 172.21.15.112:0/1877660374 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae61cc7c0 con 0x563ae61c99d0 2024-05-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.709+0000 7f1dcea00700 1 --2- 172.21.15.112:0/1877660374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c5060 0x563ae61ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.710+0000 7f1dcf400700 1 --2- 172.21.15.112:0/1877660374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c99d0 0x563ae61ba130 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800d620 tx=0x7f1dc8008000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72dac9d7a8895cd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.710+0000 7f1dbf400700 1 -- 172.21.15.112:0/1877660374 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc800ddc0 con 0x563ae61c99d0 2024-05-08T06:02:08.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.710+0000 7f1dbf400700 1 -- 172.21.15.112:0/1877660374 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc8004ce0 con 0x563ae61c99d0 2024-05-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.710+0000 7f1dbf400700 1 -- 172.21.15.112:0/1877660374 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc802a480 con 0x563ae61c99d0 2024-05-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.711+0000 7f1de028fb80 1 -- 172.21.15.112:0/1877660374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c99d0 msgr2=0x563ae61ba130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.711+0000 7f1de028fb80 1 --2- 172.21.15.112:0/1877660374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c99d0 0x563ae61ba130 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800d620 tx=0x7f1dc8008000 comp rx=0 tx=0).stop 2024-05-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.711+0000 7f1de028fb80 1 -- 172.21.15.112:0/1877660374 shutdown_connections 2024-05-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.711+0000 7f1de028fb80 1 --2- 172.21.15.112:0/1877660374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c99d0 0x563ae61ba130 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.711+0000 7f1de028fb80 1 --2- 172.21.15.112:0/1877660374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c5060 0x563ae61ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.711+0000 7f1de028fb80 1 --2- 172.21.15.112:0/1877660374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5980 0x563ae61da9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.711+0000 7f1de028fb80 1 -- 172.21.15.112:0/1877660374 >> 172.21.15.112:0/1877660374 conn(0x563ae60f3b60 msgr2=0x563ae61b9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.711+0000 7f1de028fb80 1 -- 172.21.15.112:0/1877660374 shutdown_connections 2024-05-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.711+0000 7f1de028fb80 1 -- 172.21.15.112:0/1877660374 wait complete. 2024-05-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.712+0000 7f1de028fb80 1 -- start start 2024-05-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.712+0000 7f1de028fb80 1 Processor -- start 2024-05-08T06:02:08.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.713+0000 7f1de028fb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5060 0x563ae61d7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.713+0000 7f1de028fb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5980 0x563ae61e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.713+0000 7f1de028fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 0x563ae61e6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.713+0000 7f1de028fb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd01ae2d0 con 0x563ae61c5060 2024-05-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.713+0000 7f1de028fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd01af740 con 0x563ae61c99d0 2024-05-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.713+0000 7f1de028fb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd01ae9a0 con 0x563ae61c5980 2024-05-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.713+0000 7f1de028fb80 10 client.0 _ll_register_callbacks cb 0x563ae60bec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.713+0000 7f1dcf400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 0x563ae61e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.713+0000 7f1dcea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5060 0x563ae61d7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.713+0000 7f1dcea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5060 0x563ae61d7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48070/0 (socket says 172.21.15.112:48070) 2024-05-08T06:02:08.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.713+0000 7f1dcea00700 1 -- 172.21.15.112:0/720447828 learned_addr learned my addr 172.21.15.112:0/720447828 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:08.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.714+0000 7f1dcea00700 1 -- 172.21.15.112:0/720447828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5980 msgr2=0x563ae61e5b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.714+0000 7f1dcea00700 1 --2- 172.21.15.112:0/720447828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5980 0x563ae61e5b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.714+0000 7f1dcea00700 1 -- 172.21.15.112:0/720447828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 msgr2=0x563ae61e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.714+0000 7f1dcea00700 1 --2- 172.21.15.112:0/720447828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 0x563ae61e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.714+0000 7f1dcea00700 1 -- 172.21.15.112:0/720447828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae61e68f0 con 0x563ae61c5060 2024-05-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.714+0000 7f1dcf400700 1 --2- 172.21.15.112:0/720447828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 0x563ae61e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.714+0000 7f1dcea00700 1 --2- 172.21.15.112:0/720447828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5060 0x563ae61d7340 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0005cb0 tx=0x7f1dc00a9b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57af8ce62dda0eac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.714+0000 7f1dbcc00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.714+0000 7f1dbcc00700 1 -- 172.21.15.112:0/720447828 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc00aa920 con 0x563ae61c5060 2024-05-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.714+0000 7f1dbcc00700 1 -- 172.21.15.112:0/720447828 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f1dc00aa480 con 0x563ae61c5060 2024-05-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.715+0000 7f1de028fb80 1 -- 172.21.15.112:0/720447828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dc8004ce0 con 0x563ae61c5060 2024-05-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.715+0000 7f1dbcc00700 1 -- 172.21.15.112:0/720447828 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1dc00aa6c0 con 0x563ae61c5060 2024-05-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.715+0000 7f1de028fb80 10 client.15717 fetch_fsmap learned FSMap version 33 2024-05-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.715+0000 7f1de028fb80 1 -- 172.21.15.112:0/720447828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563ae62490f0 con 0x563ae61c5060 2024-05-08T06:02:08.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.717+0000 7f1dbcc00700 1 -- 172.21.15.112:0/720447828 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc8004ce0 con 0x563ae61c5060 2024-05-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.717+0000 7f1dbcc00700 1 -- 172.21.15.112:0/720447828 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f1dc00ae100 con 0x563ae61c5060 2024-05-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.718+0000 7f1de028fb80 10 client.15717 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.718+0000 7f1de028fb80 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.718+0000 7f1de028fb80 20 client.15717 populate_metadata read hostname 'smithi112' 2024-05-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.718+0000 7f1de028fb80 1 --2- 172.21.15.112:0/720447828 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563ae6249660 0x563ae624ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.718+0000 7f1de028fb80 4 client.15717 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:02:08.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.718+0000 7f1de028fb80 1 -- 172.21.15.112:0/720447828 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ae6125400 con 0x563ae6249660 2024-05-08T06:02:08.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.720+0000 7f1dce000700 1 --2- 172.21.15.112:0/720447828 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563ae6249660 0x563ae624ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:08.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.722+0000 7f1dce000700 1 --2- 172.21.15.112:0/720447828 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563ae6249660 0x563ae624ba70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef61ce133ebd122c server_cookie=7a89c257ea46fb60 in_seq=0 out_seq=0 2024-05-08T06:02:08.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.722+0000 7f1dbcc00700 10 client.15717 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:08.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.723+0000 7f1dbcc00700 1 -- 172.21.15.112:0/720447828 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ae6125400 con 0x563ae6249660 2024-05-08T06:02:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.723+0000 7f1dbcc00700 10 client.15717 handle_command_reply: tid=0 2024-05-08T06:02:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.723+0000 7f1de028fb80 1 client.15717 shutdown 2024-05-08T06:02:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.724+0000 7f1de028fb80 1 -- 172.21.15.112:0/720447828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5060 msgr2=0x563ae61d7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.724+0000 7f1de028fb80 1 --2- 172.21.15.112:0/720447828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5060 0x563ae61d7340 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0005cb0 tx=0x7f1dc00a9b70 comp rx=0 tx=0).stop 2024-05-08T06:02:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.724+0000 7f1de028fb80 1 -- 172.21.15.112:0/720447828 shutdown_connections 2024-05-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.724+0000 7f1de028fb80 1 --2- 172.21.15.112:0/720447828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ae61c99d0 0x563ae61e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.724+0000 7f1de028fb80 1 --2- 172.21.15.112:0/720447828 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563ae6249660 0x563ae624ba70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.724+0000 7f1de028fb80 1 --2- 172.21.15.112:0/720447828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563ae61c5060 0x563ae61d7340 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.724+0000 7f1de028fb80 1 --2- 172.21.15.112:0/720447828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563ae61c5980 0x563ae61e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.724+0000 7f1de028fb80 1 -- 172.21.15.112:0/720447828 >> 172.21.15.112:0/720447828 conn(0x563ae60f3b60 msgr2=0x563ae61ba2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:08.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.724+0000 7f1dbcc00700 0 client.15717 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:08.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.725+0000 7f1de028fb80 1 -- 172.21.15.112:0/720447828 shutdown_connections 2024-05-08T06:02:08.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.725+0000 7f1de028fb80 1 -- 172.21.15.112:0/720447828 wait complete. 2024-05-08T06:02:08.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.725+0000 7f1de028fb80 20 client.15717 trim_cache size 0 max 16384 2024-05-08T06:02:08.726 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T06:02:08.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.726+0000 7f1dd5a00700 1 -- 172.21.15.112:0/3280316322 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1db00b2900 msgr2=0x7f1db00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:08.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.726+0000 7f1dd5a00700 1 --2- 172.21.15.112:0/3280316322 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1db00b2900 0x7f1db00b4dc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1dd006ac70 tx=0x7f1dc8009260 comp rx=0 tx=0).stop 2024-05-08T06:02:08.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.726+0000 7f1dd5a00700 1 -- 172.21.15.112:0/3280316322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dd00756d0 msgr2=0x7f1dd019bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:08.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.726+0000 7f1dd5a00700 1 --2- 172.21.15.112:0/3280316322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dd00756d0 0x7f1dd019bd80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0000c00 tx=0x7f1dc00057c0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.726+0000 7f1dd5a00700 1 -- 172.21.15.112:0/3280316322 shutdown_connections 2024-05-08T06:02:08.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.726+0000 7f1dd5a00700 1 --2- 172.21.15.112:0/3280316322 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f1db00b2900 0x7f1db00b4dc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.726+0000 7f1dd5a00700 1 --2- 172.21.15.112:0/3280316322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dd010fed0 0x7f1dd01a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.726+0000 7f1dd5a00700 1 --2- 172.21.15.112:0/3280316322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1dd00756d0 0x7f1dd019bd80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.726+0000 7f1dd5a00700 1 --2- 172.21.15.112:0/3280316322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1dd0075ff0 0x7f1dd019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:08.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.726+0000 7f1dd5a00700 1 -- 172.21.15.112:0/3280316322 >> 172.21.15.112:0/3280316322 conn(0x7f1dd007a570 msgr2=0x7f1dd01006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:08.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.726+0000 7f1dd5a00700 1 -- 172.21.15.112:0/3280316322 shutdown_connections 2024-05-08T06:02:08.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:08.726+0000 7f1dd5a00700 1 -- 172.21.15.112:0/3280316322 wait complete. 2024-05-08T06:02:09.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:09 smithi162 ceph-mon[34800]: pgmap v646: 97 pgs: 97 active+clean; 132 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.2 MiB/s wr, 403 op/s 2024-05-08T06:02:09.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:09 smithi112 ceph-mon[30003]: pgmap v646: 97 pgs: 97 active+clean; 132 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.2 MiB/s wr, 403 op/s 2024-05-08T06:02:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:09 smithi150 ceph-mon[34315]: pgmap v646: 97 pgs: 97 active+clean; 132 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.2 MiB/s wr, 403 op/s 2024-05-08T06:02:10.853 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T06:02:10.853 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T06:02:11.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:11 smithi112 ceph-mon[30003]: pgmap v647: 97 pgs: 97 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.3 MiB/s wr, 442 op/s 2024-05-08T06:02:11.340 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:02:11.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:11 smithi162 ceph-mon[34800]: pgmap v647: 97 pgs: 97 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.3 MiB/s wr, 442 op/s 2024-05-08T06:02:11.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:11 smithi150 ceph-mon[34315]: pgmap v647: 97 pgs: 97 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.3 MiB/s wr, 442 op/s 2024-05-08T06:02:13.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:13 smithi112 ceph-mon[30003]: pgmap v648: 97 pgs: 97 active+clean; 120 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.3 MiB/s wr, 431 op/s 2024-05-08T06:02:13.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:13 smithi162 ceph-mon[34800]: pgmap v648: 97 pgs: 97 active+clean; 120 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.3 MiB/s wr, 431 op/s 2024-05-08T06:02:13.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:13 smithi150 ceph-mon[34315]: pgmap v648: 97 pgs: 97 active+clean; 120 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.3 MiB/s wr, 431 op/s 2024-05-08T06:02:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.687+0000 7feeef800700 1 -- 172.21.15.112:0/3976760918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee810b6a0 msgr2=0x7feee810ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.687+0000 7feeef800700 1 --2- 172.21.15.112:0/3976760918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee810b6a0 0x7feee810ba80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7feedc00b0a0 tx=0x7feedc00b3b0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.687+0000 7feeef800700 1 -- 172.21.15.112:0/3976760918 shutdown_connections 2024-05-08T06:02:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.688+0000 7feeef800700 1 --2- 172.21.15.112:0/3976760918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee8106060 0x7feee8112570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.688+0000 7feeef800700 1 --2- 172.21.15.112:0/3976760918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee810b6a0 0x7feee810ba80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.688+0000 7feeef800700 1 --2- 172.21.15.112:0/3976760918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feee81056a0 0x7feee8105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.688+0000 7feeef800700 1 -- 172.21.15.112:0/3976760918 >> 172.21.15.112:0/3976760918 conn(0x7feee81011b0 msgr2=0x7feee81035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.688+0000 7feeef800700 1 -- 172.21.15.112:0/3976760918 shutdown_connections 2024-05-08T06:02:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.688+0000 7feeef800700 1 -- 172.21.15.112:0/3976760918 wait complete. 2024-05-08T06:02:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.689+0000 7feeef800700 1 Processor -- start 2024-05-08T06:02:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.689+0000 7feeef800700 1 -- start start 2024-05-08T06:02:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.689+0000 7feeef800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feee81056a0 0x7feee819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.689+0000 7feeef800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee8106060 0x7feee819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.690+0000 7feeef800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee810b6a0 0x7feee81a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.690+0000 7feeef800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feee8115880 con 0x7feee8106060 2024-05-08T06:02:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.690+0000 7feeef800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feee8115700 con 0x7feee810b6a0 2024-05-08T06:02:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.690+0000 7feeef800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7feee8115a00 con 0x7feee81056a0 2024-05-08T06:02:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.690+0000 7feeece00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feee81056a0 0x7feee819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.690+0000 7feeece00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feee81056a0 0x7feee819bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:53764/0 (socket says 172.21.15.112:53764) 2024-05-08T06:02:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.690+0000 7feeece00700 1 -- 172.21.15.112:0/1011873134 learned_addr learned my addr 172.21.15.112:0/1011873134 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.690+0000 7feeed800700 1 --2- 172.21.15.112:0/1011873134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee810b6a0 0x7feee81a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.690+0000 7feee7e00700 1 --2- 172.21.15.112:0/1011873134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee8106060 0x7feee819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.690+0000 7feeece00700 1 -- 172.21.15.112:0/1011873134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee810b6a0 msgr2=0x7feee81a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:13.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.690+0000 7feeece00700 1 --2- 172.21.15.112:0/1011873134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee810b6a0 0x7feee81a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.690+0000 7feeece00700 1 -- 172.21.15.112:0/1011873134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee8106060 msgr2=0x7feee819c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:13.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.690+0000 7feeece00700 1 --2- 172.21.15.112:0/1011873134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee8106060 0x7feee819c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.690+0000 7feeece00700 1 -- 172.21.15.112:0/1011873134 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee81a0d40 con 0x7feee81056a0 2024-05-08T06:02:13.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.690+0000 7feee7e00700 1 --2- 172.21.15.112:0/1011873134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee8106060 0x7feee819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:13.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.691+0000 7feeece00700 1 --2- 172.21.15.112:0/1011873134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feee81056a0 0x7feee819bd30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7feedc00b070 tx=0x7feedc004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:13.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.691+0000 7feee5600700 1 -- 172.21.15.112:0/1011873134 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc021070 con 0x7feee81056a0 2024-05-08T06:02:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.691+0000 7feeef800700 1 -- 172.21.15.112:0/1011873134 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee81a0fd0 con 0x7feee81056a0 2024-05-08T06:02:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.691+0000 7feee5600700 1 -- 172.21.15.112:0/1011873134 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feedc003c00 con 0x7feee81056a0 2024-05-08T06:02:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.691+0000 7feee5600700 1 -- 172.21.15.112:0/1011873134 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc004850 con 0x7feee81056a0 2024-05-08T06:02:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.691+0000 7feeef800700 1 -- 172.21.15.112:0/1011873134 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee81a88b0 con 0x7feee81056a0 2024-05-08T06:02:13.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.695+0000 7feee5600700 1 -- 172.21.15.112:0/1011873134 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7feedc004000 con 0x7feee81056a0 2024-05-08T06:02:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.697+0000 7feefa285b80 1 Processor -- start 2024-05-08T06:02:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.697+0000 7feefa285b80 1 -- start start 2024-05-08T06:02:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.697+0000 7feefa285b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7d0e0 0x563504d7d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.697+0000 7feefa285b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d7da00 0x563504d81510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.698+0000 7feefa285b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d81a50 0x563504d81eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.698+0000 7feefa285b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563504ccfaa0 con 0x563504d7d0e0 2024-05-08T06:02:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.698+0000 7feefa285b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563504c6d7e0 con 0x563504d81a50 2024-05-08T06:02:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.698+0000 7feefa285b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563504ccba60 con 0x563504d7da00 2024-05-08T06:02:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.698+0000 7feee5600700 1 --2- 172.21.15.112:0/1011873134 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feed00b2880 0x7feed00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.698+0000 7feee7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d81a50 0x563504d81eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.698+0000 7feee7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d81a50 0x563504d81eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:57374/0 (socket says 172.21.15.112:57374) 2024-05-08T06:02:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.698+0000 7feee7e00700 1 -- 172.21.15.112:0/1260365449 learned_addr learned my addr 172.21.15.112:0/1260365449 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.698+0000 7feeece00700 1 --2- 172.21.15.112:0/1260365449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d7da00 0x563504d81510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.698+0000 7feee7e00700 1 -- 172.21.15.112:0/1260365449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d7da00 msgr2=0x563504d81510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.698+0000 7feee7e00700 1 --2- 172.21.15.112:0/1260365449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d7da00 0x563504d81510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.698+0000 7feee7e00700 1 -- 172.21.15.112:0/1260365449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7d0e0 msgr2=0x563504d7d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.698+0000 7feee7e00700 1 --2- 172.21.15.112:0/1260365449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7d0e0 0x563504d7d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.698+0000 7feee7e00700 1 -- 172.21.15.112:0/1260365449 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563504c1c950 con 0x563504d81a50 2024-05-08T06:02:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.699+0000 7feee5600700 1 -- 172.21.15.112:0/1011873134 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7feedc00f020 con 0x7feee81056a0 2024-05-08T06:02:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.699+0000 7feee7e00700 1 --2- 172.21.15.112:0/1260365449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d81a50 0x563504d81eb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feed4009d80 tx=0x7feed40116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a310ffd955d8b42e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.699+0000 7feeda000700 1 -- 172.21.15.112:0/1260365449 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4011e20 con 0x563504d81a50 2024-05-08T06:02:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.699+0000 7feeda000700 1 -- 172.21.15.112:0/1260365449 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed4012900 con 0x563504d81a50 2024-05-08T06:02:13.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.699+0000 7feeda000700 1 -- 172.21.15.112:0/1260365449 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed400f870 con 0x563504d81a50 2024-05-08T06:02:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.699+0000 7feeed800700 1 --2- 172.21.15.112:0/1011873134 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feed00b2880 0x7feed00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.700+0000 7feefa285b80 1 -- 172.21.15.112:0/1260365449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d81a50 msgr2=0x563504d81eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.700+0000 7feefa285b80 1 --2- 172.21.15.112:0/1260365449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d81a50 0x563504d81eb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feed4009d80 tx=0x7feed40116c0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.700+0000 7feefa285b80 1 -- 172.21.15.112:0/1260365449 shutdown_connections 2024-05-08T06:02:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.700+0000 7feefa285b80 1 --2- 172.21.15.112:0/1260365449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d81a50 0x563504d81eb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.700+0000 7feefa285b80 1 --2- 172.21.15.112:0/1260365449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7d0e0 0x563504d7d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.700+0000 7feefa285b80 1 --2- 172.21.15.112:0/1260365449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d7da00 0x563504d81510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.700+0000 7feefa285b80 1 -- 172.21.15.112:0/1260365449 >> 172.21.15.112:0/1260365449 conn(0x563504cabbe0 msgr2=0x563504d77820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.700+0000 7feefa285b80 1 -- 172.21.15.112:0/1260365449 shutdown_connections 2024-05-08T06:02:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.700+0000 7feefa285b80 1 -- 172.21.15.112:0/1260365449 wait complete. 2024-05-08T06:02:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.701+0000 7feeed800700 1 --2- 172.21.15.112:0/1011873134 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feed00b2880 0x7feed00b4d40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feee0009c90 tx=0x7feee0009400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:13.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.701+0000 7feefa285b80 1 -- start start 2024-05-08T06:02:13.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.701+0000 7feefa285b80 1 Processor -- start 2024-05-08T06:02:13.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.702+0000 7feefa285b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d7d0e0 0x563504d88e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.702+0000 7feefa285b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7da00 0x563504d86200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.702+0000 7feefa285b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d81a50 0x563504d86740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.702+0000 7feefa285b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563504d89c60 con 0x563504d7da00 2024-05-08T06:02:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.702+0000 7feefa285b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563504c35ba0 con 0x563504d81a50 2024-05-08T06:02:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.702+0000 7feefa285b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563504d89de0 con 0x563504d7d0e0 2024-05-08T06:02:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.702+0000 7feefa285b80 10 client.0 _ll_register_callbacks cb 0x563504c76cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:02:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.702+0000 7feeece00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d7d0e0 0x563504d88e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.702+0000 7feee7e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7da00 0x563504d86200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.702+0000 7feeed800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d81a50 0x563504d86740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.702+0000 7feeece00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d7d0e0 0x563504d88e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:53770/0 (socket says 172.21.15.112:53770) 2024-05-08T06:02:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.702+0000 7feeece00700 1 -- 172.21.15.112:0/3428797406 learned_addr learned my addr 172.21.15.112:0/3428797406 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.703+0000 7feeece00700 1 -- 172.21.15.112:0/3428797406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d81a50 msgr2=0x563504d86740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.703+0000 7feeece00700 1 --2- 172.21.15.112:0/3428797406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d81a50 0x563504d86740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.703+0000 7feeece00700 1 -- 172.21.15.112:0/3428797406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7da00 msgr2=0x563504d86200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.703+0000 7feeece00700 1 --2- 172.21.15.112:0/3428797406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7da00 0x563504d86200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.703+0000 7feeece00700 1 -- 172.21.15.112:0/3428797406 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563504c954a0 con 0x563504d7d0e0 2024-05-08T06:02:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.703+0000 7feeece00700 1 --2- 172.21.15.112:0/3428797406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d7d0e0 0x563504d88e60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feedc004fd0 tx=0x7feedc0aa760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aeb8e7e52024955 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.703+0000 7feecea00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:02:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.703+0000 7feecea00700 1 -- 172.21.15.112:0/3428797406 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc0aa8a0 con 0x563504d7d0e0 2024-05-08T06:02:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.704+0000 7feecea00700 1 -- 172.21.15.112:0/3428797406 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7feedc0e12f0 con 0x563504d7d0e0 2024-05-08T06:02:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.704+0000 7feefa285b80 1 -- 172.21.15.112:0/3428797406 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed4012900 con 0x563504d7d0e0 2024-05-08T06:02:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.704+0000 7feecea00700 1 -- 172.21.15.112:0/3428797406 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7feedc009cb0 con 0x563504d7d0e0 2024-05-08T06:02:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.704+0000 7feefa285b80 10 client.25936 fetch_fsmap learned FSMap version 33 2024-05-08T06:02:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.704+0000 7feefa285b80 1 -- 172.21.15.112:0/3428797406 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563504d89cc0 con 0x563504d7d0e0 2024-05-08T06:02:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.704+0000 7feecea00700 1 -- 172.21.15.112:0/3428797406 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed4012900 con 0x563504d7d0e0 2024-05-08T06:02:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.704+0000 7feecea00700 1 -- 172.21.15.112:0/3428797406 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7feedc011070 con 0x563504d7d0e0 2024-05-08T06:02:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.705+0000 7feefa285b80 10 client.25936 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:02:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.705+0000 7feefa285b80 10 client.25936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:02:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.705+0000 7feefa285b80 20 client.25936 populate_metadata read hostname 'smithi112' 2024-05-08T06:02:13.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.705+0000 7feefa285b80 1 --2- 172.21.15.112:0/3428797406 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563504da7a00 0x563504da9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.705+0000 7feefa285b80 4 client.25936 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:02:13.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.705+0000 7feefa285b80 1 -- 172.21.15.112:0/3428797406 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563504d4a120 con 0x563504da7a00 2024-05-08T06:02:13.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.705+0000 7feee7e00700 1 --2- 172.21.15.112:0/3428797406 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563504da7a00 0x563504da9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.706+0000 7feee7e00700 1 --2- 172.21.15.112:0/3428797406 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563504da7a00 0x563504da9df0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d1b2a99b0a8328f server_cookie=28a334e6c33feb1c in_seq=0 out_seq=0 2024-05-08T06:02:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.706+0000 7feecea00700 10 client.25936 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.707+0000 7feecea00700 1 -- 172.21.15.112:0/3428797406 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563504d4a120 con 0x563504da7a00 2024-05-08T06:02:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.707+0000 7feecea00700 10 client.25936 handle_command_reply: tid=0 2024-05-08T06:02:13.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.707+0000 7feefa285b80 1 client.25936 shutdown 2024-05-08T06:02:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.707+0000 7feefa285b80 1 -- 172.21.15.112:0/3428797406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d7d0e0 msgr2=0x563504d88e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.708+0000 7feefa285b80 1 --2- 172.21.15.112:0/3428797406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d7d0e0 0x563504d88e60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feedc004fd0 tx=0x7feedc0aa760 comp rx=0 tx=0).stop 2024-05-08T06:02:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.708+0000 7feefa285b80 1 -- 172.21.15.112:0/3428797406 shutdown_connections 2024-05-08T06:02:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.708+0000 7feefa285b80 1 --2- 172.21.15.112:0/3428797406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d81a50 0x563504d86740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.708+0000 7feefa285b80 1 --2- 172.21.15.112:0/3428797406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d7d0e0 0x563504d88e60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.708+0000 7feefa285b80 1 --2- 172.21.15.112:0/3428797406 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563504da7a00 0x563504da9df0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.708+0000 7feefa285b80 1 --2- 172.21.15.112:0/3428797406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7da00 0x563504d86200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.708+0000 7feefa285b80 1 -- 172.21.15.112:0/3428797406 >> 172.21.15.112:0/3428797406 conn(0x563504cabbe0 msgr2=0x563504d72330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.708+0000 7feecea00700 0 client.25936 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.708+0000 7feefa285b80 1 -- 172.21.15.112:0/3428797406 shutdown_connections 2024-05-08T06:02:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.708+0000 7feefa285b80 1 -- 172.21.15.112:0/3428797406 wait complete. 2024-05-08T06:02:13.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.709+0000 7feefa285b80 20 client.25936 trim_cache size 0 max 16384 2024-05-08T06:02:13.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.736+0000 7feefa285b80 1 Processor -- start 2024-05-08T06:02:13.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.736+0000 7feefa285b80 1 -- start start 2024-05-08T06:02:13.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.736+0000 7feefa285b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7d0e0 0x563504d7c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.736+0000 7feefa285b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d7da00 0x563504daf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.736+0000 7feefa285b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d81a50 0x563504dafc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.736+0000 7feefa285b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feee80726e0 con 0x563504d7d0e0 2024-05-08T06:02:13.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.736+0000 7feefa285b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feee806ff70 con 0x563504d7da00 2024-05-08T06:02:13.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.736+0000 7feefa285b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7feee807b9e0 con 0x563504d81a50 2024-05-08T06:02:13.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.737+0000 7feeece00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7d0e0 0x563504d7c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.737+0000 7feee7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d7da00 0x563504daf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.737+0000 7feeece00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7d0e0 0x563504d7c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48134/0 (socket says 172.21.15.112:48134) 2024-05-08T06:02:13.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.737+0000 7feeece00700 1 -- 172.21.15.112:0/2413019251 learned_addr learned my addr 172.21.15.112:0/2413019251 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:13.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.737+0000 7feeed800700 1 --2- 172.21.15.112:0/2413019251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d81a50 0x563504dafc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.737+0000 7feeece00700 1 -- 172.21.15.112:0/2413019251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d81a50 msgr2=0x563504dafc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:13.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.737+0000 7feeece00700 1 --2- 172.21.15.112:0/2413019251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d81a50 0x563504dafc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.737+0000 7feeece00700 1 -- 172.21.15.112:0/2413019251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d7da00 msgr2=0x563504daf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:13.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.737+0000 7feeece00700 1 --2- 172.21.15.112:0/2413019251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d7da00 0x563504daf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.737+0000 7feeece00700 1 -- 172.21.15.112:0/2413019251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563504d74470 con 0x563504d7d0e0 2024-05-08T06:02:13.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.737+0000 7feee7e00700 1 --2- 172.21.15.112:0/2413019251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d7da00 0x563504daf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:02:13.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.738+0000 7feeece00700 1 --2- 172.21.15.112:0/2413019251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7d0e0 0x563504d7c780 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feedc01d020 tx=0x7feedc0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03e716cc61a006c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:13.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.738+0000 7feed9600700 1 -- 172.21.15.112:0/2413019251 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc0ab490 con 0x563504d7d0e0 2024-05-08T06:02:13.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.738+0000 7feed9600700 1 -- 172.21.15.112:0/2413019251 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feedc0b3b60 con 0x563504d7d0e0 2024-05-08T06:02:13.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.738+0000 7feed9600700 1 -- 172.21.15.112:0/2413019251 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc0b3930 con 0x563504d7d0e0 2024-05-08T06:02:13.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.739+0000 7feefa285b80 1 -- 172.21.15.112:0/2413019251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7d0e0 msgr2=0x563504d7c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:13.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.739+0000 7feefa285b80 1 --2- 172.21.15.112:0/2413019251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7d0e0 0x563504d7c780 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feedc01d020 tx=0x7feedc0099e0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.739+0000 7feefa285b80 1 -- 172.21.15.112:0/2413019251 shutdown_connections 2024-05-08T06:02:13.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.739+0000 7feefa285b80 1 --2- 172.21.15.112:0/2413019251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d81a50 0x563504dafc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.739+0000 7feefa285b80 1 --2- 172.21.15.112:0/2413019251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7d0e0 0x563504d7c780 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.739+0000 7feefa285b80 1 --2- 172.21.15.112:0/2413019251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d7da00 0x563504daf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.739+0000 7feefa285b80 1 -- 172.21.15.112:0/2413019251 >> 172.21.15.112:0/2413019251 conn(0x563504cabbe0 msgr2=0x563504d72360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:13.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.739+0000 7feefa285b80 1 -- 172.21.15.112:0/2413019251 shutdown_connections 2024-05-08T06:02:13.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.739+0000 7feefa285b80 1 -- 172.21.15.112:0/2413019251 wait complete. 2024-05-08T06:02:13.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.740+0000 7feefa285b80 1 -- start start 2024-05-08T06:02:13.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.740+0000 7feefa285b80 1 Processor -- start 2024-05-08T06:02:13.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.741+0000 7feefa285b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d7d0e0 0x563504d8f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.741+0000 7feefa285b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7da00 0x563504d8f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.741+0000 7feefa285b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d81a50 0x563504d8b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.741+0000 7feefa285b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feee806e430 con 0x563504d7da00 2024-05-08T06:02:13.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.741+0000 7feefa285b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feee806f8a0 con 0x563504d7d0e0 2024-05-08T06:02:13.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.741+0000 7feefa285b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7feee806eb00 con 0x563504d81a50 2024-05-08T06:02:13.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.741+0000 7feefa285b80 10 client.0 _ll_register_callbacks cb 0x563504c76cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:02:13.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.741+0000 7feeece00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d7d0e0 0x563504d8f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.741+0000 7feeece00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d7d0e0 0x563504d8f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:57400/0 (socket says 172.21.15.112:57400) 2024-05-08T06:02:13.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.741+0000 7feeece00700 1 -- 172.21.15.112:0/1543848392 learned_addr learned my addr 172.21.15.112:0/1543848392 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:13.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.741+0000 7feee7e00700 1 --2- 172.21.15.112:0/1543848392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7da00 0x563504d8f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.741+0000 7feeed800700 1 --2- 172.21.15.112:0/1543848392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d81a50 0x563504d8b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.742+0000 7feeece00700 1 -- 172.21.15.112:0/1543848392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d81a50 msgr2=0x563504d8b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:13.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.742+0000 7feeece00700 1 --2- 172.21.15.112:0/1543848392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d81a50 0x563504d8b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.742+0000 7feeece00700 1 -- 172.21.15.112:0/1543848392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7da00 msgr2=0x563504d8f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:13.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.742+0000 7feeece00700 1 --2- 172.21.15.112:0/1543848392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7da00 0x563504d8f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.742+0000 7feeece00700 1 -- 172.21.15.112:0/1543848392 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563504d8c010 con 0x563504d7d0e0 2024-05-08T06:02:13.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.742+0000 7feeed800700 1 --2- 172.21.15.112:0/1543848392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d81a50 0x563504d8b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:13.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.742+0000 7feee7e00700 1 --2- 172.21.15.112:0/1543848392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7da00 0x563504d8f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:02:13.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.742+0000 7feeece00700 1 --2- 172.21.15.112:0/1543848392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d7d0e0 0x563504d8f240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feedc0aad20 tx=0x7feedc0aa7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f71baa7f964256f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:13.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.742+0000 7feecea00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:02:13.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.742+0000 7feecea00700 1 -- 172.21.15.112:0/1543848392 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc0af140 con 0x563504d7d0e0 2024-05-08T06:02:13.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.742+0000 7feecea00700 1 -- 172.21.15.112:0/1543848392 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7feedc0b38e0 con 0x563504d7d0e0 2024-05-08T06:02:13.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.742+0000 7feefa285b80 1 -- 172.21.15.112:0/1543848392 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feedc0b3b60 con 0x563504d7d0e0 2024-05-08T06:02:13.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.743+0000 7feecea00700 1 -- 172.21.15.112:0/1543848392 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7feedc0ab490 con 0x563504d7d0e0 2024-05-08T06:02:13.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.743+0000 7feefa285b80 10 client.25901 fetch_fsmap learned FSMap version 33 2024-05-08T06:02:13.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.743+0000 7feefa285b80 1 -- 172.21.15.112:0/1543848392 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563504d8c010 con 0x563504d7d0e0 2024-05-08T06:02:13.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.743+0000 7feecea00700 1 -- 172.21.15.112:0/1543848392 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feedc0b3b60 con 0x563504d7d0e0 2024-05-08T06:02:13.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.743+0000 7feecea00700 1 -- 172.21.15.112:0/1543848392 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7feedc011040 con 0x563504d7d0e0 2024-05-08T06:02:13.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.743+0000 7feefa285b80 10 client.25901 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:02:13.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.743+0000 7feefa285b80 10 client.25901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:02:13.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.743+0000 7feefa285b80 20 client.25901 populate_metadata read hostname 'smithi112' 2024-05-08T06:02:13.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.743+0000 7feefa285b80 1 --2- 172.21.15.112:0/1543848392 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563504e13230 0x563504e15620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:13.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.743+0000 7feefa285b80 4 client.25901 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T06:02:13.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.743+0000 7feefa285b80 1 -- 172.21.15.112:0/1543848392 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563504cdd480 con 0x563504e13230 2024-05-08T06:02:13.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.745+0000 7feee7e00700 1 --2- 172.21.15.112:0/1543848392 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563504e13230 0x563504e15620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:13.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.746+0000 7feee7e00700 1 --2- 172.21.15.112:0/1543848392 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563504e13230 0x563504e15620 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9f1de3c64d418 server_cookie=da149a1894d284c4 in_seq=0 out_seq=0 2024-05-08T06:02:13.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.746+0000 7feecea00700 10 client.25901 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:13.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.748+0000 7feecea00700 1 -- 172.21.15.112:0/1543848392 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563504cdd480 con 0x563504e13230 2024-05-08T06:02:13.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.748+0000 7feecea00700 10 client.25901 handle_command_reply: tid=0 2024-05-08T06:02:13.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.748+0000 7feefa285b80 1 client.25901 shutdown 2024-05-08T06:02:13.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.749+0000 7feefa285b80 1 -- 172.21.15.112:0/1543848392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d7d0e0 msgr2=0x563504d8f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:13.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.749+0000 7feefa285b80 1 --2- 172.21.15.112:0/1543848392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d7d0e0 0x563504d8f240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feedc0aad20 tx=0x7feedc0aa7e0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.750+0000 7feefa285b80 1 -- 172.21.15.112:0/1543848392 shutdown_connections 2024-05-08T06:02:13.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.750+0000 7feefa285b80 1 --2- 172.21.15.112:0/1543848392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563504d81a50 0x563504d8b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.750+0000 7feefa285b80 1 --2- 172.21.15.112:0/1543848392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563504d7d0e0 0x563504d8f240 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.750+0000 7feefa285b80 1 --2- 172.21.15.112:0/1543848392 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563504e13230 0x563504e15620 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.750+0000 7feefa285b80 1 --2- 172.21.15.112:0/1543848392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563504d7da00 0x563504d8f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.750+0000 7feefa285b80 1 -- 172.21.15.112:0/1543848392 >> 172.21.15.112:0/1543848392 conn(0x563504cabbe0 msgr2=0x563504d7c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:13.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.750+0000 7feecea00700 0 client.25901 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:13.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.750+0000 7feefa285b80 1 -- 172.21.15.112:0/1543848392 shutdown_connections 2024-05-08T06:02:13.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.750+0000 7feefa285b80 1 -- 172.21.15.112:0/1543848392 wait complete. 2024-05-08T06:02:13.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.750+0000 7feefa285b80 20 client.25901 trim_cache size 0 max 16384 2024-05-08T06:02:13.752 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:02:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-08T06:02:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "d1c5a122-e074-4ab4-9079-6264fa5adf82", 2024-05-08T06:02:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-08T06:02:13.752 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:02:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.751+0000 7feeef800700 1 -- 172.21.15.112:0/1011873134 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feed00b2880 msgr2=0x7feed00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.751+0000 7feeef800700 1 --2- 172.21.15.112:0/1011873134 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feed00b2880 0x7feed00b4d40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feee0009c90 tx=0x7feee0009400 comp rx=0 tx=0).stop 2024-05-08T06:02:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.751+0000 7feeef800700 1 -- 172.21.15.112:0/1011873134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feee81056a0 msgr2=0x7feee819bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.751+0000 7feeef800700 1 --2- 172.21.15.112:0/1011873134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feee81056a0 0x7feee819bd30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7feedc00b070 tx=0x7feedc004db0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.752+0000 7feeef800700 1 -- 172.21.15.112:0/1011873134 shutdown_connections 2024-05-08T06:02:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.752+0000 7feeef800700 1 --2- 172.21.15.112:0/1011873134 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feed00b2880 0x7feed00b4d40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.752+0000 7feeef800700 1 --2- 172.21.15.112:0/1011873134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feee810b6a0 0x7feee81a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.752+0000 7feeef800700 1 --2- 172.21.15.112:0/1011873134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feee81056a0 0x7feee819bd30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.752+0000 7feeef800700 1 --2- 172.21.15.112:0/1011873134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee8106060 0x7feee819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.752+0000 7feeef800700 1 -- 172.21.15.112:0/1011873134 >> 172.21.15.112:0/1011873134 conn(0x7feee81011b0 msgr2=0x7feee81019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.752+0000 7feeef800700 1 -- 172.21.15.112:0/1011873134 shutdown_connections 2024-05-08T06:02:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:13.752+0000 7feeef800700 1 -- 172.21.15.112:0/1011873134 wait complete. 2024-05-08T06:02:14.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:14 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T06:02:14.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:14 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T06:02:14.644 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T06:02:14.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:14 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T06:02:15.147 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:02:15.555 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:15 smithi112 ceph-mon[30003]: scrub queued for path: / 2024-05-08T06:02:15.555 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:15 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:02:15.555 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:15 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T06:02:15.555 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:15 smithi112 ceph-mon[30003]: pgmap v649: 97 pgs: 97 active+clean; 120 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.1 MiB/s wr, 360 op/s 2024-05-08T06:02:15.555 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:15 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:02:15.555 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:15 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T06:02:15.555 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:15 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:02:15.555 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:15 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:02:15.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:15 smithi162 ceph-mon[34800]: scrub queued for path: / 2024-05-08T06:02:15.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:15 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:02:15.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:15 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T06:02:15.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:15 smithi162 ceph-mon[34800]: pgmap v649: 97 pgs: 97 active+clean; 120 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.1 MiB/s wr, 360 op/s 2024-05-08T06:02:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:15 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:02:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:15 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T06:02:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:15 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:02:15.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:15 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:02:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:15 smithi150 ceph-mon[34315]: scrub queued for path: / 2024-05-08T06:02:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:15 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:02:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:15 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T06:02:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:15 smithi150 ceph-mon[34315]: pgmap v649: 97 pgs: 97 active+clean; 120 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.1 MiB/s wr, 360 op/s 2024-05-08T06:02:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:15 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:02:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:15 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T06:02:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:15 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:02:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:15 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:02:17.476 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:17 smithi112 ceph-mon[30003]: pgmap v650: 97 pgs: 97 active+clean; 119 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.3 MiB/s wr, 476 op/s 2024-05-08T06:02:17.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:17 smithi162 ceph-mon[34800]: pgmap v650: 97 pgs: 97 active+clean; 119 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.3 MiB/s wr, 476 op/s 2024-05-08T06:02:17.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:17 smithi150 ceph-mon[34315]: pgmap v650: 97 pgs: 97 active+clean; 119 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.3 MiB/s wr, 476 op/s 2024-05-08T06:02:17.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.976+0000 7fa538200700 1 -- 172.21.15.112:0/2566654467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa530114670 msgr2=0x7fa530116a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:17.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.976+0000 7fa538200700 1 --2- 172.21.15.112:0/2566654467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa530114670 0x7fa530116a60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa52800b0a0 tx=0x7fa52800b3b0 comp rx=0 tx=0).stop 2024-05-08T06:02:17.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.976+0000 7fa538200700 1 -- 172.21.15.112:0/2566654467 shutdown_connections 2024-05-08T06:02:17.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.976+0000 7fa538200700 1 --2- 172.21.15.112:0/2566654467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa530114670 0x7fa530116a60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:17.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.976+0000 7fa538200700 1 --2- 172.21.15.112:0/2566654467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5300756d0 0x7fa530075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:17.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.976+0000 7fa538200700 1 --2- 172.21.15.112:0/2566654467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa530075ff0 0x7fa530114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:17.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.976+0000 7fa538200700 1 -- 172.21.15.112:0/2566654467 >> 172.21.15.112:0/2566654467 conn(0x7fa53007a600 msgr2=0x7fa53007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:17.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.976+0000 7fa538200700 1 -- 172.21.15.112:0/2566654467 shutdown_connections 2024-05-08T06:02:17.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.976+0000 7fa538200700 1 -- 172.21.15.112:0/2566654467 wait complete. 2024-05-08T06:02:17.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.977+0000 7fa538200700 1 Processor -- start 2024-05-08T06:02:17.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.978+0000 7fa538200700 1 -- start start 2024-05-08T06:02:17.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.978+0000 7fa538200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5300756d0 0x7fa5301a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:17.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.978+0000 7fa538200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa530075ff0 0x7fa5301a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:17.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.978+0000 7fa538200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa530114670 0x7fa5301a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:17.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.978+0000 7fa538200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa530119c80 con 0x7fa5300756d0 2024-05-08T06:02:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.978+0000 7fa538200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa530119b00 con 0x7fa530075ff0 2024-05-08T06:02:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.978+0000 7fa538200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa530119e00 con 0x7fa530114670 2024-05-08T06:02:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.979+0000 7fa535800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5300756d0 0x7fa5301a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.979+0000 7fa534e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa530075ff0 0x7fa5301a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.979+0000 7fa535800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5300756d0 0x7fa5301a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44672/0 (socket says 172.21.15.112:44672) 2024-05-08T06:02:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.979+0000 7fa535800700 1 -- 172.21.15.112:0/502597755 learned_addr learned my addr 172.21.15.112:0/502597755 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:17.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.979+0000 7fa536200700 1 --2- 172.21.15.112:0/502597755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa530114670 0x7fa5301a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:17.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.979+0000 7fa534e00700 1 -- 172.21.15.112:0/502597755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa530114670 msgr2=0x7fa5301a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:17.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.979+0000 7fa534e00700 1 --2- 172.21.15.112:0/502597755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa530114670 0x7fa5301a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:17.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.979+0000 7fa534e00700 1 -- 172.21.15.112:0/502597755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5300756d0 msgr2=0x7fa5301a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:17.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.979+0000 7fa534e00700 1 --2- 172.21.15.112:0/502597755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5300756d0 0x7fa5301a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:17.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.979+0000 7fa534e00700 1 -- 172.21.15.112:0/502597755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5301a5180 con 0x7fa530075ff0 2024-05-08T06:02:17.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.979+0000 7fa536200700 1 --2- 172.21.15.112:0/502597755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa530114670 0x7fa5301a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:17.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.980+0000 7fa534e00700 1 --2- 172.21.15.112:0/502597755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa530075ff0 0x7fa5301a06b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00e9f0 tx=0x7fa51c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:17.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.980+0000 7fa52e000700 1 -- 172.21.15.112:0/502597755 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00cbd0 con 0x7fa530075ff0 2024-05-08T06:02:17.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.980+0000 7fa52e000700 1 -- 172.21.15.112:0/502597755 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa51c00cd70 con 0x7fa530075ff0 2024-05-08T06:02:17.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.980+0000 7fa52e000700 1 -- 172.21.15.112:0/502597755 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c018680 con 0x7fa530075ff0 2024-05-08T06:02:17.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.980+0000 7fa538200700 1 -- 172.21.15.112:0/502597755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5301a5470 con 0x7fa530075ff0 2024-05-08T06:02:17.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.980+0000 7fa538200700 1 -- 172.21.15.112:0/502597755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5301accf0 con 0x7fa530075ff0 2024-05-08T06:02:17.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.984+0000 7fa52e000700 1 -- 172.21.15.112:0/502597755 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa51c010430 con 0x7fa530075ff0 2024-05-08T06:02:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.986+0000 7fa52e000700 1 --2- 172.21.15.112:0/502597755 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa5180b2900 0x7fa5180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.986+0000 7fa52e000700 1 -- 172.21.15.112:0/502597755 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fa51c0e22e0 con 0x7fa530075ff0 2024-05-08T06:02:17.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.986+0000 7fa535800700 1 --2- 172.21.15.112:0/502597755 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa5180b2900 0x7fa5180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:17.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.987+0000 7fa535800700 1 --2- 172.21.15.112:0/502597755 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa5180b2900 0x7fa5180b4dc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa524009680 tx=0x7fa524009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.987+0000 7fa542c83b80 1 Processor -- start 2024-05-08T06:02:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.987+0000 7fa542c83b80 1 -- start start 2024-05-08T06:02:17.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.988+0000 7fa542c83b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9060 0x5622f21c9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.988+0000 7fa542c83b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9980 0x5622f21cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.988+0000 7fa542c83b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21cd9d0 0x5622f21cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.988+0000 7fa542c83b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5622f211ba20 con 0x5622f21cd9d0 2024-05-08T06:02:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.988+0000 7fa542c83b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5622f20b9760 con 0x5622f21c9980 2024-05-08T06:02:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.988+0000 7fa542c83b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5622f21179e0 con 0x5622f21c9060 2024-05-08T06:02:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.988+0000 7fa535800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9060 0x5622f21c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.988+0000 7fa535800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9060 0x5622f21c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:46108/0 (socket says 172.21.15.112:46108) 2024-05-08T06:02:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.988+0000 7fa535800700 1 -- 172.21.15.112:0/4017539968 learned_addr learned my addr 172.21.15.112:0/4017539968 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.988+0000 7fa534e00700 1 --2- 172.21.15.112:0/4017539968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9980 0x5622f21cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.989+0000 7fa535800700 1 -- 172.21.15.112:0/4017539968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9980 msgr2=0x5622f21cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.989+0000 7fa535800700 1 --2- 172.21.15.112:0/4017539968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9980 0x5622f21cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.989+0000 7fa535800700 1 -- 172.21.15.112:0/4017539968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21cd9d0 msgr2=0x5622f21cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.989+0000 7fa535800700 1 --2- 172.21.15.112:0/4017539968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21cd9d0 0x5622f21cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.989+0000 7fa535800700 1 -- 172.21.15.112:0/4017539968 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f20688d0 con 0x5622f21c9060 2024-05-08T06:02:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.989+0000 7fa535800700 1 --2- 172.21.15.112:0/4017539968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9060 0x5622f21c9440 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa52401c570 tx=0x7fa524014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b727768fa3167d0a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.989+0000 7fa522a00700 1 -- 172.21.15.112:0/4017539968 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524027430 con 0x5622f21c9060 2024-05-08T06:02:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.989+0000 7fa522a00700 1 -- 172.21.15.112:0/4017539968 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa524027a80 con 0x5622f21c9060 2024-05-08T06:02:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.989+0000 7fa522a00700 1 -- 172.21.15.112:0/4017539968 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52401dab0 con 0x5622f21c9060 2024-05-08T06:02:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.990+0000 7fa542c83b80 1 -- 172.21.15.112:0/4017539968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9060 msgr2=0x5622f21c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.990+0000 7fa542c83b80 1 --2- 172.21.15.112:0/4017539968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9060 0x5622f21c9440 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa52401c570 tx=0x7fa524014000 comp rx=0 tx=0).stop 2024-05-08T06:02:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.990+0000 7fa542c83b80 1 -- 172.21.15.112:0/4017539968 shutdown_connections 2024-05-08T06:02:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.990+0000 7fa542c83b80 1 --2- 172.21.15.112:0/4017539968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21cd9d0 0x5622f21cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.990+0000 7fa542c83b80 1 --2- 172.21.15.112:0/4017539968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9060 0x5622f21c9440 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.990+0000 7fa542c83b80 1 --2- 172.21.15.112:0/4017539968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9980 0x5622f21cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.990+0000 7fa542c83b80 1 -- 172.21.15.112:0/4017539968 >> 172.21.15.112:0/4017539968 conn(0x5622f20f7b60 msgr2=0x5622f21c37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.991+0000 7fa542c83b80 1 -- 172.21.15.112:0/4017539968 shutdown_connections 2024-05-08T06:02:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.991+0000 7fa542c83b80 1 -- 172.21.15.112:0/4017539968 wait complete. 2024-05-08T06:02:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.991+0000 7fa542c83b80 1 -- start start 2024-05-08T06:02:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.992+0000 7fa542c83b80 1 Processor -- start 2024-05-08T06:02:17.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.992+0000 7fa542c83b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9060 0x5622f21d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:17.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.992+0000 7fa542c83b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 0x5622f21d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:17.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.992+0000 7fa542c83b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21cd9d0 0x5622f21d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:17.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.992+0000 7fa542c83b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5622f21d5be0 con 0x5622f21cd9d0 2024-05-08T06:02:17.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.992+0000 7fa542c83b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5622f2081b20 con 0x5622f21c9060 2024-05-08T06:02:17.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.992+0000 7fa542c83b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5622f21d5d60 con 0x5622f21c9980 2024-05-08T06:02:17.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.993+0000 7fa542c83b80 10 client.0 _ll_register_callbacks cb 0x5622f20c2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:02:17.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.993+0000 7fa534e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 0x5622f21d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:17.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.993+0000 7fa534e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 0x5622f21d21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:46116/0 (socket says 172.21.15.112:46116) 2024-05-08T06:02:17.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.993+0000 7fa534e00700 1 -- 172.21.15.112:0/1327109781 learned_addr learned my addr 172.21.15.112:0/1327109781 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:17.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.993+0000 7fa535800700 1 --2- 172.21.15.112:0/1327109781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9060 0x5622f21d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:17.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.993+0000 7fa534e00700 1 -- 172.21.15.112:0/1327109781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9060 msgr2=0x5622f21d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:17.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.993+0000 7fa534e00700 1 --2- 172.21.15.112:0/1327109781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9060 0x5622f21d4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:17.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.993+0000 7fa534e00700 1 -- 172.21.15.112:0/1327109781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21cd9d0 msgr2=0x5622f21d2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:17.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.993+0000 7fa534e00700 1 --2- 172.21.15.112:0/1327109781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21cd9d0 0x5622f21d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:17.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.993+0000 7fa534e00700 1 -- 172.21.15.112:0/1327109781 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f20e1420 con 0x5622f21c9980 2024-05-08T06:02:17.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.993+0000 7fa535800700 1 --2- 172.21.15.112:0/1327109781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9060 0x5622f21d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:17.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.993+0000 7fa534e00700 1 --2- 172.21.15.112:0/1327109781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 0x5622f21d21d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0e0120 tx=0x7fa51c0a9e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7569c98e92b9ef5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:17.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.993+0000 7fa517400700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:02:17.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.994+0000 7fa517400700 1 -- 172.21.15.112:0/1327109781 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0a9040 con 0x5622f21c9980 2024-05-08T06:02:17.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.994+0000 7fa517400700 1 -- 172.21.15.112:0/1327109781 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fa51c010930 con 0x5622f21c9980 2024-05-08T06:02:17.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.994+0000 7fa542c83b80 1 -- 172.21.15.112:0/1327109781 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa524027a80 con 0x5622f21c9980 2024-05-08T06:02:17.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.994+0000 7fa517400700 1 -- 172.21.15.112:0/1327109781 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa51c0a5060 con 0x5622f21c9980 2024-05-08T06:02:17.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.994+0000 7fa542c83b80 10 client.25954 fetch_fsmap learned FSMap version 33 2024-05-08T06:02:17.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.994+0000 7fa542c83b80 1 -- 172.21.15.112:0/1327109781 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5622f21e04a0 con 0x5622f21c9980 2024-05-08T06:02:17.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.995+0000 7fa517400700 1 -- 172.21.15.112:0/1327109781 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa524027a80 con 0x5622f21c9980 2024-05-08T06:02:17.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.995+0000 7fa517400700 1 -- 172.21.15.112:0/1327109781 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7fa51c0cab80 con 0x5622f21c9980 2024-05-08T06:02:17.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.995+0000 7fa542c83b80 10 client.25954 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:02:17.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.995+0000 7fa542c83b80 10 client.25954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:02:17.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.995+0000 7fa542c83b80 20 client.25954 populate_metadata read hostname 'smithi112' 2024-05-08T06:02:17.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.995+0000 7fa542c83b80 1 --2- 172.21.15.112:0/1327109781 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622f21e2ae0 0x5622f21e4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:17.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.995+0000 7fa542c83b80 4 client.25954 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:02:17.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.995+0000 7fa542c83b80 1 -- 172.21.15.112:0/1327109781 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622f21960a0 con 0x5622f21e2ae0 2024-05-08T06:02:17.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.996+0000 7fa535800700 1 --2- 172.21.15.112:0/1327109781 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622f21e2ae0 0x5622f21e4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:17.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.998+0000 7fa535800700 1 --2- 172.21.15.112:0/1327109781 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622f21e2ae0 0x5622f21e4ef0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f387b56ed77f71 server_cookie=12e15889c5728ae9 in_seq=0 out_seq=0 2024-05-08T06:02:17.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:17.998+0000 7fa517400700 10 client.25954 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:18.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.001+0000 7fa517400700 1 -- 172.21.15.112:0/1327109781 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5622f21960a0 con 0x5622f21e2ae0 2024-05-08T06:02:18.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.001+0000 7fa517400700 10 client.25954 handle_command_reply: tid=0 2024-05-08T06:02:18.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.002+0000 7fa542c83b80 1 client.25954 shutdown 2024-05-08T06:02:18.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.002+0000 7fa542c83b80 1 -- 172.21.15.112:0/1327109781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 msgr2=0x5622f21d21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:18.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.002+0000 7fa542c83b80 1 --2- 172.21.15.112:0/1327109781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 0x5622f21d21d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0e0120 tx=0x7fa51c0a9e10 comp rx=0 tx=0).stop 2024-05-08T06:02:18.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.002+0000 7fa542c83b80 1 -- 172.21.15.112:0/1327109781 shutdown_connections 2024-05-08T06:02:18.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.002+0000 7fa542c83b80 1 --2- 172.21.15.112:0/1327109781 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622f21e2ae0 0x5622f21e4ef0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.002+0000 7fa542c83b80 1 --2- 172.21.15.112:0/1327109781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21cd9d0 0x5622f21d2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.002+0000 7fa542c83b80 1 --2- 172.21.15.112:0/1327109781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9060 0x5622f21d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.002+0000 7fa542c83b80 1 --2- 172.21.15.112:0/1327109781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 0x5622f21d21d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.002+0000 7fa542c83b80 1 -- 172.21.15.112:0/1327109781 >> 172.21.15.112:0/1327109781 conn(0x5622f20f7b60 msgr2=0x5622f21be2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:18.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.003+0000 7fa517400700 0 client.25954 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:18.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.003+0000 7fa542c83b80 1 -- 172.21.15.112:0/1327109781 shutdown_connections 2024-05-08T06:02:18.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.003+0000 7fa542c83b80 1 -- 172.21.15.112:0/1327109781 wait complete. 2024-05-08T06:02:18.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.004+0000 7fa542c83b80 20 client.25954 trim_cache size 0 max 16384 2024-05-08T06:02:18.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.029+0000 7fa542c83b80 1 Processor -- start 2024-05-08T06:02:18.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.029+0000 7fa542c83b80 1 -- start start 2024-05-08T06:02:18.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.030+0000 7fa542c83b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21c9060 0x5622f21be130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:18.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.030+0000 7fa542c83b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 0x5622f21cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:18.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.030+0000 7fa542c83b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21cd9d0 0x5622f21ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:18.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.030+0000 7fa542c83b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301b6c80 con 0x5622f21c9060 2024-05-08T06:02:18.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.030+0000 7fa542c83b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301b4510 con 0x5622f21cd9d0 2024-05-08T06:02:18.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.030+0000 7fa542c83b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301b2360 con 0x5622f21c9980 2024-05-08T06:02:18.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.030+0000 7fa536200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21cd9d0 0x5622f21ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:18.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.030+0000 7fa536200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21cd9d0 0x5622f21ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:59950/0 (socket says 172.21.15.112:59950) 2024-05-08T06:02:18.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.030+0000 7fa536200700 1 -- 172.21.15.112:0/320993716 learned_addr learned my addr 172.21.15.112:0/320993716 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:18.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.030+0000 7fa534e00700 1 --2- 172.21.15.112:0/320993716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 0x5622f21cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:18.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.030+0000 7fa535800700 1 --2- 172.21.15.112:0/320993716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21c9060 0x5622f21be130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:18.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.031+0000 7fa536200700 1 -- 172.21.15.112:0/320993716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 msgr2=0x5622f21cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:18.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.031+0000 7fa536200700 1 --2- 172.21.15.112:0/320993716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 0x5622f21cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.031+0000 7fa536200700 1 -- 172.21.15.112:0/320993716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21c9060 msgr2=0x5622f21be130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:18.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.031+0000 7fa536200700 1 --2- 172.21.15.112:0/320993716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21c9060 0x5622f21be130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.031+0000 7fa536200700 1 -- 172.21.15.112:0/320993716 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f21c4610 con 0x5622f21cd9d0 2024-05-08T06:02:18.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.031+0000 7fa535800700 1 --2- 172.21.15.112:0/320993716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21c9060 0x5622f21be130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:18.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.031+0000 7fa536200700 1 --2- 172.21.15.112:0/320993716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21cd9d0 0x5622f21ccd90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa52800b0a0 tx=0x7fa528009470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f10039eac7cd112 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:18.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.031+0000 7fa522000700 1 -- 172.21.15.112:0/320993716 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52800bab0 con 0x5622f21cd9d0 2024-05-08T06:02:18.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.031+0000 7fa522000700 1 -- 172.21.15.112:0/320993716 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa52800bc50 con 0x5622f21cd9d0 2024-05-08T06:02:18.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.032+0000 7fa522000700 1 -- 172.21.15.112:0/320993716 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528007cf0 con 0x5622f21cd9d0 2024-05-08T06:02:18.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.033+0000 7fa542c83b80 1 -- 172.21.15.112:0/320993716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21cd9d0 msgr2=0x5622f21ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:18.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.033+0000 7fa542c83b80 1 --2- 172.21.15.112:0/320993716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21cd9d0 0x5622f21ccd90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa52800b0a0 tx=0x7fa528009470 comp rx=0 tx=0).stop 2024-05-08T06:02:18.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.033+0000 7fa542c83b80 1 -- 172.21.15.112:0/320993716 shutdown_connections 2024-05-08T06:02:18.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.033+0000 7fa542c83b80 1 --2- 172.21.15.112:0/320993716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21cd9d0 0x5622f21ccd90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.033+0000 7fa542c83b80 1 --2- 172.21.15.112:0/320993716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21c9060 0x5622f21be130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.033+0000 7fa542c83b80 1 --2- 172.21.15.112:0/320993716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 0x5622f21cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.033+0000 7fa542c83b80 1 -- 172.21.15.112:0/320993716 >> 172.21.15.112:0/320993716 conn(0x5622f20f7b60 msgr2=0x5622f21bda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:18.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.033+0000 7fa542c83b80 1 -- 172.21.15.112:0/320993716 shutdown_connections 2024-05-08T06:02:18.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.033+0000 7fa542c83b80 1 -- 172.21.15.112:0/320993716 wait complete. 2024-05-08T06:02:18.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.033+0000 7fa542c83b80 1 -- start start 2024-05-08T06:02:18.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.034+0000 7fa542c83b80 1 Processor -- start 2024-05-08T06:02:18.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.034+0000 7fa542c83b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9060 0x5622f21db2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:18.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.034+0000 7fa542c83b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 0x5622f21d7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.034+0000 7fa542c83b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21cd9d0 0x5622f21d7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.034+0000 7fa542c83b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301b29d0 con 0x5622f21cd9d0 2024-05-08T06:02:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.034+0000 7fa542c83b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301b3e40 con 0x5622f21c9060 2024-05-08T06:02:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.034+0000 7fa542c83b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301b30a0 con 0x5622f21c9980 2024-05-08T06:02:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.034+0000 7fa542c83b80 10 client.0 _ll_register_callbacks cb 0x5622f20c2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:02:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.035+0000 7fa535800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9060 0x5622f21db2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.035+0000 7fa535800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9060 0x5622f21db2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:59966/0 (socket says 172.21.15.112:59966) 2024-05-08T06:02:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.035+0000 7fa535800700 1 -- 172.21.15.112:0/2959000127 learned_addr learned my addr 172.21.15.112:0/2959000127 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.035+0000 7fa534e00700 1 --2- 172.21.15.112:0/2959000127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 0x5622f21d7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.035+0000 7fa536200700 1 --2- 172.21.15.112:0/2959000127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21cd9d0 0x5622f21d7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.035+0000 7fa534e00700 1 -- 172.21.15.112:0/2959000127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9060 msgr2=0x5622f21db2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.035+0000 7fa534e00700 1 --2- 172.21.15.112:0/2959000127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9060 0x5622f21db2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.035+0000 7fa534e00700 1 -- 172.21.15.112:0/2959000127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21cd9d0 msgr2=0x5622f21d7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:18.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.035+0000 7fa534e00700 1 --2- 172.21.15.112:0/2959000127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21cd9d0 0x5622f21d7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.035+0000 7fa534e00700 1 -- 172.21.15.112:0/2959000127 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f21d8170 con 0x5622f21c9980 2024-05-08T06:02:18.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.035+0000 7fa536200700 1 --2- 172.21.15.112:0/2959000127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21cd9d0 0x5622f21d7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:02:18.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.036+0000 7fa534e00700 1 --2- 172.21.15.112:0/2959000127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 0x5622f21d7670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0a9f30 tx=0x7fa51c0a56d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=116858fd6db4cbf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:18.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.036+0000 7fa517400700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:02:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.036+0000 7fa517400700 1 -- 172.21.15.112:0/2959000127 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0a9040 con 0x5622f21c9980 2024-05-08T06:02:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.036+0000 7fa542c83b80 1 -- 172.21.15.112:0/2959000127 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa52800bc50 con 0x5622f21c9980 2024-05-08T06:02:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.036+0000 7fa517400700 1 -- 172.21.15.112:0/2959000127 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fa51c0a5ec0 con 0x5622f21c9980 2024-05-08T06:02:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.036+0000 7fa517400700 1 -- 172.21.15.112:0/2959000127 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa51c0a93e0 con 0x5622f21c9980 2024-05-08T06:02:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.036+0000 7fa542c83b80 10 client.25960 fetch_fsmap learned FSMap version 33 2024-05-08T06:02:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.036+0000 7fa542c83b80 1 -- 172.21.15.112:0/2959000127 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5622f224e140 con 0x5622f21c9980 2024-05-08T06:02:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.036+0000 7fa517400700 1 -- 172.21.15.112:0/2959000127 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa52800bc50 con 0x5622f21c9980 2024-05-08T06:02:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.037+0000 7fa517400700 1 -- 172.21.15.112:0/2959000127 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7fa51c0aee10 con 0x5622f21c9980 2024-05-08T06:02:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.037+0000 7fa542c83b80 10 client.25960 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:02:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.037+0000 7fa542c83b80 10 client.25960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:02:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.037+0000 7fa542c83b80 20 client.25960 populate_metadata read hostname 'smithi112' 2024-05-08T06:02:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.037+0000 7fa542c83b80 1 --2- 172.21.15.112:0/2959000127 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622f224e7e0 0x5622f2250c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.037+0000 7fa542c83b80 4 client.25960 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T06:02:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.037+0000 7fa542c83b80 1 -- 172.21.15.112:0/2959000127 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5622f2129400 con 0x5622f224e7e0 2024-05-08T06:02:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.038+0000 7fa535800700 1 --2- 172.21.15.112:0/2959000127 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622f224e7e0 0x5622f2250c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:18.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.038+0000 7fa535800700 1 --2- 172.21.15.112:0/2959000127 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622f224e7e0 0x5622f2250c00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeb2930de0e16c6f server_cookie=31c6b578924dd4b0 in_seq=0 out_seq=0 2024-05-08T06:02:18.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.038+0000 7fa517400700 10 client.25960 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:18.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.039+0000 7fa517400700 1 -- 172.21.15.112:0/2959000127 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5622f2129400 con 0x5622f224e7e0 2024-05-08T06:02:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.039+0000 7fa517400700 10 client.25960 handle_command_reply: tid=0 2024-05-08T06:02:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.039+0000 7fa542c83b80 1 client.25960 shutdown 2024-05-08T06:02:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.039+0000 7fa542c83b80 1 -- 172.21.15.112:0/2959000127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 msgr2=0x5622f21d7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.039+0000 7fa542c83b80 1 --2- 172.21.15.112:0/2959000127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 0x5622f21d7670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0a9f30 tx=0x7fa51c0a56d0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.039+0000 7fa542c83b80 1 -- 172.21.15.112:0/2959000127 shutdown_connections 2024-05-08T06:02:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.039+0000 7fa542c83b80 1 --2- 172.21.15.112:0/2959000127 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5622f224e7e0 0x5622f2250c00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.039+0000 7fa542c83b80 1 --2- 172.21.15.112:0/2959000127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f21cd9d0 0x5622f21d7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.039+0000 7fa542c83b80 1 --2- 172.21.15.112:0/2959000127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5622f21c9060 0x5622f21db2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.039+0000 7fa517400700 0 client.25960 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.039+0000 7fa542c83b80 1 --2- 172.21.15.112:0/2959000127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f21c9980 0x5622f21d7670 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.039+0000 7fa542c83b80 1 -- 172.21.15.112:0/2959000127 >> 172.21.15.112:0/2959000127 conn(0x5622f20f7b60 msgr2=0x5622f21be2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.040+0000 7fa542c83b80 1 -- 172.21.15.112:0/2959000127 shutdown_connections 2024-05-08T06:02:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.040+0000 7fa542c83b80 1 -- 172.21.15.112:0/2959000127 wait complete. 2024-05-08T06:02:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.040+0000 7fa542c83b80 20 client.25960 trim_cache size 0 max 16384 2024-05-08T06:02:18.042 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:02:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T06:02:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-05-08T06:02:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: "d1c5a122-e074-4ab4-9079-6264fa5adf82": { 2024-05-08T06:02:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-05-08T06:02:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "d1c5a122-e074-4ab4-9079-6264fa5adf82", 2024-05-08T06:02:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-05-08T06:02:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:02:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:02:18.042 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:02:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.041+0000 7fa538200700 1 -- 172.21.15.112:0/502597755 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa5180b2900 msgr2=0x7fa5180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.041+0000 7fa538200700 1 --2- 172.21.15.112:0/502597755 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa5180b2900 0x7fa5180b4dc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa524009680 tx=0x7fa524009260 comp rx=0 tx=0).stop 2024-05-08T06:02:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.041+0000 7fa538200700 1 -- 172.21.15.112:0/502597755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa530075ff0 msgr2=0x7fa5301a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.041+0000 7fa538200700 1 --2- 172.21.15.112:0/502597755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa530075ff0 0x7fa5301a06b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00e9f0 tx=0x7fa51c00ed00 comp rx=0 tx=0).stop 2024-05-08T06:02:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.041+0000 7fa538200700 1 -- 172.21.15.112:0/502597755 shutdown_connections 2024-05-08T06:02:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.041+0000 7fa538200700 1 --2- 172.21.15.112:0/502597755 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa5180b2900 0x7fa5180b4dc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.041+0000 7fa538200700 1 --2- 172.21.15.112:0/502597755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa530114670 0x7fa5301a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.041+0000 7fa538200700 1 --2- 172.21.15.112:0/502597755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5300756d0 0x7fa5301a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.041+0000 7fa538200700 1 --2- 172.21.15.112:0/502597755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa530075ff0 0x7fa5301a06b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.041+0000 7fa538200700 1 -- 172.21.15.112:0/502597755 >> 172.21.15.112:0/502597755 conn(0x7fa53007a600 msgr2=0x7fa530113440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:18.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.041+0000 7fa538200700 1 -- 172.21.15.112:0/502597755 shutdown_connections 2024-05-08T06:02:18.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:18.041+0000 7fa538200700 1 -- 172.21.15.112:0/502597755 wait complete. 2024-05-08T06:02:18.919 INFO:tasks.cephfs.filesystem:scrub status for tag:d1c5a122-e074-4ab4-9079-6264fa5adf82 - {'path': '/', 'tag': 'd1c5a122-e074-4ab4-9079-6264fa5adf82', 'options': 'recursive,force'} 2024-05-08T06:02:19.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:19 smithi112 ceph-mon[30003]: pgmap v651: 97 pgs: 97 active+clean; 121 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.0 MiB/s wr, 404 op/s 2024-05-08T06:02:19.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:19 smithi162 ceph-mon[34800]: pgmap v651: 97 pgs: 97 active+clean; 121 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.0 MiB/s wr, 404 op/s 2024-05-08T06:02:19.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:19 smithi150 ceph-mon[34315]: pgmap v651: 97 pgs: 97 active+clean; 121 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.0 MiB/s wr, 404 op/s 2024-05-08T06:02:21.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:21 smithi150 ceph-mon[34315]: pgmap v652: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.1 MiB/s wr, 450 op/s 2024-05-08T06:02:21.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:21 smithi112 ceph-mon[30003]: pgmap v652: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.1 MiB/s wr, 450 op/s 2024-05-08T06:02:21.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:21 smithi162 ceph-mon[34800]: pgmap v652: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.1 MiB/s wr, 450 op/s 2024-05-08T06:02:23.618 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:23 smithi162 ceph-mon[34800]: pgmap v653: 97 pgs: 97 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.1 MiB/s wr, 440 op/s 2024-05-08T06:02:23.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:23 smithi150 ceph-mon[34315]: pgmap v653: 97 pgs: 97 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.1 MiB/s wr, 440 op/s 2024-05-08T06:02:23.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:23 smithi112 ceph-mon[30003]: pgmap v653: 97 pgs: 97 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.1 MiB/s wr, 440 op/s 2024-05-08T06:02:25.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:25 smithi150 ceph-mon[34315]: pgmap v654: 97 pgs: 97 active+clean; 130 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 940 KiB/s wr, 374 op/s 2024-05-08T06:02:25.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:25 smithi112 ceph-mon[30003]: pgmap v654: 97 pgs: 97 active+clean; 130 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 940 KiB/s wr, 374 op/s 2024-05-08T06:02:25.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:25 smithi162 ceph-mon[34800]: pgmap v654: 97 pgs: 97 active+clean; 130 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 940 KiB/s wr, 374 op/s 2024-05-08T06:02:26.377 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x63ef9ee 2024-05-08T06:02:27.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:27 smithi150 ceph-mon[34315]: pgmap v655: 97 pgs: 97 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.1 MiB/s wr, 456 op/s 2024-05-08T06:02:27.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:27 smithi112 ceph-mon[30003]: pgmap v655: 97 pgs: 97 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.1 MiB/s wr, 456 op/s 2024-05-08T06:02:27.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:27 smithi162 ceph-mon[34800]: pgmap v655: 97 pgs: 97 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.1 MiB/s wr, 456 op/s 2024-05-08T06:02:29.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:29 smithi150 ceph-mon[34315]: pgmap v656: 97 pgs: 97 active+clean; 120 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 904 KiB/s wr, 364 op/s 2024-05-08T06:02:29.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:29 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T06:02:29.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:29 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T06:02:29.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:29 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T06:02:29.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:29 smithi112 ceph-mon[30003]: pgmap v656: 97 pgs: 97 active+clean; 120 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 904 KiB/s wr, 364 op/s 2024-05-08T06:02:29.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:29 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T06:02:29.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:29 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T06:02:29.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:29 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T06:02:29.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:29 smithi162 ceph-mon[34800]: pgmap v656: 97 pgs: 97 active+clean; 120 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 904 KiB/s wr, 364 op/s 2024-05-08T06:02:29.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:29 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T06:02:29.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:29 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T06:02:29.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:29 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T06:02:31.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:31 smithi150 ceph-mon[34315]: pgmap v657: 97 pgs: 97 active+clean; 130 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 949 KiB/s wr, 369 op/s 2024-05-08T06:02:31.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:31 smithi112 ceph-mon[30003]: pgmap v657: 97 pgs: 97 active+clean; 130 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 949 KiB/s wr, 369 op/s 2024-05-08T06:02:31.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:31 smithi162 ceph-mon[34800]: pgmap v657: 97 pgs: 97 active+clean; 130 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 949 KiB/s wr, 369 op/s 2024-05-08T06:02:33.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:33 smithi150 ceph-mon[34315]: pgmap v658: 97 pgs: 97 active+clean; 130 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 957 KiB/s wr, 362 op/s 2024-05-08T06:02:33.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:33 smithi112 ceph-mon[30003]: pgmap v658: 97 pgs: 97 active+clean; 130 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 957 KiB/s wr, 362 op/s 2024-05-08T06:02:33.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:33 smithi162 ceph-mon[34800]: pgmap v658: 97 pgs: 97 active+clean; 130 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 957 KiB/s wr, 362 op/s 2024-05-08T06:02:35.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:35 smithi150 ceph-mon[34315]: pgmap v659: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 811 KiB/s wr, 297 op/s 2024-05-08T06:02:35.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:35 smithi112 ceph-mon[30003]: pgmap v659: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 811 KiB/s wr, 297 op/s 2024-05-08T06:02:35.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:35 smithi162 ceph-mon[34800]: pgmap v659: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 811 KiB/s wr, 297 op/s 2024-05-08T06:02:36.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:36 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:36.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:36 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:36.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:36 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:37.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:37 smithi150 ceph-mon[34315]: pgmap v660: 97 pgs: 97 active+clean; 125 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1008 KiB/s wr, 386 op/s 2024-05-08T06:02:37.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:37 smithi112 ceph-mon[30003]: pgmap v660: 97 pgs: 97 active+clean; 125 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1008 KiB/s wr, 386 op/s 2024-05-08T06:02:37.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:37 smithi162 ceph-mon[34800]: pgmap v660: 97 pgs: 97 active+clean; 125 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1008 KiB/s wr, 386 op/s 2024-05-08T06:02:38.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:38 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:38.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:38 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:38.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:38 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:02:38.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:38 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:38.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:38 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:38.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:38 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:02:38.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:38 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:38.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:38 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:38.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:38 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:02:39.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:39 smithi150 ceph-mon[34315]: pgmap v661: 97 pgs: 97 active+clean; 129 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 975 KiB/s wr, 333 op/s 2024-05-08T06:02:39.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:39 smithi112 ceph-mon[30003]: pgmap v661: 97 pgs: 97 active+clean; 129 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 975 KiB/s wr, 333 op/s 2024-05-08T06:02:39.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:39 smithi162 ceph-mon[34800]: pgmap v661: 97 pgs: 97 active+clean; 129 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 975 KiB/s wr, 333 op/s 2024-05-08T06:02:41.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:41 smithi150 ceph-mon[34315]: pgmap v662: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1008 KiB/s wr, 352 op/s 2024-05-08T06:02:41.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:41 smithi112 ceph-mon[30003]: pgmap v662: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1008 KiB/s wr, 352 op/s 2024-05-08T06:02:41.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:41 smithi162 ceph-mon[34800]: pgmap v662: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1008 KiB/s wr, 352 op/s 2024-05-08T06:02:43.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:43 smithi150 ceph-mon[34315]: pgmap v663: 97 pgs: 97 active+clean; 139 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.1 MiB/s wr, 391 op/s 2024-05-08T06:02:43.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:43 smithi112 ceph-mon[30003]: pgmap v663: 97 pgs: 97 active+clean; 139 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.1 MiB/s wr, 391 op/s 2024-05-08T06:02:43.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:43 smithi162 ceph-mon[34800]: pgmap v663: 97 pgs: 97 active+clean; 139 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.1 MiB/s wr, 391 op/s 2024-05-08T06:02:45.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:45 smithi150 ceph-mon[34315]: pgmap v664: 97 pgs: 97 active+clean; 135 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 877 KiB/s wr, 315 op/s 2024-05-08T06:02:45.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:45 smithi112 ceph-mon[30003]: pgmap v664: 97 pgs: 97 active+clean; 135 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 877 KiB/s wr, 315 op/s 2024-05-08T06:02:45.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:45 smithi162 ceph-mon[34800]: pgmap v664: 97 pgs: 97 active+clean; 135 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 877 KiB/s wr, 315 op/s 2024-05-08T06:02:47.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:47 smithi150 ceph-mon[34315]: pgmap v665: 97 pgs: 97 active+clean; 132 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.1 MiB/s wr, 400 op/s 2024-05-08T06:02:47.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:47 smithi112 ceph-mon[30003]: pgmap v665: 97 pgs: 97 active+clean; 132 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.1 MiB/s wr, 400 op/s 2024-05-08T06:02:47.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:47 smithi162 ceph-mon[34800]: pgmap v665: 97 pgs: 97 active+clean; 132 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.1 MiB/s wr, 400 op/s 2024-05-08T06:02:48.920 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T06:02:49.398 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:02:49.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:49 smithi150 ceph-mon[34315]: pgmap v666: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 953 KiB/s wr, 341 op/s 2024-05-08T06:02:49.749 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:49 smithi112 ceph-mon[30003]: pgmap v666: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 953 KiB/s wr, 341 op/s 2024-05-08T06:02:49.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:49 smithi162 ceph-mon[34800]: pgmap v666: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 953 KiB/s wr, 341 op/s 2024-05-08T06:02:51.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:51 smithi112 ceph-mon[30003]: pgmap v667: 97 pgs: 97 active+clean; 135 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 930 KiB/s wr, 356 op/s 2024-05-08T06:02:51.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:51 smithi150 ceph-mon[34315]: pgmap v667: 97 pgs: 97 active+clean; 135 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 930 KiB/s wr, 356 op/s 2024-05-08T06:02:51.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.752+0000 7f6d2ee00700 1 -- 172.21.15.112:0/3285313489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d28108850 msgr2=0x7f6d281103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:51.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.752+0000 7f6d2ee00700 1 --2- 172.21.15.112:0/3285313489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d28108850 0x7f6d281103d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000b0d0 tx=0x7f6d2000b3e0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.752+0000 7f6d2ee00700 1 -- 172.21.15.112:0/3285313489 shutdown_connections 2024-05-08T06:02:51.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.752+0000 7f6d2ee00700 1 --2- 172.21.15.112:0/3285313489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d28108850 0x7f6d281103d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.752+0000 7f6d2ee00700 1 --2- 172.21.15.112:0/3285313489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d28103750 0x7f6d28103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.752+0000 7f6d2ee00700 1 --2- 172.21.15.112:0/3285313489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d28104100 0x7f6d28108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.752+0000 7f6d2ee00700 1 -- 172.21.15.112:0/3285313489 >> 172.21.15.112:0/3285313489 conn(0x7f6d2807a600 msgr2=0x7f6d2807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:51.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.752+0000 7f6d2ee00700 1 -- 172.21.15.112:0/3285313489 shutdown_connections 2024-05-08T06:02:51.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.752+0000 7f6d2ee00700 1 -- 172.21.15.112:0/3285313489 wait complete. 2024-05-08T06:02:51.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.753+0000 7f6d2ee00700 1 Processor -- start 2024-05-08T06:02:51.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.754+0000 7f6d2ee00700 1 -- start start 2024-05-08T06:02:51.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.754+0000 7f6d2ee00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d28103750 0x7f6d2819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.754+0000 7f6d2ee00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d28104100 0x7f6d2819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.754+0000 7f6d2ee00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d28108850 0x7f6d281998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.754+0000 7f6d2ee00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d28113670 con 0x7f6d28103750 2024-05-08T06:02:51.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.754+0000 7f6d2ee00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281134f0 con 0x7f6d28108850 2024-05-08T06:02:51.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.754+0000 7f6d2ee00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281137f0 con 0x7f6d28104100 2024-05-08T06:02:51.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.754+0000 7f6d27e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d28103750 0x7f6d2819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.754+0000 7f6d2ce00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d28108850 0x7f6d281998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.754+0000 7f6d27e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d28103750 0x7f6d2819f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58386/0 (socket says 172.21.15.112:58386) 2024-05-08T06:02:51.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.754+0000 7f6d27e00700 1 -- 172.21.15.112:0/2219325390 learned_addr learned my addr 172.21.15.112:0/2219325390 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:51.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.754+0000 7f6d2ce00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d28108850 0x7f6d281998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:45084/0 (socket says 172.21.15.112:45084) 2024-05-08T06:02:51.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.755+0000 7f6d27400700 1 --2- 172.21.15.112:0/2219325390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d28104100 0x7f6d2819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.755+0000 7f6d2ce00700 1 -- 172.21.15.112:0/2219325390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d28104100 msgr2=0x7f6d2819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:51.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.755+0000 7f6d2ce00700 1 --2- 172.21.15.112:0/2219325390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d28104100 0x7f6d2819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.755+0000 7f6d2ce00700 1 -- 172.21.15.112:0/2219325390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d28103750 msgr2=0x7f6d2819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:51.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.755+0000 7f6d2ce00700 1 --2- 172.21.15.112:0/2219325390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d28103750 0x7f6d2819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.755+0000 7f6d2ce00700 1 -- 172.21.15.112:0/2219325390 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2819a050 con 0x7f6d28108850 2024-05-08T06:02:51.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.755+0000 7f6d27e00700 1 --2- 172.21.15.112:0/2219325390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d28103750 0x7f6d2819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:02:51.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.755+0000 7f6d2ce00700 1 --2- 172.21.15.112:0/2219325390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d28108850 0x7f6d281998f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000b770 tx=0x7f6d20007980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:51.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.756+0000 7f6d24c00700 1 -- 172.21.15.112:0/2219325390 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20004c50 con 0x7f6d28108850 2024-05-08T06:02:51.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.756+0000 7f6d24c00700 1 -- 172.21.15.112:0/2219325390 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d20007dc0 con 0x7f6d28108850 2024-05-08T06:02:51.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.756+0000 7f6d24c00700 1 -- 172.21.15.112:0/2219325390 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20005410 con 0x7f6d28108850 2024-05-08T06:02:51.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.756+0000 7f6d2ee00700 1 -- 172.21.15.112:0/2219325390 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2819a2e0 con 0x7f6d28108850 2024-05-08T06:02:51.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.756+0000 7f6d2ee00700 1 -- 172.21.15.112:0/2219325390 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d280756d0 con 0x7f6d28108850 2024-05-08T06:02:51.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.759+0000 7f6d24c00700 1 -- 172.21.15.112:0/2219325390 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6d200055b0 con 0x7f6d28108850 2024-05-08T06:02:51.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.760+0000 7f6d39841b80 1 Processor -- start 2024-05-08T06:02:51.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.761+0000 7f6d39841b80 1 -- start start 2024-05-08T06:02:51.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.761+0000 7f6d24c00700 1 --2- 172.21.15.112:0/2219325390 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6d0c0b2900 0x7f6d0c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.761+0000 7f6d39841b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d740bc0 0x56037d740fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.761+0000 7f6d39841b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d7414e0 0x56037d73b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.761+0000 7f6d24c00700 1 -- 172.21.15.112:0/2219325390 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f6d200e6ff0 con 0x7f6d28108850 2024-05-08T06:02:51.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.761+0000 7f6d39841b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d73bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.761+0000 7f6d39841b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56037d68da20 con 0x56037d7414e0 2024-05-08T06:02:51.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.761+0000 7f6d39841b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56037d62b760 con 0x56037d740bc0 2024-05-08T06:02:51.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.761+0000 7f6d39841b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56037d6899e0 con 0x56037d73b8f0 2024-05-08T06:02:51.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.761+0000 7f6d27400700 1 --2- 172.21.15.112:0/2219325390 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6d0c0b2900 0x7f6d0c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.762+0000 7f6d27e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d740bc0 0x56037d740fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.762+0000 7f6d27400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d7414e0 0x56037d73b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.762+0000 7f6d27e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d740bc0 0x56037d740fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:45098/0 (socket says 172.21.15.112:45098) 2024-05-08T06:02:51.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.762+0000 7f6d27e00700 1 -- 172.21.15.112:0/1876390263 learned_addr learned my addr 172.21.15.112:0/1876390263 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:51.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.762+0000 7f6d2ce00700 1 --2- 172.21.15.112:0/1876390263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d73bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.762+0000 7f6d27e00700 1 -- 172.21.15.112:0/1876390263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 msgr2=0x56037d73bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:51.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.762+0000 7f6d27e00700 1 --2- 172.21.15.112:0/1876390263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d73bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.762+0000 7f6d27e00700 1 -- 172.21.15.112:0/1876390263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d7414e0 msgr2=0x56037d73b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:51.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.762+0000 7f6d27e00700 1 --2- 172.21.15.112:0/1876390263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d7414e0 0x56037d73b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.762+0000 7f6d27e00700 1 -- 172.21.15.112:0/1876390263 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56037d5da8d0 con 0x56037d740bc0 2024-05-08T06:02:51.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.762+0000 7f6d27400700 1 --2- 172.21.15.112:0/1876390263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d7414e0 0x56037d73b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.762+0000 7f6d27e00700 1 --2- 172.21.15.112:0/1876390263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d740bc0 0x56037d740fa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6d18011680 tx=0x7f6d18011990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=931cbb373a1de6fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.763+0000 7f6d1d600700 1 -- 172.21.15.112:0/1876390263 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18004030 con 0x56037d740bc0 2024-05-08T06:02:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.763+0000 7f6d27400700 1 --2- 172.21.15.112:0/2219325390 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6d0c0b2900 0x7f6d0c0b4dc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x56037d7418e0 tx=0x7f6d140047c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.763+0000 7f6d1d600700 1 -- 172.21.15.112:0/1876390263 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d18003680 con 0x56037d740bc0 2024-05-08T06:02:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.763+0000 7f6d1d600700 1 -- 172.21.15.112:0/1876390263 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d180197f0 con 0x56037d740bc0 2024-05-08T06:02:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.764+0000 7f6d39841b80 1 -- 172.21.15.112:0/1876390263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d740bc0 msgr2=0x56037d740fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.764+0000 7f6d39841b80 1 --2- 172.21.15.112:0/1876390263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d740bc0 0x56037d740fa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6d18011680 tx=0x7f6d18011990 comp rx=0 tx=0).stop 2024-05-08T06:02:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.764+0000 7f6d39841b80 1 -- 172.21.15.112:0/1876390263 shutdown_connections 2024-05-08T06:02:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.764+0000 7f6d39841b80 1 --2- 172.21.15.112:0/1876390263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d73bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.764+0000 7f6d39841b80 1 --2- 172.21.15.112:0/1876390263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d740bc0 0x56037d740fa0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.764+0000 7f6d39841b80 1 --2- 172.21.15.112:0/1876390263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d7414e0 0x56037d73b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.764+0000 7f6d39841b80 1 -- 172.21.15.112:0/1876390263 >> 172.21.15.112:0/1876390263 conn(0x56037d6b36d0 msgr2=0x56037d735b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.764+0000 7f6d39841b80 1 -- 172.21.15.112:0/1876390263 shutdown_connections 2024-05-08T06:02:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.764+0000 7f6d39841b80 1 -- 172.21.15.112:0/1876390263 wait complete. 2024-05-08T06:02:51.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.765+0000 7f6d39841b80 1 -- start start 2024-05-08T06:02:51.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.765+0000 7f6d39841b80 1 Processor -- start 2024-05-08T06:02:51.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.766+0000 7f6d39841b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d747220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.766+0000 7f6d39841b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d740bc0 0x56037d7445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.766+0000 7f6d39841b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d7414e0 0x56037d744b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.766+0000 7f6d39841b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56037d747f20 con 0x56037d740bc0 2024-05-08T06:02:51.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.766+0000 7f6d39841b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56037d5f3b20 con 0x56037d7414e0 2024-05-08T06:02:51.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.766+0000 7f6d39841b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56037d7480a0 con 0x56037d73b8f0 2024-05-08T06:02:51.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.766+0000 7f6d39841b80 10 client.0 _ll_register_callbacks cb 0x56037d634c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:02:51.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.766+0000 7f6d27e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d747220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.766+0000 7f6d27400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d740bc0 0x56037d7445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.766+0000 7f6d27e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d747220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:41102/0 (socket says 172.21.15.112:41102) 2024-05-08T06:02:51.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.766+0000 7f6d27e00700 1 -- 172.21.15.112:0/3707689567 learned_addr learned my addr 172.21.15.112:0/3707689567 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:51.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.767+0000 7f6d2ce00700 1 --2- 172.21.15.112:0/3707689567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d7414e0 0x56037d744b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.767+0000 7f6d2ce00700 1 -- 172.21.15.112:0/3707689567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 msgr2=0x56037d747220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:51.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.767+0000 7f6d2ce00700 1 --2- 172.21.15.112:0/3707689567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d747220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.767+0000 7f6d2ce00700 1 -- 172.21.15.112:0/3707689567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d740bc0 msgr2=0x56037d7445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:51.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.767+0000 7f6d2ce00700 1 --2- 172.21.15.112:0/3707689567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d740bc0 0x56037d7445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.767+0000 7f6d2ce00700 1 -- 172.21.15.112:0/3707689567 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56037d653420 con 0x56037d7414e0 2024-05-08T06:02:51.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.767+0000 7f6d27e00700 1 --2- 172.21.15.112:0/3707689567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d747220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:02:51.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.767+0000 7f6d27400700 1 --2- 172.21.15.112:0/3707689567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d740bc0 0x56037d7445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:02:51.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.768+0000 7f6d2ce00700 1 --2- 172.21.15.112:0/3707689567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d7414e0 0x56037d744b00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6d200e4b10 tx=0x7f6d2000b3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7b1e45dc175059 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:51.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.768+0000 7f6d12000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:02:51.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.768+0000 7f6d12000700 1 -- 172.21.15.112:0/3707689567 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d200aed70 con 0x56037d7414e0 2024-05-08T06:02:51.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.768+0000 7f6d12000700 1 -- 172.21.15.112:0/3707689567 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f6d200058a0 con 0x56037d7414e0 2024-05-08T06:02:51.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.768+0000 7f6d39841b80 1 -- 172.21.15.112:0/3707689567 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d18003680 con 0x56037d7414e0 2024-05-08T06:02:51.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.768+0000 7f6d12000700 1 -- 172.21.15.112:0/3707689567 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6d200c8050 con 0x56037d7414e0 2024-05-08T06:02:51.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.768+0000 7f6d39841b80 10 client.25931 fetch_fsmap learned FSMap version 33 2024-05-08T06:02:51.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.768+0000 7f6d39841b80 1 -- 172.21.15.112:0/3707689567 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56037d747f20 con 0x56037d7414e0 2024-05-08T06:02:51.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.768+0000 7f6d12000700 1 -- 172.21.15.112:0/3707689567 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d18003680 con 0x56037d7414e0 2024-05-08T06:02:51.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.769+0000 7f6d12000700 1 -- 172.21.15.112:0/3707689567 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f6d20023070 con 0x56037d7414e0 2024-05-08T06:02:51.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.769+0000 7f6d39841b80 10 client.25931 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:02:51.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.769+0000 7f6d39841b80 10 client.25931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:02:51.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.769+0000 7f6d39841b80 20 client.25931 populate_metadata read hostname 'smithi112' 2024-05-08T06:02:51.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.769+0000 7f6d39841b80 1 --2- 172.21.15.112:0/3707689567 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56037d75d5e0 0x56037d75fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.769+0000 7f6d39841b80 4 client.25931 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:02:51.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.769+0000 7f6d39841b80 1 -- 172.21.15.112:0/3707689567 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56037d708140 con 0x56037d75d5e0 2024-05-08T06:02:51.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.769+0000 7f6d27e00700 1 --2- 172.21.15.112:0/3707689567 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56037d75d5e0 0x56037d75fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.770+0000 7f6d27e00700 1 --2- 172.21.15.112:0/3707689567 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56037d75d5e0 0x56037d75fa20 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d7aadd5cc6c2858 server_cookie=ef686802a16b1fee in_seq=0 out_seq=0 2024-05-08T06:02:51.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.770+0000 7f6d12000700 10 client.25931 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:51.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.771+0000 7f6d12000700 1 -- 172.21.15.112:0/3707689567 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56037d708140 con 0x56037d75d5e0 2024-05-08T06:02:51.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.771+0000 7f6d12000700 10 client.25931 handle_command_reply: tid=0 2024-05-08T06:02:51.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.771+0000 7f6d39841b80 1 client.25931 shutdown 2024-05-08T06:02:51.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.771+0000 7f6d39841b80 1 -- 172.21.15.112:0/3707689567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d7414e0 msgr2=0x56037d744b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:51.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.771+0000 7f6d39841b80 1 --2- 172.21.15.112:0/3707689567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d7414e0 0x56037d744b00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6d200e4b10 tx=0x7f6d2000b3e0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.771+0000 7f6d39841b80 1 -- 172.21.15.112:0/3707689567 shutdown_connections 2024-05-08T06:02:51.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.771+0000 7f6d39841b80 1 --2- 172.21.15.112:0/3707689567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d7414e0 0x56037d744b00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.771+0000 7f6d39841b80 1 --2- 172.21.15.112:0/3707689567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d747220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.771+0000 7f6d39841b80 1 --2- 172.21.15.112:0/3707689567 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56037d75d5e0 0x56037d75fa20 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.771+0000 7f6d39841b80 1 --2- 172.21.15.112:0/3707689567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d740bc0 0x56037d7445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.771+0000 7f6d39841b80 1 -- 172.21.15.112:0/3707689567 >> 172.21.15.112:0/3707689567 conn(0x56037d6b36d0 msgr2=0x56037d735190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:51.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.771+0000 7f6d12000700 0 client.25931 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:51.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.772+0000 7f6d39841b80 1 -- 172.21.15.112:0/3707689567 shutdown_connections 2024-05-08T06:02:51.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.772+0000 7f6d39841b80 1 -- 172.21.15.112:0/3707689567 wait complete. 2024-05-08T06:02:51.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.772+0000 7f6d39841b80 20 client.25931 trim_cache size 0 max 16384 2024-05-08T06:02:51.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.798+0000 7f6d39841b80 1 Processor -- start 2024-05-08T06:02:51.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.798+0000 7f6d39841b80 1 -- start start 2024-05-08T06:02:51.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.799+0000 7f6d39841b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d73b8f0 0x56037d73b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.799+0000 7f6d39841b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d740bc0 0x56037d759840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.799+0000 7f6d39841b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d7414e0 0x56037d742020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.799+0000 7f6d39841b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281ae0f0 con 0x56037d73b8f0 2024-05-08T06:02:51.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.799+0000 7f6d39841b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281ab980 con 0x56037d7414e0 2024-05-08T06:02:51.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.799+0000 7f6d39841b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281a97f0 con 0x56037d740bc0 2024-05-08T06:02:51.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.799+0000 7f6d27400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d740bc0 0x56037d759840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.799+0000 7f6d27400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d740bc0 0x56037d759840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:41110/0 (socket says 172.21.15.112:41110) 2024-05-08T06:02:51.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.799+0000 7f6d27400700 1 -- 172.21.15.112:0/1680434057 learned_addr learned my addr 172.21.15.112:0/1680434057 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:51.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.799+0000 7f6d27e00700 1 --2- 172.21.15.112:0/1680434057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d73b8f0 0x56037d73b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.800+0000 7f6d27400700 1 -- 172.21.15.112:0/1680434057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d7414e0 msgr2=0x56037d742020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T06:02:51.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.800+0000 7f6d27400700 1 --2- 172.21.15.112:0/1680434057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d7414e0 0x56037d742020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.800+0000 7f6d27400700 1 -- 172.21.15.112:0/1680434057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d73b8f0 msgr2=0x56037d73b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:51.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.800+0000 7f6d27400700 1 --2- 172.21.15.112:0/1680434057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d73b8f0 0x56037d73b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.800+0000 7f6d27400700 1 -- 172.21.15.112:0/1680434057 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56037d742b30 con 0x56037d740bc0 2024-05-08T06:02:51.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.800+0000 7f6d27400700 1 --2- 172.21.15.112:0/1680434057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d740bc0 0x56037d759840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6d1401f480 tx=0x7f6d14012040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3a25d9ae2d7d332 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:51.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.801+0000 7f6d1cc00700 1 -- 172.21.15.112:0/1680434057 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14004830 con 0x56037d740bc0 2024-05-08T06:02:51.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.801+0000 7f6d1cc00700 1 -- 172.21.15.112:0/1680434057 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d14005020 con 0x56037d740bc0 2024-05-08T06:02:51.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.801+0000 7f6d1cc00700 1 -- 172.21.15.112:0/1680434057 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1401e920 con 0x56037d740bc0 2024-05-08T06:02:51.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.802+0000 7f6d39841b80 1 -- 172.21.15.112:0/1680434057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d740bc0 msgr2=0x56037d759840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:51.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.802+0000 7f6d39841b80 1 --2- 172.21.15.112:0/1680434057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d740bc0 0x56037d759840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6d1401f480 tx=0x7f6d14012040 comp rx=0 tx=0).stop 2024-05-08T06:02:51.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.802+0000 7f6d39841b80 1 -- 172.21.15.112:0/1680434057 shutdown_connections 2024-05-08T06:02:51.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.802+0000 7f6d39841b80 1 --2- 172.21.15.112:0/1680434057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d7414e0 0x56037d742020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.802+0000 7f6d39841b80 1 --2- 172.21.15.112:0/1680434057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d73b8f0 0x56037d73b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.802+0000 7f6d39841b80 1 --2- 172.21.15.112:0/1680434057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d740bc0 0x56037d759840 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.802+0000 7f6d39841b80 1 -- 172.21.15.112:0/1680434057 >> 172.21.15.112:0/1680434057 conn(0x56037d6b36d0 msgr2=0x56037d741910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:51.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.803+0000 7f6d39841b80 1 -- 172.21.15.112:0/1680434057 shutdown_connections 2024-05-08T06:02:51.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.803+0000 7f6d39841b80 1 -- 172.21.15.112:0/1680434057 wait complete. 2024-05-08T06:02:51.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.803+0000 7f6d39841b80 1 -- start start 2024-05-08T06:02:51.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.803+0000 7f6d39841b80 1 Processor -- start 2024-05-08T06:02:51.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.804+0000 7f6d39841b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d764c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.804+0000 7f6d39841b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d740bc0 0x56037d7651c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.804+0000 7f6d39841b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d7414e0 0x56037d765700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.804+0000 7f6d39841b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281a9e40 con 0x56037d7414e0 2024-05-08T06:02:51.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.804+0000 7f6d39841b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281ab2b0 con 0x56037d740bc0 2024-05-08T06:02:51.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.804+0000 7f6d39841b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281aa510 con 0x56037d73b8f0 2024-05-08T06:02:51.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.805+0000 7f6d39841b80 10 client.0 _ll_register_callbacks cb 0x56037d634c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:02:51.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.805+0000 7f6d27e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d764c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.805+0000 7f6d27e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d764c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:41124/0 (socket says 172.21.15.112:41124) 2024-05-08T06:02:51.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.805+0000 7f6d27e00700 1 -- 172.21.15.112:0/2671733040 learned_addr learned my addr 172.21.15.112:0/2671733040 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:51.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.805+0000 7f6d27400700 1 --2- 172.21.15.112:0/2671733040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d740bc0 0x56037d7651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.805+0000 7f6d27e00700 1 -- 172.21.15.112:0/2671733040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d740bc0 msgr2=0x56037d7651c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:51.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.805+0000 7f6d27e00700 1 --2- 172.21.15.112:0/2671733040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d740bc0 0x56037d7651c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.805+0000 7f6d27e00700 1 -- 172.21.15.112:0/2671733040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d7414e0 msgr2=0x56037d765700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:02:51.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.805+0000 7f6d2ce00700 1 --2- 172.21.15.112:0/2671733040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d7414e0 0x56037d765700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.805+0000 7f6d27e00700 1 --2- 172.21.15.112:0/2671733040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d7414e0 0x56037d765700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.805+0000 7f6d27e00700 1 -- 172.21.15.112:0/2671733040 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56037d7c7080 con 0x56037d73b8f0 2024-05-08T06:02:51.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.805+0000 7f6d27400700 1 --2- 172.21.15.112:0/2671733040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d740bc0 0x56037d7651c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:02:51.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.805+0000 7f6d2ce00700 1 --2- 172.21.15.112:0/2671733040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d7414e0 0x56037d765700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:02:51.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.806+0000 7f6d27e00700 1 --2- 172.21.15.112:0/2671733040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d764c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009670 tx=0x7f6d18003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=769984e2d7639cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:51.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.806+0000 7f6d12000700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:02:51.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.806+0000 7f6d12000700 1 -- 172.21.15.112:0/2671733040 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d180054a0 con 0x56037d73b8f0 2024-05-08T06:02:51.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.806+0000 7f6d12000700 1 -- 172.21.15.112:0/2671733040 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f6d1802a050 con 0x56037d73b8f0 2024-05-08T06:02:51.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.806+0000 7f6d39841b80 1 -- 172.21.15.112:0/2671733040 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d14005020 con 0x56037d73b8f0 2024-05-08T06:02:51.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.806+0000 7f6d12000700 1 -- 172.21.15.112:0/2671733040 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6d18005640 con 0x56037d73b8f0 2024-05-08T06:02:51.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.806+0000 7f6d39841b80 10 client.25978 fetch_fsmap learned FSMap version 33 2024-05-08T06:02:51.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.806+0000 7f6d39841b80 1 -- 172.21.15.112:0/2671733040 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56037d7c8330 con 0x56037d73b8f0 2024-05-08T06:02:51.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.807+0000 7f6d12000700 1 -- 172.21.15.112:0/2671733040 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d14005020 con 0x56037d73b8f0 2024-05-08T06:02:51.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.807+0000 7f6d12000700 1 -- 172.21.15.112:0/2671733040 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f6d18019450 con 0x56037d73b8f0 2024-05-08T06:02:51.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.807+0000 7f6d39841b80 10 client.25978 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:02:51.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.807+0000 7f6d39841b80 10 client.25978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:02:51.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.807+0000 7f6d39841b80 20 client.25978 populate_metadata read hostname 'smithi112' 2024-05-08T06:02:51.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.807+0000 7f6d39841b80 1 --2- 172.21.15.112:0/2671733040 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56037d7c88a0 0x56037d7cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:51.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.807+0000 7f6d39841b80 4 client.25978 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T06:02:51.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.807+0000 7f6d39841b80 1 -- 172.21.15.112:0/2671733040 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56037d69b400 con 0x56037d7c88a0 2024-05-08T06:02:51.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.807+0000 7f6d27400700 1 --2- 172.21.15.112:0/2671733040 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56037d7c88a0 0x56037d7cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:51.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.808+0000 7f6d27400700 1 --2- 172.21.15.112:0/2671733040 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56037d7c88a0 0x56037d7cac90 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6379eea8a660e47d server_cookie=a80367594ee12441 in_seq=0 out_seq=0 2024-05-08T06:02:51.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.808+0000 7f6d12000700 10 client.25978 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:51.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.808+0000 7f6d12000700 1 -- 172.21.15.112:0/2671733040 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56037d69b400 con 0x56037d7c88a0 2024-05-08T06:02:51.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.808+0000 7f6d12000700 10 client.25978 handle_command_reply: tid=0 2024-05-08T06:02:51.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.808+0000 7f6d39841b80 1 client.25978 shutdown 2024-05-08T06:02:51.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.809+0000 7f6d39841b80 1 -- 172.21.15.112:0/2671733040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 msgr2=0x56037d764c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:51.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.809+0000 7f6d39841b80 1 --2- 172.21.15.112:0/2671733040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d764c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009670 tx=0x7f6d18003730 comp rx=0 tx=0).stop 2024-05-08T06:02:51.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.809+0000 7f6d39841b80 1 -- 172.21.15.112:0/2671733040 shutdown_connections 2024-05-08T06:02:51.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.809+0000 7f6d39841b80 1 --2- 172.21.15.112:0/2671733040 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x56037d7c88a0 0x56037d7cac90 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.809+0000 7f6d39841b80 1 --2- 172.21.15.112:0/2671733040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56037d7414e0 0x56037d765700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.809+0000 7f6d39841b80 1 --2- 172.21.15.112:0/2671733040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56037d73b8f0 0x56037d764c80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.809+0000 7f6d39841b80 1 --2- 172.21.15.112:0/2671733040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56037d740bc0 0x56037d7651c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.809+0000 7f6d39841b80 1 -- 172.21.15.112:0/2671733040 >> 172.21.15.112:0/2671733040 conn(0x56037d6b36d0 msgr2=0x56037d73f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:51.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.809+0000 7f6d12000700 0 client.25978 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:02:51.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.809+0000 7f6d39841b80 1 -- 172.21.15.112:0/2671733040 shutdown_connections 2024-05-08T06:02:51.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.809+0000 7f6d39841b80 1 -- 172.21.15.112:0/2671733040 wait complete. 2024-05-08T06:02:51.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.809+0000 7f6d39841b80 20 client.25978 trim_cache size 0 max 16384 2024-05-08T06:02:51.811 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:02:51.811 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-08T06:02:51.811 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-08T06:02:51.812 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:02:51.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.810+0000 7f6d2ee00700 1 -- 172.21.15.112:0/2219325390 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6d0c0b2900 msgr2=0x7f6d0c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:51.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.810+0000 7f6d2ee00700 1 --2- 172.21.15.112:0/2219325390 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6d0c0b2900 0x7f6d0c0b4dc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x56037d7418e0 tx=0x7f6d140047c0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.810+0000 7f6d2ee00700 1 -- 172.21.15.112:0/2219325390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d28108850 msgr2=0x7f6d281998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:51.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.810+0000 7f6d2ee00700 1 --2- 172.21.15.112:0/2219325390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d28108850 0x7f6d281998f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000b770 tx=0x7f6d20007980 comp rx=0 tx=0).stop 2024-05-08T06:02:51.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.810+0000 7f6d2ee00700 1 -- 172.21.15.112:0/2219325390 shutdown_connections 2024-05-08T06:02:51.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.810+0000 7f6d2ee00700 1 --2- 172.21.15.112:0/2219325390 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f6d0c0b2900 0x7f6d0c0b4dc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.810+0000 7f6d2ee00700 1 --2- 172.21.15.112:0/2219325390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d28108850 0x7f6d281998f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.810+0000 7f6d2ee00700 1 --2- 172.21.15.112:0/2219325390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d28103750 0x7f6d2819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.810+0000 7f6d2ee00700 1 --2- 172.21.15.112:0/2219325390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d28104100 0x7f6d2819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:51.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.810+0000 7f6d2ee00700 1 -- 172.21.15.112:0/2219325390 >> 172.21.15.112:0/2219325390 conn(0x7f6d2807a600 msgr2=0x7f6d28101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:51.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.810+0000 7f6d2ee00700 1 -- 172.21.15.112:0/2219325390 shutdown_connections 2024-05-08T06:02:51.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:51.810+0000 7f6d2ee00700 1 -- 172.21.15.112:0/2219325390 wait complete. 2024-05-08T06:02:51.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:51 smithi162 ceph-mon[34800]: pgmap v667: 97 pgs: 97 active+clean; 135 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 930 KiB/s wr, 356 op/s 2024-05-08T06:02:53.393 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T06:02:53.394 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:02:53.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:53 smithi150 ceph-mon[34315]: pgmap v668: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.0 MiB/s wr, 401 op/s 2024-05-08T06:02:53.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:53 smithi112 ceph-mon[30003]: pgmap v668: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.0 MiB/s wr, 401 op/s 2024-05-08T06:02:53.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:53 smithi162 ceph-mon[34800]: pgmap v668: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.0 MiB/s wr, 401 op/s 2024-05-08T06:02:53.887 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:02:54.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:54 smithi112 ceph-mon[30003]: pgmap v669: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 825 KiB/s wr, 321 op/s 2024-05-08T06:02:54.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:54 smithi162 ceph-mon[34800]: pgmap v669: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 825 KiB/s wr, 321 op/s 2024-05-08T06:02:54.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:54 smithi150 ceph-mon[34315]: pgmap v669: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 825 KiB/s wr, 321 op/s 2024-05-08T06:02:56.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.439+0000 7f3458e00700 1 -- 172.21.15.112:0/124593810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3454103750 msgr2=0x7f3454103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:56.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.439+0000 7f3458e00700 1 --2- 172.21.15.112:0/124593810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3454103750 0x7f3454103b30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f34440099c0 tx=0x7f3444009cd0 comp rx=0 tx=0).stop 2024-05-08T06:02:56.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.439+0000 7f3458e00700 1 -- 172.21.15.112:0/124593810 shutdown_connections 2024-05-08T06:02:56.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.439+0000 7f3458e00700 1 --2- 172.21.15.112:0/124593810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3454108850 0x7f34541103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:56.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.439+0000 7f3458e00700 1 --2- 172.21.15.112:0/124593810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3454103750 0x7f3454103b30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:56.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.439+0000 7f3458e00700 1 --2- 172.21.15.112:0/124593810 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3454104100 0x7f3454108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.439+0000 7f3458e00700 1 -- 172.21.15.112:0/124593810 >> 172.21.15.112:0/124593810 conn(0x7f345407a600 msgr2=0x7f345407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.439+0000 7f3458e00700 1 -- 172.21.15.112:0/124593810 shutdown_connections 2024-05-08T06:02:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.439+0000 7f3458e00700 1 -- 172.21.15.112:0/124593810 wait complete. 2024-05-08T06:02:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.440+0000 7f3458e00700 1 Processor -- start 2024-05-08T06:02:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.441+0000 7f3458e00700 1 -- start start 2024-05-08T06:02:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.441+0000 7f3458e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3454103750 0x7f345419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:56.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.441+0000 7f3458e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3454104100 0x7f345419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:56.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.441+0000 7f3458e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3454108850 0x7f34541a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:56.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.441+0000 7f3458e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3454113620 con 0x7f3454103750 2024-05-08T06:02:56.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.441+0000 7f3458e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f34541134a0 con 0x7f3454104100 2024-05-08T06:02:56.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.441+0000 7f3458e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f34541137a0 con 0x7f3454108850 2024-05-08T06:02:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.441+0000 7f3451e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3454103750 0x7f345419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.441+0000 7f3452800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3454108850 0x7f34541a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.441+0000 7f3451e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3454103750 0x7f345419bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58588/0 (socket says 172.21.15.112:58588) 2024-05-08T06:02:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.441+0000 7f3451e00700 1 -- 172.21.15.112:0/3038527918 learned_addr learned my addr 172.21.15.112:0/3038527918 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:02:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.441+0000 7f3451400700 1 --2- 172.21.15.112:0/3038527918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3454104100 0x7f345419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.442+0000 7f3452800700 1 -- 172.21.15.112:0/3038527918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3454104100 msgr2=0x7f345419c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.442+0000 7f3452800700 1 --2- 172.21.15.112:0/3038527918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3454104100 0x7f345419c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.442+0000 7f3452800700 1 -- 172.21.15.112:0/3038527918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3454103750 msgr2=0x7f345419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.442+0000 7f3452800700 1 --2- 172.21.15.112:0/3038527918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3454103750 0x7f345419bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.442+0000 7f3452800700 1 -- 172.21.15.112:0/3038527918 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34541a0d20 con 0x7f3454108850 2024-05-08T06:02:56.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.442+0000 7f3452800700 1 --2- 172.21.15.112:0/3038527918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3454108850 0x7f34541a05e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f344c00efc0 tx=0x7f344c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:56.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.442+0000 7f344aa00700 1 -- 172.21.15.112:0/3038527918 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c0092e0 con 0x7f3454108850 2024-05-08T06:02:56.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.442+0000 7f344aa00700 1 -- 172.21.15.112:0/3038527918 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f344c004bb0 con 0x7f3454108850 2024-05-08T06:02:56.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.442+0000 7f3458e00700 1 -- 172.21.15.112:0/3038527918 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34541a0fb0 con 0x7f3454108850 2024-05-08T06:02:56.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.442+0000 7f344aa00700 1 -- 172.21.15.112:0/3038527918 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c004020 con 0x7f3454108850 2024-05-08T06:02:56.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.443+0000 7f3458e00700 1 -- 172.21.15.112:0/3038527918 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34541a8890 con 0x7f3454108850 2024-05-08T06:02:56.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.444+0000 7f3458e00700 1 -- 172.21.15.112:0/3038527918 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f345406a190 con 0x7f3454108850 2024-05-08T06:02:56.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.447+0000 7f344aa00700 1 -- 172.21.15.112:0/3038527918 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f344c01b030 con 0x7f3454108850 2024-05-08T06:02:56.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.448+0000 7f344aa00700 1 --2- 172.21.15.112:0/3038527918 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f34380b2900 0x7f34380b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:56.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.449+0000 7f344aa00700 1 -- 172.21.15.112:0/3038527918 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f344c0e1cf0 con 0x7f3454108850 2024-05-08T06:02:56.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.449+0000 7f3451e00700 1 --2- 172.21.15.112:0/3038527918 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f34380b2900 0x7f34380b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:56.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.449+0000 7f3451e00700 1 --2- 172.21.15.112:0/3038527918 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f34380b2900 0x7f34380b4dc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3444009990 tx=0x7f3444005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:56.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.452+0000 7f344aa00700 1 -- 172.21.15.112:0/3038527918 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f344c022020 con 0x7f3454108850 2024-05-08T06:02:56.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.787+0000 7f3458e00700 1 -- 172.21.15.112:0/3038527918 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34541a8a60 con 0x7f3454108850 2024-05-08T06:02:56.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.788+0000 7f344aa00700 1 -- 172.21.15.112:0/3038527918 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11891 (secure 0 0 0) 0x7f344c010040 con 0x7f3454108850 2024-05-08T06:02:56.789 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T06:02:56.789 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T06:01:28.310032+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24518,24511]},"id":1}]} 2024-05-08T06:02:56.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.791+0000 7f3458e00700 1 -- 172.21.15.112:0/3038527918 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f34380b2900 msgr2=0x7f34380b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:56.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.791+0000 7f3458e00700 1 --2- 172.21.15.112:0/3038527918 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f34380b2900 0x7f34380b4dc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3444009990 tx=0x7f3444005960 comp rx=0 tx=0).stop 2024-05-08T06:02:56.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.791+0000 7f3458e00700 1 -- 172.21.15.112:0/3038527918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3454108850 msgr2=0x7f34541a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:56.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.791+0000 7f3458e00700 1 --2- 172.21.15.112:0/3038527918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3454108850 0x7f34541a05e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f344c00efc0 tx=0x7f344c00c3a0 comp rx=0 tx=0).stop 2024-05-08T06:02:56.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.791+0000 7f3458e00700 1 -- 172.21.15.112:0/3038527918 shutdown_connections 2024-05-08T06:02:56.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.791+0000 7f3458e00700 1 --2- 172.21.15.112:0/3038527918 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f34380b2900 0x7f34380b4dc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:56.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.791+0000 7f3458e00700 1 --2- 172.21.15.112:0/3038527918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3454108850 0x7f34541a05e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:56.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.791+0000 7f3458e00700 1 --2- 172.21.15.112:0/3038527918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3454103750 0x7f345419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:56.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.791+0000 7f3458e00700 1 --2- 172.21.15.112:0/3038527918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3454104100 0x7f345419c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:56.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.791+0000 7f3458e00700 1 -- 172.21.15.112:0/3038527918 >> 172.21.15.112:0/3038527918 conn(0x7f345407a600 msgr2=0x7f34541099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:56.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.791+0000 7f3458e00700 1 -- 172.21.15.112:0/3038527918 shutdown_connections 2024-05-08T06:02:56.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:02:56.792+0000 7f3458e00700 1 -- 172.21.15.112:0/3038527918 wait complete. 2024-05-08T06:02:56.793 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 33 2024-05-08T06:02:57.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:56 smithi112 ceph-mon[30003]: pgmap v670: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.1 MiB/s wr, 426 op/s 2024-05-08T06:02:57.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:56 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3038527918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:02:57.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:56 smithi150 ceph-mon[34315]: pgmap v670: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.1 MiB/s wr, 426 op/s 2024-05-08T06:02:57.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:56 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3038527918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:02:57.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:56 smithi162 ceph-mon[34800]: pgmap v670: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.1 MiB/s wr, 426 op/s 2024-05-08T06:02:57.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:56 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3038527918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:02:57.736 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-08T06:02:58.210 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:02:59.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:02:58 smithi112 ceph-mon[30003]: pgmap v671: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 945 KiB/s wr, 379 op/s 2024-05-08T06:02:59.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:02:58 smithi150 ceph-mon[34315]: pgmap v671: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 945 KiB/s wr, 379 op/s 2024-05-08T06:02:59.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:02:58 smithi162 ceph-mon[34800]: pgmap v671: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 945 KiB/s wr, 379 op/s 2024-05-08T06:03:00.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.627+0000 7fd93ba00700 1 -- 172.21.15.112:0/2986549855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd934104100 msgr2=0x7fd934108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:00.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.627+0000 7fd93ba00700 1 --2- 172.21.15.112:0/2986549855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd934104100 0x7fd934108120 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd9200099c0 tx=0x7fd920009cd0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.627+0000 7fd93ba00700 1 -- 172.21.15.112:0/2986549855 shutdown_connections 2024-05-08T06:03:00.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.627+0000 7fd93ba00700 1 --2- 172.21.15.112:0/2986549855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd934108850 0x7fd9341103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.627+0000 7fd93ba00700 1 --2- 172.21.15.112:0/2986549855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd934103750 0x7fd934103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.627+0000 7fd93ba00700 1 --2- 172.21.15.112:0/2986549855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd934104100 0x7fd934108120 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.627+0000 7fd93ba00700 1 -- 172.21.15.112:0/2986549855 >> 172.21.15.112:0/2986549855 conn(0x7fd93407a600 msgr2=0x7fd93407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:00.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.627+0000 7fd93ba00700 1 -- 172.21.15.112:0/2986549855 shutdown_connections 2024-05-08T06:03:00.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.627+0000 7fd93ba00700 1 -- 172.21.15.112:0/2986549855 wait complete. 2024-05-08T06:03:00.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.628+0000 7fd93ba00700 1 Processor -- start 2024-05-08T06:03:00.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.628+0000 7fd93ba00700 1 -- start start 2024-05-08T06:03:00.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.629+0000 7fd93ba00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd934103750 0x7fd9341a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.629+0000 7fd93ba00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd934104100 0x7fd9341a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.629+0000 7fd93ba00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd934108850 0x7fd93419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.629+0000 7fd93ba00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd934113680 con 0x7fd934108850 2024-05-08T06:03:00.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.629+0000 7fd93ba00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd934113500 con 0x7fd934103750 2024-05-08T06:03:00.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.629+0000 7fd93ba00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd934113800 con 0x7fd934104100 2024-05-08T06:03:00.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.629+0000 7fd939000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd934103750 0x7fd9341a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.629+0000 7fd939000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd934103750 0x7fd9341a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:37666/0 (socket says 172.21.15.112:37666) 2024-05-08T06:03:00.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.629+0000 7fd939000700 1 -- 172.21.15.112:0/2468677434 learned_addr learned my addr 172.21.15.112:0/2468677434 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:00.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.629+0000 7fd933e00700 1 --2- 172.21.15.112:0/2468677434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd934104100 0x7fd9341a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.629+0000 7fd939a00700 1 --2- 172.21.15.112:0/2468677434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd934108850 0x7fd93419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.629+0000 7fd939000700 1 -- 172.21.15.112:0/2468677434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd934104100 msgr2=0x7fd9341a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:00.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.629+0000 7fd939000700 1 --2- 172.21.15.112:0/2468677434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd934104100 0x7fd9341a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.630+0000 7fd939000700 1 -- 172.21.15.112:0/2468677434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd934108850 msgr2=0x7fd93419bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:00.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.630+0000 7fd939000700 1 --2- 172.21.15.112:0/2468677434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd934108850 0x7fd93419bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.630+0000 7fd939000700 1 -- 172.21.15.112:0/2468677434 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93419c260 con 0x7fd934103750 2024-05-08T06:03:00.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.630+0000 7fd939000700 1 --2- 172.21.15.112:0/2468677434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd934103750 0x7fd9341a1940 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd92800e9a0 tx=0x7fd92800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:00.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.630+0000 7fd931600700 1 -- 172.21.15.112:0/2468677434 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92800cbd0 con 0x7fd934103750 2024-05-08T06:03:00.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.630+0000 7fd931600700 1 -- 172.21.15.112:0/2468677434 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd92800cd70 con 0x7fd934103750 2024-05-08T06:03:00.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.630+0000 7fd93ba00700 1 -- 172.21.15.112:0/2468677434 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93419c550 con 0x7fd934103750 2024-05-08T06:03:00.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.630+0000 7fd931600700 1 -- 172.21.15.112:0/2468677434 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928018680 con 0x7fd934103750 2024-05-08T06:03:00.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.630+0000 7fd93ba00700 1 -- 172.21.15.112:0/2468677434 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9341a8810 con 0x7fd934103750 2024-05-08T06:03:00.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.636+0000 7fd931600700 1 -- 172.21.15.112:0/2468677434 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd928010430 con 0x7fd934103750 2024-05-08T06:03:00.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.637+0000 7fd9463d0b80 1 Processor -- start 2024-05-08T06:03:00.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.637+0000 7fd9463d0b80 1 -- start start 2024-05-08T06:03:00.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.637+0000 7fd9463d0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a060 0x55bf1461a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.637+0000 7fd9463d0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a980 0x55bf1461e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.637+0000 7fd9463d0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 0x55bf1461ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.637+0000 7fd9463d0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bf1456ca20 con 0x55bf1461a060 2024-05-08T06:03:00.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.637+0000 7fd9463d0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bf1450a760 con 0x55bf1461a980 2024-05-08T06:03:00.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.637+0000 7fd9463d0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55bf145689e0 con 0x55bf1461e9d0 2024-05-08T06:03:00.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.638+0000 7fd939a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a060 0x55bf1461a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.638+0000 7fd933e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 0x55bf1461ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.638+0000 7fd939a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a060 0x55bf1461a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58630/0 (socket says 172.21.15.112:58630) 2024-05-08T06:03:00.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.638+0000 7fd933e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 0x55bf1461ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:52530/0 (socket says 172.21.15.112:52530) 2024-05-08T06:03:00.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.638+0000 7fd939000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a980 0x55bf1461e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.638+0000 7fd939a00700 1 -- 172.21.15.112:0/1719612942 learned_addr learned my addr 172.21.15.112:0/1719612942 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:00.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.638+0000 7fd931600700 1 --2- 172.21.15.112:0/2468677434 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd91c0b2900 0x7fd91c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.638+0000 7fd933e00700 1 -- 172.21.15.112:0/1719612942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a980 msgr2=0x55bf1461e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:00.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.638+0000 7fd933e00700 1 --2- 172.21.15.112:0/1719612942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a980 0x55bf1461e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.638+0000 7fd933e00700 1 -- 172.21.15.112:0/1719612942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a060 msgr2=0x55bf1461a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:00.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.638+0000 7fd933e00700 1 --2- 172.21.15.112:0/1719612942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a060 0x55bf1461a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.638+0000 7fd933e00700 1 -- 172.21.15.112:0/1719612942 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf144b98d0 con 0x55bf1461e9d0 2024-05-08T06:03:00.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.638+0000 7fd939a00700 1 --2- 172.21.15.112:0/1719612942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a060 0x55bf1461a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:03:00.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.638+0000 7fd931600700 1 -- 172.21.15.112:0/2468677434 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd9280e2450 con 0x7fd934103750 2024-05-08T06:03:00.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.639+0000 7fd933e00700 1 --2- 172.21.15.112:0/1719612942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 0x55bf1461ee30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd92000b580 tx=0x7fd9200036c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ea46f1fa065f0bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:00.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.639+0000 7fd939a00700 1 --2- 172.21.15.112:0/2468677434 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd91c0b2900 0x7fd91c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.639+0000 7fd926000700 1 -- 172.21.15.112:0/1719612942 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920028070 con 0x55bf1461e9d0 2024-05-08T06:03:00.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.639+0000 7fd926000700 1 -- 172.21.15.112:0/1719612942 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9200095d0 con 0x55bf1461e9d0 2024-05-08T06:03:00.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.639+0000 7fd926000700 1 -- 172.21.15.112:0/1719612942 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92000b9f0 con 0x55bf1461e9d0 2024-05-08T06:03:00.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.640+0000 7fd9463d0b80 1 -- 172.21.15.112:0/1719612942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 msgr2=0x55bf1461ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:00.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.640+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/1719612942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 0x55bf1461ee30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd92000b580 tx=0x7fd9200036c0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.642+0000 7fd9463d0b80 1 -- 172.21.15.112:0/1719612942 shutdown_connections 2024-05-08T06:03:00.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.642+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/1719612942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 0x55bf1461ee30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.642+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/1719612942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a060 0x55bf1461a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.642+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/1719612942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a980 0x55bf1461e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.642+0000 7fd9463d0b80 1 -- 172.21.15.112:0/1719612942 >> 172.21.15.112:0/1719612942 conn(0x55bf14548b60 msgr2=0x55bf146147a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:00.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.642+0000 7fd939a00700 1 --2- 172.21.15.112:0/2468677434 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd91c0b2900 0x7fd91c0b4dc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009fd0 tx=0x7fd92c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:00.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.642+0000 7fd9463d0b80 1 -- 172.21.15.112:0/1719612942 shutdown_connections 2024-05-08T06:03:00.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.642+0000 7fd9463d0b80 1 -- 172.21.15.112:0/1719612942 wait complete. 2024-05-08T06:03:00.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.642+0000 7fd9463d0b80 1 -- start start 2024-05-08T06:03:00.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.643+0000 7fd9463d0b80 1 Processor -- start 2024-05-08T06:03:00.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.643+0000 7fd9463d0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a060 0x55bf14625ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.643+0000 7fd9463d0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a980 0x55bf14623280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.643+0000 7fd9463d0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 0x55bf146237c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.643+0000 7fd9463d0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bf14626b80 con 0x55bf1461a060 2024-05-08T06:03:00.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.643+0000 7fd9463d0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bf144d2b20 con 0x55bf1461a980 2024-05-08T06:03:00.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.643+0000 7fd9463d0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55bf14626d00 con 0x55bf1461e9d0 2024-05-08T06:03:00.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.643+0000 7fd9463d0b80 10 client.0 _ll_register_callbacks cb 0x55bf14513c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:03:00.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.644+0000 7fd933e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a980 0x55bf14623280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.644+0000 7fd939a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 0x55bf146237c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.644+0000 7fd933e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a980 0x55bf14623280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:37676/0 (socket says 172.21.15.112:37676) 2024-05-08T06:03:00.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.644+0000 7fd933e00700 1 -- 172.21.15.112:0/147771479 learned_addr learned my addr 172.21.15.112:0/147771479 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:00.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.644+0000 7fd933e00700 1 -- 172.21.15.112:0/147771479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 msgr2=0x55bf146237c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:00.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.644+0000 7fd933e00700 1 --2- 172.21.15.112:0/147771479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 0x55bf146237c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.644+0000 7fd933e00700 1 -- 172.21.15.112:0/147771479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a060 msgr2=0x55bf14625ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:00.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.644+0000 7fd933e00700 1 --2- 172.21.15.112:0/147771479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a060 0x55bf14625ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.644+0000 7fd933e00700 1 -- 172.21.15.112:0/147771479 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf14532420 con 0x55bf1461a980 2024-05-08T06:03:00.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.644+0000 7fd939a00700 1 --2- 172.21.15.112:0/147771479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 0x55bf146237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:03:00.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.644+0000 7fd933e00700 1 --2- 172.21.15.112:0/147771479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a980 0x55bf14623280 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd920009d10 tx=0x7fd920005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cad73bd0c5ed30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:00.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.645+0000 7fd91aa00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:03:00.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.645+0000 7fd91aa00700 1 -- 172.21.15.112:0/147771479 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920028070 con 0x55bf1461a980 2024-05-08T06:03:00.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.645+0000 7fd91aa00700 1 -- 172.21.15.112:0/147771479 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd92000b990 con 0x55bf1461a980 2024-05-08T06:03:00.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.645+0000 7fd9463d0b80 1 -- 172.21.15.112:0/147771479 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9200095d0 con 0x55bf1461a980 2024-05-08T06:03:00.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.645+0000 7fd91aa00700 1 -- 172.21.15.112:0/147771479 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd920005900 con 0x55bf1461a980 2024-05-08T06:03:00.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.645+0000 7fd9463d0b80 10 client.25952 fetch_fsmap learned FSMap version 33 2024-05-08T06:03:00.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.645+0000 7fd9463d0b80 1 -- 172.21.15.112:0/147771479 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bf14631480 con 0x55bf1461a980 2024-05-08T06:03:00.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.646+0000 7fd91aa00700 1 -- 172.21.15.112:0/147771479 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9200095d0 con 0x55bf1461a980 2024-05-08T06:03:00.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.646+0000 7fd91aa00700 1 -- 172.21.15.112:0/147771479 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7fd92000fc60 con 0x55bf1461a980 2024-05-08T06:03:00.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.646+0000 7fd9463d0b80 10 client.25952 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:03:00.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.646+0000 7fd9463d0b80 10 client.25952 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:03:00.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.646+0000 7fd9463d0b80 20 client.25952 populate_metadata read hostname 'smithi112' 2024-05-08T06:03:00.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.646+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/147771479 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55bf14633950 0x55bf14635d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.646+0000 7fd9463d0b80 4 client.25952 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:03:00.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.646+0000 7fd9463d0b80 1 -- 172.21.15.112:0/147771479 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf145e70a0 con 0x55bf14633950 2024-05-08T06:03:00.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.647+0000 7fd939000700 1 --2- 172.21.15.112:0/147771479 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55bf14633950 0x55bf14635d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.647+0000 7fd939000700 1 --2- 172.21.15.112:0/147771479 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55bf14633950 0x55bf14635d40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7b744d483360046 server_cookie=68e8f9040ac6bb03 in_seq=0 out_seq=0 2024-05-08T06:03:00.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.647+0000 7fd91aa00700 10 client.25952 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T06:03:00.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.648+0000 7fd91aa00700 1 -- 172.21.15.112:0/147771479 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bf145e70a0 con 0x55bf14633950 2024-05-08T06:03:00.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.648+0000 7fd91aa00700 10 client.25952 handle_command_reply: tid=0 2024-05-08T06:03:00.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.648+0000 7fd9463d0b80 1 client.25952 shutdown 2024-05-08T06:03:00.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.648+0000 7fd9463d0b80 1 -- 172.21.15.112:0/147771479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a980 msgr2=0x55bf14623280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:00.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.648+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/147771479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a980 0x55bf14623280 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd920009d10 tx=0x7fd920005280 comp rx=0 tx=0).stop 2024-05-08T06:03:00.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.648+0000 7fd9463d0b80 1 -- 172.21.15.112:0/147771479 shutdown_connections 2024-05-08T06:03:00.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.648+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/147771479 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55bf14633950 0x55bf14635d40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.648+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/147771479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 0x55bf146237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.648+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/147771479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a060 0x55bf14625ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.648+0000 7fd91aa00700 0 client.25952 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T06:03:00.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.648+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/147771479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a980 0x55bf14623280 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.648+0000 7fd9463d0b80 1 -- 172.21.15.112:0/147771479 >> 172.21.15.112:0/147771479 conn(0x55bf14548b60 msgr2=0x55bf1460f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:00.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.649+0000 7fd9463d0b80 1 -- 172.21.15.112:0/147771479 shutdown_connections 2024-05-08T06:03:00.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.649+0000 7fd9463d0b80 1 -- 172.21.15.112:0/147771479 wait complete. 2024-05-08T06:03:00.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.649+0000 7fd9463d0b80 20 client.25952 trim_cache size 0 max 16384 2024-05-08T06:03:00.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.666+0000 7fd9463d0b80 1 Processor -- start 2024-05-08T06:03:00.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.666+0000 7fd9463d0b80 1 -- start start 2024-05-08T06:03:00.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.666+0000 7fd9463d0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461a060 0x55bf14620ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.666+0000 7fd9463d0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a980 0x55bf14638d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.666+0000 7fd9463d0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461e9d0 0x55bf146392a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.666+0000 7fd9463d0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341b25a0 con 0x55bf1461a980 2024-05-08T06:03:00.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.666+0000 7fd9463d0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341afe30 con 0x55bf1461e9d0 2024-05-08T06:03:00.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.666+0000 7fd9463d0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341adc80 con 0x55bf1461a060 2024-05-08T06:03:00.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.667+0000 7fd939a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461e9d0 0x55bf146392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.667+0000 7fd939a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461e9d0 0x55bf146392a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:37680/0 (socket says 172.21.15.112:37680) 2024-05-08T06:03:00.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.667+0000 7fd939a00700 1 -- 172.21.15.112:0/1051882469 learned_addr learned my addr 172.21.15.112:0/1051882469 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:00.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.667+0000 7fd939000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461a060 0x55bf14620ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.667+0000 7fd933e00700 1 --2- 172.21.15.112:0/1051882469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a980 0x55bf14638d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.667+0000 7fd939a00700 1 -- 172.21.15.112:0/1051882469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461a060 msgr2=0x55bf14620ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:00.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.667+0000 7fd939a00700 1 --2- 172.21.15.112:0/1051882469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461a060 0x55bf14620ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.667+0000 7fd939a00700 1 -- 172.21.15.112:0/1051882469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a980 msgr2=0x55bf14638d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:00.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.667+0000 7fd939a00700 1 --2- 172.21.15.112:0/1051882469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a980 0x55bf14638d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.667+0000 7fd939a00700 1 -- 172.21.15.112:0/1051882469 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf146397e0 con 0x55bf1461e9d0 2024-05-08T06:03:00.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.667+0000 7fd939000700 1 --2- 172.21.15.112:0/1051882469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461a060 0x55bf14620ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:03:00.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.667+0000 7fd933e00700 1 --2- 172.21.15.112:0/1051882469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a980 0x55bf14638d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:03:00.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.667+0000 7fd939a00700 1 --2- 172.21.15.112:0/1051882469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461e9d0 0x55bf146392a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd92c017ab0 tx=0x7fd92c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dc73ae52ebbee0b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:00.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.668+0000 7fd925600700 1 -- 172.21.15.112:0/1051882469 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c028070 con 0x55bf1461e9d0 2024-05-08T06:03:00.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.668+0000 7fd925600700 1 -- 172.21.15.112:0/1051882469 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd92c016d20 con 0x55bf1461e9d0 2024-05-08T06:03:00.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.668+0000 7fd925600700 1 -- 172.21.15.112:0/1051882469 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c024d50 con 0x55bf1461e9d0 2024-05-08T06:03:00.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.669+0000 7fd9463d0b80 1 -- 172.21.15.112:0/1051882469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461e9d0 msgr2=0x55bf146392a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:00.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.669+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/1051882469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461e9d0 0x55bf146392a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd92c017ab0 tx=0x7fd92c004b40 comp rx=0 tx=0).stop 2024-05-08T06:03:00.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.669+0000 7fd9463d0b80 1 -- 172.21.15.112:0/1051882469 shutdown_connections 2024-05-08T06:03:00.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.669+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/1051882469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461e9d0 0x55bf146392a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.669+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/1051882469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461a060 0x55bf14620ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.669+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/1051882469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a980 0x55bf14638d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.669+0000 7fd9463d0b80 1 -- 172.21.15.112:0/1051882469 >> 172.21.15.112:0/1051882469 conn(0x55bf14548b60 msgr2=0x55bf1461f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:00.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.669+0000 7fd9463d0b80 1 -- 172.21.15.112:0/1051882469 shutdown_connections 2024-05-08T06:03:00.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.669+0000 7fd9463d0b80 1 -- 172.21.15.112:0/1051882469 wait complete. 2024-05-08T06:03:00.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.670+0000 7fd9463d0b80 1 -- start start 2024-05-08T06:03:00.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.670+0000 7fd9463d0b80 1 Processor -- start 2024-05-08T06:03:00.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.670+0000 7fd9463d0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a060 0x55bf14638ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.670+0000 7fd9463d0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a980 0x55bf146363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.670+0000 7fd9463d0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 0x55bf14636920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.670+0000 7fd9463d0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341ae2f0 con 0x55bf1461a980 2024-05-08T06:03:00.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.670+0000 7fd9463d0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341af760 con 0x55bf1461a060 2024-05-08T06:03:00.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.670+0000 7fd9463d0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341ae9c0 con 0x55bf1461e9d0 2024-05-08T06:03:00.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.670+0000 7fd9463d0b80 10 client.0 _ll_register_callbacks cb 0x55bf14513c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:03:00.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.671+0000 7fd933e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a980 0x55bf146363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.671+0000 7fd939000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a060 0x55bf14638ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.671+0000 7fd933e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a980 0x55bf146363e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58662/0 (socket says 172.21.15.112:58662) 2024-05-08T06:03:00.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.671+0000 7fd933e00700 1 -- 172.21.15.112:0/2934364863 learned_addr learned my addr 172.21.15.112:0/2934364863 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:00.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.671+0000 7fd939000700 1 -- 172.21.15.112:0/2934364863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 msgr2=0x55bf14636920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T06:03:00.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.671+0000 7fd939000700 1 --2- 172.21.15.112:0/2934364863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 0x55bf14636920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.671+0000 7fd939000700 1 -- 172.21.15.112:0/2934364863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a980 msgr2=0x55bf146363e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:00.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.671+0000 7fd939000700 1 --2- 172.21.15.112:0/2934364863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a980 0x55bf146363e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.671+0000 7fd939000700 1 -- 172.21.15.112:0/2934364863 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf14637070 con 0x55bf1461a060 2024-05-08T06:03:00.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.671+0000 7fd933e00700 1 --2- 172.21.15.112:0/2934364863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a980 0x55bf146363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:03:00.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.671+0000 7fd939000700 1 --2- 172.21.15.112:0/2934364863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a060 0x55bf14638ff0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd9280dff70 tx=0x7fd928010cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dba05594882b884 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:00.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.671+0000 7fd91aa00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:03:00.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.672+0000 7fd91aa00700 1 -- 172.21.15.112:0/2934364863 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9280b2040 con 0x55bf1461a060 2024-05-08T06:03:00.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.672+0000 7fd91aa00700 1 -- 172.21.15.112:0/2934364863 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd9280b2610 con 0x55bf1461a060 2024-05-08T06:03:00.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.672+0000 7fd9463d0b80 1 -- 172.21.15.112:0/2934364863 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd92c016d20 con 0x55bf1461a060 2024-05-08T06:03:00.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.672+0000 7fd91aa00700 1 -- 172.21.15.112:0/2934364863 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd9280a9390 con 0x55bf1461a060 2024-05-08T06:03:00.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.672+0000 7fd9463d0b80 10 client.25958 fetch_fsmap learned FSMap version 33 2024-05-08T06:03:00.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.672+0000 7fd9463d0b80 1 -- 172.21.15.112:0/2934364863 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bf1469de30 con 0x55bf1461a060 2024-05-08T06:03:00.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.672+0000 7fd91aa00700 1 -- 172.21.15.112:0/2934364863 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd92c016d20 con 0x55bf1461a060 2024-05-08T06:03:00.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.672+0000 7fd91aa00700 1 -- 172.21.15.112:0/2934364863 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7fd9280b9ce0 con 0x55bf1461a060 2024-05-08T06:03:00.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.672+0000 7fd9463d0b80 10 client.25958 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:03:00.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.672+0000 7fd9463d0b80 10 client.25958 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:03:00.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.672+0000 7fd9463d0b80 20 client.25958 populate_metadata read hostname 'smithi112' 2024-05-08T06:03:00.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.672+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/2934364863 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55bf1469e440 0x55bf146a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:00.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.672+0000 7fd9463d0b80 4 client.25958 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:03:00.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.672+0000 7fd9463d0b80 1 -- 172.21.15.112:0/2934364863 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf1457a400 con 0x55bf1469e440 2024-05-08T06:03:00.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.673+0000 7fd933e00700 1 --2- 172.21.15.112:0/2934364863 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55bf1469e440 0x55bf146a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:00.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.673+0000 7fd933e00700 1 --2- 172.21.15.112:0/2934364863 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55bf1469e440 0x55bf146a08b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d8db685cf5e172c server_cookie=f7b639ff5a5e5295 in_seq=0 out_seq=0 2024-05-08T06:03:00.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.673+0000 7fd91aa00700 10 client.25958 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T06:03:00.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.673+0000 7fd91aa00700 1 -- 172.21.15.112:0/2934364863 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf1457a400 con 0x55bf1469e440 2024-05-08T06:03:00.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.673+0000 7fd91aa00700 10 client.25958 handle_command_reply: tid=0 2024-05-08T06:03:00.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.674+0000 7fd9463d0b80 1 client.25958 shutdown 2024-05-08T06:03:00.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.674+0000 7fd9463d0b80 1 -- 172.21.15.112:0/2934364863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a060 msgr2=0x55bf14638ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:00.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.674+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/2934364863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a060 0x55bf14638ff0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd9280dff70 tx=0x7fd928010cb0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.674+0000 7fd9463d0b80 1 -- 172.21.15.112:0/2934364863 shutdown_connections 2024-05-08T06:03:00.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.674+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/2934364863 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x55bf1469e440 0x55bf146a08b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.674+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/2934364863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bf1461e9d0 0x55bf14636920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.674+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/2934364863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf1461a060 0x55bf14638ff0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.674+0000 7fd9463d0b80 1 --2- 172.21.15.112:0/2934364863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bf1461a980 0x55bf146363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.674+0000 7fd91aa00700 0 client.25958 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T06:03:00.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.674+0000 7fd9463d0b80 1 -- 172.21.15.112:0/2934364863 >> 172.21.15.112:0/2934364863 conn(0x55bf14548b60 msgr2=0x55bf14622420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:00.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.674+0000 7fd9463d0b80 1 -- 172.21.15.112:0/2934364863 shutdown_connections 2024-05-08T06:03:00.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.674+0000 7fd9463d0b80 1 -- 172.21.15.112:0/2934364863 wait complete. 2024-05-08T06:03:00.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.674+0000 7fd9463d0b80 20 client.25958 trim_cache size 0 max 16384 2024-05-08T06:03:00.677 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T06:03:00.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.675+0000 7fd93ba00700 1 -- 172.21.15.112:0/2468677434 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd91c0b2900 msgr2=0x7fd91c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:00.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.675+0000 7fd93ba00700 1 --2- 172.21.15.112:0/2468677434 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd91c0b2900 0x7fd91c0b4dc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009fd0 tx=0x7fd92c009040 comp rx=0 tx=0).stop 2024-05-08T06:03:00.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.675+0000 7fd93ba00700 1 -- 172.21.15.112:0/2468677434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd934103750 msgr2=0x7fd9341a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:00.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.675+0000 7fd93ba00700 1 --2- 172.21.15.112:0/2468677434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd934103750 0x7fd9341a1940 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd92800e9a0 tx=0x7fd92800ecb0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.675+0000 7fd93ba00700 1 -- 172.21.15.112:0/2468677434 shutdown_connections 2024-05-08T06:03:00.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.675+0000 7fd93ba00700 1 --2- 172.21.15.112:0/2468677434 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd91c0b2900 0x7fd91c0b4dc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.675+0000 7fd93ba00700 1 --2- 172.21.15.112:0/2468677434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd934108850 0x7fd93419bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.675+0000 7fd93ba00700 1 --2- 172.21.15.112:0/2468677434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd934103750 0x7fd9341a1940 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.675+0000 7fd93ba00700 1 --2- 172.21.15.112:0/2468677434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd934104100 0x7fd9341a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:00.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.675+0000 7fd93ba00700 1 -- 172.21.15.112:0/2468677434 >> 172.21.15.112:0/2468677434 conn(0x7fd93407a600 msgr2=0x7fd934101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:00.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.675+0000 7fd93ba00700 1 -- 172.21.15.112:0/2468677434 shutdown_connections 2024-05-08T06:03:00.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:00.675+0000 7fd93ba00700 1 -- 172.21.15.112:0/2468677434 wait complete. 2024-05-08T06:03:01.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:00 smithi150 ceph-mon[34315]: pgmap v672: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.0 MiB/s wr, 409 op/s 2024-05-08T06:03:01.206 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:00 smithi112 ceph-mon[30003]: pgmap v672: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.0 MiB/s wr, 409 op/s 2024-05-08T06:03:01.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:00 smithi162 ceph-mon[34800]: pgmap v672: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.0 MiB/s wr, 409 op/s 2024-05-08T06:03:01.755 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-08T06:03:02.219 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:03:02.951 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:02 smithi112 ceph-mon[30003]: pgmap v673: 97 pgs: 97 active+clean; 134 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.1 MiB/s wr, 455 op/s 2024-05-08T06:03:03.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:02 smithi150 ceph-mon[34315]: pgmap v673: 97 pgs: 97 active+clean; 134 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.1 MiB/s wr, 455 op/s 2024-05-08T06:03:03.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:02 smithi162 ceph-mon[34800]: pgmap v673: 97 pgs: 97 active+clean; 134 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.1 MiB/s wr, 455 op/s 2024-05-08T06:03:04.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.677+0000 7feb6be00700 1 -- 172.21.15.112:0/2825399247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb64101a90 msgr2=0x7feb64101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:04.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.677+0000 7feb6be00700 1 --2- 172.21.15.112:0/2825399247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb64101a90 0x7feb64101f10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7feb500098f0 tx=0x7feb50009c00 comp rx=0 tx=0).stop 2024-05-08T06:03:04.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.677+0000 7feb6be00700 1 -- 172.21.15.112:0/2825399247 shutdown_connections 2024-05-08T06:03:04.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.677+0000 7feb6be00700 1 --2- 172.21.15.112:0/2825399247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb64102530 0x7feb64112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.677+0000 7feb6be00700 1 --2- 172.21.15.112:0/2825399247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb641043b0 0x7feb64104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.677+0000 7feb6be00700 1 --2- 172.21.15.112:0/2825399247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb64101a90 0x7feb64101f10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.677+0000 7feb6be00700 1 -- 172.21.15.112:0/2825399247 >> 172.21.15.112:0/2825399247 conn(0x7feb64078a10 msgr2=0x7feb640ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:04.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.677+0000 7feb6be00700 1 -- 172.21.15.112:0/2825399247 shutdown_connections 2024-05-08T06:03:04.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.678+0000 7feb6be00700 1 -- 172.21.15.112:0/2825399247 wait complete. 2024-05-08T06:03:04.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.679+0000 7feb6be00700 1 Processor -- start 2024-05-08T06:03:04.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.679+0000 7feb6be00700 1 -- start start 2024-05-08T06:03:04.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.679+0000 7feb6be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb64101a90 0x7feb6419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.679+0000 7feb6be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb64102530 0x7feb6419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.679+0000 7feb6be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb641043b0 0x7feb641a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.679+0000 7feb6be00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb64115860 con 0x7feb641043b0 2024-05-08T06:03:04.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.679+0000 7feb6be00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feb641156e0 con 0x7feb64102530 2024-05-08T06:03:04.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.679+0000 7feb6be00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7feb641159e0 con 0x7feb64101a90 2024-05-08T06:03:04.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.680+0000 7feb68a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb64102530 0x7feb6419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.680+0000 7feb69400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb64101a90 0x7feb6419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.680+0000 7feb68a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb64102530 0x7feb6419c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:39500/0 (socket says 172.21.15.112:39500) 2024-05-08T06:03:04.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.680+0000 7feb68a00700 1 -- 172.21.15.112:0/2901517468 learned_addr learned my addr 172.21.15.112:0/2901517468 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:04.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.680+0000 7feb69e00700 1 --2- 172.21.15.112:0/2901517468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb641043b0 0x7feb641a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.680+0000 7feb68a00700 1 -- 172.21.15.112:0/2901517468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb64101a90 msgr2=0x7feb6419be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:04.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.680+0000 7feb68a00700 1 --2- 172.21.15.112:0/2901517468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb64101a90 0x7feb6419be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.680+0000 7feb68a00700 1 -- 172.21.15.112:0/2901517468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb641043b0 msgr2=0x7feb641a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:04.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.680+0000 7feb68a00700 1 --2- 172.21.15.112:0/2901517468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb641043b0 0x7feb641a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.680+0000 7feb68a00700 1 -- 172.21.15.112:0/2901517468 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb641a0db0 con 0x7feb64102530 2024-05-08T06:03:04.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.680+0000 7feb68a00700 1 --2- 172.21.15.112:0/2901517468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb64102530 0x7feb6419c380 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7feb5000f990 tx=0x7feb5000fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:04.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.681+0000 7feb62000700 1 -- 172.21.15.112:0/2901517468 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50005400 con 0x7feb64102530 2024-05-08T06:03:04.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.681+0000 7feb6be00700 1 -- 172.21.15.112:0/2901517468 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb641a1040 con 0x7feb64102530 2024-05-08T06:03:04.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.681+0000 7feb62000700 1 -- 172.21.15.112:0/2901517468 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb500055a0 con 0x7feb64102530 2024-05-08T06:03:04.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.681+0000 7feb62000700 1 -- 172.21.15.112:0/2901517468 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50004dc0 con 0x7feb64102530 2024-05-08T06:03:04.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.681+0000 7feb6be00700 1 -- 172.21.15.112:0/2901517468 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb641a8920 con 0x7feb64102530 2024-05-08T06:03:04.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.688+0000 7feb62000700 1 -- 172.21.15.112:0/2901517468 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7feb50029400 con 0x7feb64102530 2024-05-08T06:03:04.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.689+0000 7feb767e9b80 1 Processor -- start 2024-05-08T06:03:04.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.689+0000 7feb767e9b80 1 -- start start 2024-05-08T06:03:04.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.689+0000 7feb767e9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7060 0x55a2d3db7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.689+0000 7feb767e9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3db7980 0x55a2d3dbb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.689+0000 7feb767e9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3dbbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.689+0000 7feb767e9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a2d3d09a20 con 0x55a2d3db7060 2024-05-08T06:03:04.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.689+0000 7feb767e9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a2d3ca7760 con 0x55a2d3db7980 2024-05-08T06:03:04.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.689+0000 7feb767e9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a2d3d059e0 con 0x55a2d3dbb9d0 2024-05-08T06:03:04.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.690+0000 7feb69400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3db7980 0x55a2d3dbb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.690+0000 7feb68a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3dbbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.690+0000 7feb69400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3db7980 0x55a2d3dbb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:39516/0 (socket says 172.21.15.112:39516) 2024-05-08T06:03:04.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.690+0000 7feb69e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7060 0x55a2d3db7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.690+0000 7feb69400700 1 -- 172.21.15.112:0/1037845248 learned_addr learned my addr 172.21.15.112:0/1037845248 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:04.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.690+0000 7feb69400700 1 -- 172.21.15.112:0/1037845248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3dbb9d0 msgr2=0x55a2d3dbbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:04.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.690+0000 7feb69400700 1 --2- 172.21.15.112:0/1037845248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3dbbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.690+0000 7feb69400700 1 -- 172.21.15.112:0/1037845248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7060 msgr2=0x55a2d3db7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:04.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.690+0000 7feb69400700 1 --2- 172.21.15.112:0/1037845248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7060 0x55a2d3db7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.690+0000 7feb69400700 1 -- 172.21.15.112:0/1037845248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d3c568d0 con 0x55a2d3db7980 2024-05-08T06:03:04.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.691+0000 7feb62000700 1 --2- 172.21.15.112:0/2901517468 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feb4c0b2900 0x7feb4c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.691+0000 7feb69400700 1 --2- 172.21.15.112:0/1037845248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3db7980 0x55a2d3dbb490 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feb58009cf0 tx=0x7feb58011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=209714babb481223 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:04.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.691+0000 7feb62000700 1 -- 172.21.15.112:0/2901517468 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7feb500e4210 con 0x7feb64102530 2024-05-08T06:03:04.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.691+0000 7feb56a00700 1 -- 172.21.15.112:0/1037845248 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58011d80 con 0x55a2d3db7980 2024-05-08T06:03:04.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.691+0000 7feb56a00700 1 -- 172.21.15.112:0/1037845248 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb58004500 con 0x55a2d3db7980 2024-05-08T06:03:04.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.691+0000 7feb56a00700 1 -- 172.21.15.112:0/1037845248 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58005280 con 0x55a2d3db7980 2024-05-08T06:03:04.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.692+0000 7feb69e00700 1 --2- 172.21.15.112:0/2901517468 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feb4c0b2900 0x7feb4c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.692+0000 7feb767e9b80 1 -- 172.21.15.112:0/1037845248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3db7980 msgr2=0x55a2d3dbb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:04.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.692+0000 7feb767e9b80 1 --2- 172.21.15.112:0/1037845248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3db7980 0x55a2d3dbb490 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feb58009cf0 tx=0x7feb58011440 comp rx=0 tx=0).stop 2024-05-08T06:03:04.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.692+0000 7feb767e9b80 1 -- 172.21.15.112:0/1037845248 shutdown_connections 2024-05-08T06:03:04.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.692+0000 7feb767e9b80 1 --2- 172.21.15.112:0/1037845248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3dbbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.692+0000 7feb767e9b80 1 --2- 172.21.15.112:0/1037845248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7060 0x55a2d3db7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.692+0000 7feb767e9b80 1 --2- 172.21.15.112:0/1037845248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3db7980 0x55a2d3dbb490 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.692+0000 7feb767e9b80 1 -- 172.21.15.112:0/1037845248 >> 172.21.15.112:0/1037845248 conn(0x55a2d3ce5b60 msgr2=0x55a2d3db17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:04.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.692+0000 7feb767e9b80 1 -- 172.21.15.112:0/1037845248 shutdown_connections 2024-05-08T06:03:04.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.692+0000 7feb767e9b80 1 -- 172.21.15.112:0/1037845248 wait complete. 2024-05-08T06:03:04.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.693+0000 7feb69e00700 1 --2- 172.21.15.112:0/2901517468 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feb4c0b2900 0x7feb4c0b4dc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feb5c007550 tx=0x7feb5c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:04.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.693+0000 7feb767e9b80 1 -- start start 2024-05-08T06:03:04.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.693+0000 7feb767e9b80 1 Processor -- start 2024-05-08T06:03:04.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.694+0000 7feb767e9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3db7060 0x55a2d3dc2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.694+0000 7feb767e9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7980 0x55a2d3dc0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.694+0000 7feb767e9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3dc0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.694+0000 7feb767e9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a2d3dc3be0 con 0x55a2d3dbb9d0 2024-05-08T06:03:04.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.694+0000 7feb767e9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a2d3c6fb20 con 0x55a2d3db7060 2024-05-08T06:03:04.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.694+0000 7feb767e9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a2d3dc3d60 con 0x55a2d3db7980 2024-05-08T06:03:04.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.694+0000 7feb767e9b80 10 client.0 _ll_register_callbacks cb 0x55a2d3cb0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:03:04.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.694+0000 7feb68a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7980 0x55a2d3dc0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.694+0000 7feb69400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3db7060 0x55a2d3dc2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.694+0000 7feb69400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3db7060 0x55a2d3dc2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:39526/0 (socket says 172.21.15.112:39526) 2024-05-08T06:03:04.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.694+0000 7feb69e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3dc0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.694+0000 7feb69400700 1 -- 172.21.15.112:0/2533099387 learned_addr learned my addr 172.21.15.112:0/2533099387 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:04.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.695+0000 7feb69400700 1 -- 172.21.15.112:0/2533099387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7980 msgr2=0x55a2d3dc0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:04.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.695+0000 7feb69400700 1 --2- 172.21.15.112:0/2533099387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7980 0x55a2d3dc0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.695+0000 7feb69400700 1 -- 172.21.15.112:0/2533099387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3dbb9d0 msgr2=0x55a2d3dc0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:04.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.695+0000 7feb69400700 1 --2- 172.21.15.112:0/2533099387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3dc0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.695+0000 7feb69400700 1 -- 172.21.15.112:0/2533099387 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d3ccf420 con 0x55a2d3db7060 2024-05-08T06:03:04.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.695+0000 7feb68a00700 1 --2- 172.21.15.112:0/2533099387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7980 0x55a2d3dc0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:03:04.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.695+0000 7feb69400700 1 --2- 172.21.15.112:0/2533099387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3db7060 0x55a2d3dc2db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feb58004e40 tx=0x7feb580040e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=847a7c48f9039b89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:04.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.695+0000 7feb4b400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:03:04.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.695+0000 7feb4b400700 1 -- 172.21.15.112:0/2533099387 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58026020 con 0x55a2d3db7060 2024-05-08T06:03:04.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.696+0000 7feb4b400700 1 -- 172.21.15.112:0/2533099387 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7feb58004750 con 0x55a2d3db7060 2024-05-08T06:03:04.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.696+0000 7feb767e9b80 1 -- 172.21.15.112:0/2533099387 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb58004500 con 0x55a2d3db7060 2024-05-08T06:03:04.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.696+0000 7feb4b400700 1 -- 172.21.15.112:0/2533099387 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7feb58004950 con 0x55a2d3db7060 2024-05-08T06:03:04.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.696+0000 7feb767e9b80 10 client.25970 fetch_fsmap learned FSMap version 33 2024-05-08T06:03:04.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.696+0000 7feb767e9b80 1 -- 172.21.15.112:0/2533099387 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a2d3dce430 con 0x55a2d3db7060 2024-05-08T06:03:04.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.696+0000 7feb4b400700 1 -- 172.21.15.112:0/2533099387 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb58004500 con 0x55a2d3db7060 2024-05-08T06:03:04.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.696+0000 7feb4b400700 1 -- 172.21.15.112:0/2533099387 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7feb58018070 con 0x55a2d3db7060 2024-05-08T06:03:04.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.697+0000 7feb767e9b80 10 client.25970 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:03:04.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.697+0000 7feb767e9b80 10 client.25970 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T06:03:04.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.697+0000 7feb767e9b80 20 client.25970 populate_metadata read hostname 'smithi112' 2024-05-08T06:03:04.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.697+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2533099387 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55a2d3dd0900 0x55a2d3dd2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.697+0000 7feb767e9b80 4 client.25970 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.697+0000 7feb767e9b80 1 -- 172.21.15.112:0/2533099387 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2d3d840a0 con 0x55a2d3dd0900 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.697+0000 7feb68a00700 1 --2- 172.21.15.112:0/2533099387 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55a2d3dd0900 0x55a2d3dd2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.697+0000 7feb68a00700 1 --2- 172.21.15.112:0/2533099387 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55a2d3dd0900 0x55a2d3dd2cf0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72b1a8b590b162db server_cookie=4498ee651633e528 in_seq=0 out_seq=0 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.697+0000 7feb4b400700 10 client.25970 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.698+0000 7feb4b400700 1 -- 172.21.15.112:0/2533099387 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a2d3d840a0 con 0x55a2d3dd0900 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.698+0000 7feb4b400700 10 client.25970 handle_command_reply: tid=0 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.698+0000 7feb767e9b80 1 client.25970 shutdown 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.698+0000 7feb767e9b80 1 -- 172.21.15.112:0/2533099387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3db7060 msgr2=0x55a2d3dc2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.698+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2533099387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3db7060 0x55a2d3dc2db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feb58004e40 tx=0x7feb580040e0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.698+0000 7feb767e9b80 1 -- 172.21.15.112:0/2533099387 shutdown_connections 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.698+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2533099387 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55a2d3dd0900 0x55a2d3dd2cf0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.699+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2533099387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3dc0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.699+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2533099387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3db7060 0x55a2d3dc2db0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.699+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2533099387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7980 0x55a2d3dc0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.699+0000 7feb4b400700 0 client.25970 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T06:03:04.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.699+0000 7feb767e9b80 1 -- 172.21.15.112:0/2533099387 >> 172.21.15.112:0/2533099387 conn(0x55a2d3ce5b60 msgr2=0x55a2d3dac2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:04.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.699+0000 7feb767e9b80 1 -- 172.21.15.112:0/2533099387 shutdown_connections 2024-05-08T06:03:04.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.699+0000 7feb767e9b80 1 -- 172.21.15.112:0/2533099387 wait complete. 2024-05-08T06:03:04.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.699+0000 7feb767e9b80 20 client.25970 trim_cache size 0 max 16384 2024-05-08T06:03:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.717+0000 7feb767e9b80 1 Processor -- start 2024-05-08T06:03:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.717+0000 7feb767e9b80 1 -- start start 2024-05-08T06:03:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.718+0000 7feb767e9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7060 0x55a2d3dbddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.718+0000 7feb767e9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7980 0x55a2d3dccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.718+0000 7feb767e9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3dd84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.718+0000 7feb767e9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb4400fcf0 con 0x55a2d3db7980 2024-05-08T06:03:04.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.718+0000 7feb767e9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feb4400d580 con 0x55a2d3dbb9d0 2024-05-08T06:03:04.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.718+0000 7feb767e9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7feb4400b3d0 con 0x55a2d3db7060 2024-05-08T06:03:04.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.718+0000 7feb69400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7060 0x55a2d3dbddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.718+0000 7feb69400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7060 0x55a2d3dbddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60716/0 (socket says 172.21.15.112:60716) 2024-05-08T06:03:04.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.718+0000 7feb69400700 1 -- 172.21.15.112:0/2804282707 learned_addr learned my addr 172.21.15.112:0/2804282707 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:04.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.718+0000 7feb68a00700 1 --2- 172.21.15.112:0/2804282707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7980 0x55a2d3dccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.718+0000 7feb69e00700 1 --2- 172.21.15.112:0/2804282707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3dd84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.719+0000 7feb69400700 1 -- 172.21.15.112:0/2804282707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3dbb9d0 msgr2=0x55a2d3dd84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:04.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.719+0000 7feb69400700 1 --2- 172.21.15.112:0/2804282707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3dd84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.719+0000 7feb69400700 1 -- 172.21.15.112:0/2804282707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7980 msgr2=0x55a2d3dccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:04.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.719+0000 7feb69400700 1 --2- 172.21.15.112:0/2804282707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7980 0x55a2d3dccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.719+0000 7feb69400700 1 -- 172.21.15.112:0/2804282707 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d3dc09a0 con 0x55a2d3db7060 2024-05-08T06:03:04.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.719+0000 7feb69400700 1 --2- 172.21.15.112:0/2804282707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7060 0x55a2d3dbddc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7feb58006010 tx=0x7feb58011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef1e35ee14582835 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:04.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.719+0000 7feb56000700 1 -- 172.21.15.112:0/2804282707 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58026040 con 0x55a2d3db7060 2024-05-08T06:03:04.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.719+0000 7feb56000700 1 -- 172.21.15.112:0/2804282707 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb58009770 con 0x55a2d3db7060 2024-05-08T06:03:04.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.719+0000 7feb56000700 1 -- 172.21.15.112:0/2804282707 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb580044e0 con 0x55a2d3db7060 2024-05-08T06:03:04.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.720+0000 7feb767e9b80 1 -- 172.21.15.112:0/2804282707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7060 msgr2=0x55a2d3dbddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:04.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.720+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2804282707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7060 0x55a2d3dbddc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7feb58006010 tx=0x7feb58011440 comp rx=0 tx=0).stop 2024-05-08T06:03:04.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.720+0000 7feb767e9b80 1 -- 172.21.15.112:0/2804282707 shutdown_connections 2024-05-08T06:03:04.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.720+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2804282707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3dd84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.720+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2804282707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7060 0x55a2d3dbddc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.720+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2804282707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7980 0x55a2d3dccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.720+0000 7feb767e9b80 1 -- 172.21.15.112:0/2804282707 >> 172.21.15.112:0/2804282707 conn(0x55a2d3ce5b60 msgr2=0x55a2d3dbd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:04.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.720+0000 7feb767e9b80 1 -- 172.21.15.112:0/2804282707 shutdown_connections 2024-05-08T06:03:04.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.721+0000 7feb767e9b80 1 -- 172.21.15.112:0/2804282707 wait complete. 2024-05-08T06:03:04.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.721+0000 7feb767e9b80 1 -- start start 2024-05-08T06:03:04.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.721+0000 7feb767e9b80 1 Processor -- start 2024-05-08T06:03:04.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.722+0000 7feb767e9b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7060 0x55a2d3dc94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.722+0000 7feb767e9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7980 0x55a2d3db9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.722+0000 7feb767e9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3db9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.722+0000 7feb767e9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb4400ba40 con 0x55a2d3db7980 2024-05-08T06:03:04.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.722+0000 7feb767e9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feb4400ceb0 con 0x55a2d3dbb9d0 2024-05-08T06:03:04.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.722+0000 7feb767e9b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7feb4400c110 con 0x55a2d3db7060 2024-05-08T06:03:04.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.722+0000 7feb767e9b80 10 client.0 _ll_register_callbacks cb 0x55a2d3cb0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:03:04.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.722+0000 7feb69400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7060 0x55a2d3dc94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.722+0000 7feb68a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7980 0x55a2d3db9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.722+0000 7feb69400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7060 0x55a2d3dc94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60730/0 (socket says 172.21.15.112:60730) 2024-05-08T06:03:04.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.722+0000 7feb69400700 1 -- 172.21.15.112:0/2323317462 learned_addr learned my addr 172.21.15.112:0/2323317462 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:04.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.722+0000 7feb69e00700 1 --2- 172.21.15.112:0/2323317462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3db9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.723+0000 7feb68a00700 1 -- 172.21.15.112:0/2323317462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7060 msgr2=0x55a2d3dc94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:04.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.723+0000 7feb68a00700 1 --2- 172.21.15.112:0/2323317462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7060 0x55a2d3dc94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.723+0000 7feb68a00700 1 -- 172.21.15.112:0/2323317462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3dbb9d0 msgr2=0x55a2d3db9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:04.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.723+0000 7feb68a00700 1 --2- 172.21.15.112:0/2323317462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3db9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.723+0000 7feb68a00700 1 -- 172.21.15.112:0/2323317462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d3dd7a50 con 0x55a2d3db7980 2024-05-08T06:03:04.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.723+0000 7feb69400700 1 --2- 172.21.15.112:0/2323317462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7060 0x55a2d3dc94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:03:04.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.723+0000 7feb69e00700 1 --2- 172.21.15.112:0/2323317462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3db9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:03:04.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.723+0000 7feb68a00700 1 --2- 172.21.15.112:0/2323317462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7980 0x55a2d3db9410 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feb50020000 tx=0x7feb5001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670a35208224aed5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:04.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.723+0000 7feb4b400700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T06:03:04.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.723+0000 7feb4b400700 1 -- 172.21.15.112:0/2323317462 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb500aafa0 con 0x55a2d3db7980 2024-05-08T06:03:04.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.723+0000 7feb4b400700 1 -- 172.21.15.112:0/2323317462 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7feb500e1d90 con 0x55a2d3db7980 2024-05-08T06:03:04.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.723+0000 7feb767e9b80 1 -- 172.21.15.112:0/2323317462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb58009770 con 0x55a2d3db7980 2024-05-08T06:03:04.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.723+0000 7feb4b400700 1 -- 172.21.15.112:0/2323317462 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7feb500ce0f0 con 0x55a2d3db7980 2024-05-08T06:03:04.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.723+0000 7feb767e9b80 10 client.15795 fetch_fsmap learned FSMap version 33 2024-05-08T06:03:04.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.723+0000 7feb767e9b80 1 -- 172.21.15.112:0/2323317462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a2d3e3b1e0 con 0x55a2d3db7980 2024-05-08T06:03:04.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.724+0000 7feb4b400700 1 -- 172.21.15.112:0/2323317462 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb58009770 con 0x55a2d3db7980 2024-05-08T06:03:04.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.724+0000 7feb4b400700 1 -- 172.21.15.112:0/2323317462 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7feb5001f070 con 0x55a2d3db7980 2024-05-08T06:03:04.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.724+0000 7feb767e9b80 10 client.15795 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:03:04.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.724+0000 7feb767e9b80 10 client.15795 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T06:03:04.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.724+0000 7feb767e9b80 20 client.15795 populate_metadata read hostname 'smithi112' 2024-05-08T06:03:04.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.724+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2323317462 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55a2d3e3b8a0 0x55a2d3e3dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.724+0000 7feb767e9b80 4 client.15795 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:03:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.724+0000 7feb767e9b80 1 -- 172.21.15.112:0/2323317462 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2d3d17400 con 0x55a2d3e3b8a0 2024-05-08T06:03:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.724+0000 7feb69400700 1 --2- 172.21.15.112:0/2323317462 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55a2d3e3b8a0 0x55a2d3e3dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.725+0000 7feb69400700 1 --2- 172.21.15.112:0/2323317462 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55a2d3e3b8a0 0x55a2d3e3dcc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=239beb2814681963 server_cookie=ca5794d24e7bdabe in_seq=0 out_seq=0 2024-05-08T06:03:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.725+0000 7feb4b400700 10 client.15795 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T06:03:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.725+0000 7feb4b400700 1 -- 172.21.15.112:0/2323317462 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2d3d17400 con 0x55a2d3e3b8a0 2024-05-08T06:03:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.725+0000 7feb4b400700 10 client.15795 handle_command_reply: tid=0 2024-05-08T06:03:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.725+0000 7feb767e9b80 1 client.15795 shutdown 2024-05-08T06:03:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.725+0000 7feb767e9b80 1 -- 172.21.15.112:0/2323317462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7980 msgr2=0x55a2d3db9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.725+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2323317462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7980 0x55a2d3db9410 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feb50020000 tx=0x7feb5001e000 comp rx=0 tx=0).stop 2024-05-08T06:03:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.726+0000 7feb767e9b80 1 -- 172.21.15.112:0/2323317462 shutdown_connections 2024-05-08T06:03:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.726+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2323317462 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55a2d3e3b8a0 0x55a2d3e3dcc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.726+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2323317462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a2d3dbb9d0 0x55a2d3db9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.726+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2323317462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2d3db7060 0x55a2d3dc94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.726+0000 7feb767e9b80 1 --2- 172.21.15.112:0/2323317462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2d3db7980 0x55a2d3db9410 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.726+0000 7feb4b400700 0 client.15795 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T06:03:04.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.726+0000 7feb767e9b80 1 -- 172.21.15.112:0/2323317462 >> 172.21.15.112:0/2323317462 conn(0x55a2d3ce5b60 msgr2=0x55a2d3dbc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:04.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.726+0000 7feb767e9b80 1 -- 172.21.15.112:0/2323317462 shutdown_connections 2024-05-08T06:03:04.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.726+0000 7feb767e9b80 1 -- 172.21.15.112:0/2323317462 wait complete. 2024-05-08T06:03:04.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.726+0000 7feb767e9b80 20 client.15795 trim_cache size 0 max 16384 2024-05-08T06:03:04.728 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T06:03:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.727+0000 7feb6be00700 1 -- 172.21.15.112:0/2901517468 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feb4c0b2900 msgr2=0x7feb4c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.727+0000 7feb6be00700 1 --2- 172.21.15.112:0/2901517468 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feb4c0b2900 0x7feb4c0b4dc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feb5c007550 tx=0x7feb5c009040 comp rx=0 tx=0).stop 2024-05-08T06:03:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.727+0000 7feb6be00700 1 -- 172.21.15.112:0/2901517468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb64102530 msgr2=0x7feb6419c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.727+0000 7feb6be00700 1 --2- 172.21.15.112:0/2901517468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb64102530 0x7feb6419c380 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7feb5000f990 tx=0x7feb5000fa70 comp rx=0 tx=0).stop 2024-05-08T06:03:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.727+0000 7feb6be00700 1 -- 172.21.15.112:0/2901517468 shutdown_connections 2024-05-08T06:03:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.727+0000 7feb6be00700 1 --2- 172.21.15.112:0/2901517468 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7feb4c0b2900 0x7feb4c0b4dc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.727+0000 7feb6be00700 1 --2- 172.21.15.112:0/2901517468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb641043b0 0x7feb641a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.727+0000 7feb6be00700 1 --2- 172.21.15.112:0/2901517468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb64101a90 0x7feb6419be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.727+0000 7feb6be00700 1 --2- 172.21.15.112:0/2901517468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb64102530 0x7feb6419c380 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:04.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.727+0000 7feb6be00700 1 -- 172.21.15.112:0/2901517468 >> 172.21.15.112:0/2901517468 conn(0x7feb64078a10 msgr2=0x7feb640ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.727+0000 7feb6be00700 1 -- 172.21.15.112:0/2901517468 shutdown_connections 2024-05-08T06:03:04.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:04.727+0000 7feb6be00700 1 -- 172.21.15.112:0/2901517468 wait complete. 2024-05-08T06:03:05.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:04 smithi112 ceph-mon[30003]: pgmap v674: 97 pgs: 97 active+clean; 132 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 944 KiB/s wr, 370 op/s 2024-05-08T06:03:05.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:04 smithi150 ceph-mon[34315]: pgmap v674: 97 pgs: 97 active+clean; 132 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 944 KiB/s wr, 370 op/s 2024-05-08T06:03:05.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:04 smithi162 ceph-mon[34800]: pgmap v674: 97 pgs: 97 active+clean; 132 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 944 KiB/s wr, 370 op/s 2024-05-08T06:03:05.824 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-08T06:03:06.320 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:03:07.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:06 smithi150 ceph-mon[34315]: pgmap v675: 97 pgs: 97 active+clean; 144 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.2 MiB/s wr, 476 op/s 2024-05-08T06:03:07.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:06 smithi112 ceph-mon[30003]: pgmap v675: 97 pgs: 97 active+clean; 144 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.2 MiB/s wr, 476 op/s 2024-05-08T06:03:07.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:06 smithi162 ceph-mon[34800]: pgmap v675: 97 pgs: 97 active+clean; 144 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.2 MiB/s wr, 476 op/s 2024-05-08T06:03:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.639+0000 7fb189600700 1 -- 172.21.15.112:0/727096888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb184108850 msgr2=0x7fb1841103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.639+0000 7fb189600700 1 --2- 172.21.15.112:0/727096888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb184108850 0x7fb1841103d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00b0d0 tx=0x7fb17c00b3e0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.639+0000 7fb189600700 1 -- 172.21.15.112:0/727096888 shutdown_connections 2024-05-08T06:03:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.639+0000 7fb189600700 1 --2- 172.21.15.112:0/727096888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb184108850 0x7fb1841103d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.639+0000 7fb189600700 1 --2- 172.21.15.112:0/727096888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb184103750 0x7fb184103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.639+0000 7fb189600700 1 --2- 172.21.15.112:0/727096888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb184104100 0x7fb184108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.639+0000 7fb189600700 1 -- 172.21.15.112:0/727096888 >> 172.21.15.112:0/727096888 conn(0x7fb18407a600 msgr2=0x7fb18407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.639+0000 7fb189600700 1 -- 172.21.15.112:0/727096888 shutdown_connections 2024-05-08T06:03:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.639+0000 7fb189600700 1 -- 172.21.15.112:0/727096888 wait complete. 2024-05-08T06:03:08.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.640+0000 7fb189600700 1 Processor -- start 2024-05-08T06:03:08.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.640+0000 7fb189600700 1 -- start start 2024-05-08T06:03:08.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.641+0000 7fb189600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb184103750 0x7fb1841a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.641+0000 7fb189600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb184104100 0x7fb1841a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.641+0000 7fb189600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb184108850 0x7fb18419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.641+0000 7fb189600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb184113690 con 0x7fb184103750 2024-05-08T06:03:08.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.641+0000 7fb189600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb184113510 con 0x7fb184104100 2024-05-08T06:03:08.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.641+0000 7fb189600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb184113810 con 0x7fb184108850 2024-05-08T06:03:08.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.641+0000 7fb183400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb184108850 0x7fb18419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:08.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.641+0000 7fb182a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb184103750 0x7fb1841a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:08.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.641+0000 7fb183400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb184108850 0x7fb18419bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60740/0 (socket says 172.21.15.112:60740) 2024-05-08T06:03:08.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.641+0000 7fb183400700 1 -- 172.21.15.112:0/3320480835 learned_addr learned my addr 172.21.15.112:0/3320480835 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:08.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.641+0000 7fb182000700 1 --2- 172.21.15.112:0/3320480835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb184104100 0x7fb1841a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:08.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.642+0000 7fb183400700 1 -- 172.21.15.112:0/3320480835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb184104100 msgr2=0x7fb1841a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:08.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.642+0000 7fb183400700 1 --2- 172.21.15.112:0/3320480835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb184104100 0x7fb1841a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.642+0000 7fb183400700 1 -- 172.21.15.112:0/3320480835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb184103750 msgr2=0x7fb1841a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:08.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.642+0000 7fb183400700 1 --2- 172.21.15.112:0/3320480835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb184103750 0x7fb1841a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.642+0000 7fb183400700 1 -- 172.21.15.112:0/3320480835 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18419c240 con 0x7fb184108850 2024-05-08T06:03:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.642+0000 7fb183400700 1 --2- 172.21.15.112:0/3320480835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb184108850 0x7fb18419bae0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0079a0 tx=0x7fb17c0079d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.642+0000 7fb17b400700 1 -- 172.21.15.112:0/3320480835 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c01f070 con 0x7fb184108850 2024-05-08T06:03:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.643+0000 7fb17b400700 1 -- 172.21.15.112:0/3320480835 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb17c003c00 con 0x7fb184108850 2024-05-08T06:03:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.643+0000 7fb17b400700 1 -- 172.21.15.112:0/3320480835 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c01b430 con 0x7fb184108850 2024-05-08T06:03:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.643+0000 7fb189600700 1 -- 172.21.15.112:0/3320480835 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18419c4d0 con 0x7fb184108850 2024-05-08T06:03:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.643+0000 7fb189600700 1 -- 172.21.15.112:0/3320480835 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1841a87f0 con 0x7fb184108850 2024-05-08T06:03:08.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.646+0000 7fb17b400700 1 -- 172.21.15.112:0/3320480835 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb17c022070 con 0x7fb184108850 2024-05-08T06:03:08.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.649+0000 7fb193eb0b80 1 Processor -- start 2024-05-08T06:03:08.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.650+0000 7fb193eb0b80 1 -- start start 2024-05-08T06:03:08.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.650+0000 7fb193eb0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972060 0x5641b6972440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.650+0000 7fb193eb0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b6972980 0x5641b6976490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.650+0000 7fb193eb0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b69769d0 0x5641b6976e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.650+0000 7fb193eb0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5641b68c4a20 con 0x5641b6972980 2024-05-08T06:03:08.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.650+0000 7fb193eb0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5641b6862760 con 0x5641b69769d0 2024-05-08T06:03:08.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.650+0000 7fb193eb0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5641b68c09e0 con 0x5641b6972060 2024-05-08T06:03:08.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.651+0000 7fb182000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b69769d0 0x5641b6976e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:08.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.651+0000 7fb182000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b69769d0 0x5641b6976e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:39574/0 (socket says 172.21.15.112:39574) 2024-05-08T06:03:08.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.651+0000 7fb182000700 1 -- 172.21.15.112:0/1261217414 learned_addr learned my addr 172.21.15.112:0/1261217414 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:08.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.651+0000 7fb182000700 1 --2- 172.21.15.112:0/1261217414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972060 0x5641b6972440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:08.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.651+0000 7fb17b400700 1 --2- 172.21.15.112:0/3320480835 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb1680b2980 0x7fb1680b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.651+0000 7fb182000700 1 -- 172.21.15.112:0/1261217414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972060 msgr2=0x5641b6972440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.651+0000 7fb182000700 1 --2- 172.21.15.112:0/1261217414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972060 0x5641b6972440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.651+0000 7fb182000700 1 -- 172.21.15.112:0/1261217414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b6972980 msgr2=0x5641b6976490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.651+0000 7fb182000700 1 --2- 172.21.15.112:0/1261217414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b6972980 0x5641b6976490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.651+0000 7fb182000700 1 -- 172.21.15.112:0/1261217414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b68118d0 con 0x5641b69769d0 2024-05-08T06:03:08.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.652+0000 7fb17b400700 1 -- 172.21.15.112:0/3320480835 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fb17c0e30d0 con 0x7fb184108850 2024-05-08T06:03:08.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.652+0000 7fb182000700 1 --2- 172.21.15.112:0/1261217414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b69769d0 0x5641b6976e30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb170011880 tx=0x7fb170009860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e445df1bbd7653f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:08.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.652+0000 7fb183400700 1 --2- 172.21.15.112:0/3320480835 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb1680b2980 0x7fb1680b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:08.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.652+0000 7fb16fe00700 1 -- 172.21.15.112:0/1261217414 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb170004030 con 0x5641b69769d0 2024-05-08T06:03:08.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.652+0000 7fb16fe00700 1 -- 172.21.15.112:0/1261217414 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb170003680 con 0x5641b69769d0 2024-05-08T06:03:08.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.652+0000 7fb16fe00700 1 -- 172.21.15.112:0/1261217414 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17001e600 con 0x5641b69769d0 2024-05-08T06:03:08.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.653+0000 7fb183400700 1 --2- 172.21.15.112:0/3320480835 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb1680b2980 0x7fb1680b4e40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0a6770 tx=0x7fb17c0095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:08.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.654+0000 7fb193eb0b80 1 -- 172.21.15.112:0/1261217414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b69769d0 msgr2=0x5641b6976e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:08.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.654+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/1261217414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b69769d0 0x5641b6976e30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb170011880 tx=0x7fb170009860 comp rx=0 tx=0).stop 2024-05-08T06:03:08.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.654+0000 7fb193eb0b80 1 -- 172.21.15.112:0/1261217414 shutdown_connections 2024-05-08T06:03:08.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.654+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/1261217414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b69769d0 0x5641b6976e30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.654+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/1261217414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972060 0x5641b6972440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.654+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/1261217414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b6972980 0x5641b6976490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.654+0000 7fb193eb0b80 1 -- 172.21.15.112:0/1261217414 >> 172.21.15.112:0/1261217414 conn(0x5641b68a0b60 msgr2=0x5641b696c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:08.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.654+0000 7fb193eb0b80 1 -- 172.21.15.112:0/1261217414 shutdown_connections 2024-05-08T06:03:08.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.654+0000 7fb193eb0b80 1 -- 172.21.15.112:0/1261217414 wait complete. 2024-05-08T06:03:08.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.655+0000 7fb193eb0b80 1 -- start start 2024-05-08T06:03:08.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.655+0000 7fb193eb0b80 1 Processor -- start 2024-05-08T06:03:08.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.655+0000 7fb193eb0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b6972060 0x5641b697df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.655+0000 7fb193eb0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 0x5641b697b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.656+0000 7fb193eb0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b69769d0 0x5641b697b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.656+0000 7fb193eb0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5641b69873f0 con 0x5641b6972060 2024-05-08T06:03:08.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.656+0000 7fb193eb0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5641b682ab20 con 0x5641b69769d0 2024-05-08T06:03:08.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.656+0000 7fb193eb0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5641b6987570 con 0x5641b6972980 2024-05-08T06:03:08.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.656+0000 7fb193eb0b80 10 client.0 _ll_register_callbacks cb 0x5641b686bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:03:08.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.656+0000 7fb182000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b6972060 0x5641b697df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:08.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.656+0000 7fb182000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b6972060 0x5641b697df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60320/0 (socket says 172.21.15.112:60320) 2024-05-08T06:03:08.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.656+0000 7fb182000700 1 -- 172.21.15.112:0/4144266940 learned_addr learned my addr 172.21.15.112:0/4144266940 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:08.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.656+0000 7fb182000700 1 --2- 172.21.15.112:0/4144266940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b69769d0 0x5641b697b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:08.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.656+0000 7fb182000700 1 -- 172.21.15.112:0/4144266940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 msgr2=0x5641b697b2a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T06:03:08.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.656+0000 7fb182000700 1 --2- 172.21.15.112:0/4144266940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 0x5641b697b2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.656+0000 7fb182000700 1 -- 172.21.15.112:0/4144266940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b69769d0 msgr2=0x5641b697b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:08.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.656+0000 7fb182000700 1 --2- 172.21.15.112:0/4144266940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b69769d0 0x5641b697b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.656+0000 7fb182000700 1 -- 172.21.15.112:0/4144266940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b688a420 con 0x5641b6972060 2024-05-08T06:03:08.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.657+0000 7fb182000700 1 --2- 172.21.15.112:0/4144266940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b6972060 0x5641b697df00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb170009340 tx=0x7fb1700092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7f78f36dc05e67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:08.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.657+0000 7fb16cc00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T06:03:08.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.657+0000 7fb16cc00700 1 -- 172.21.15.112:0/4144266940 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb170023030 con 0x5641b6972060 2024-05-08T06:03:08.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.657+0000 7fb16cc00700 1 -- 172.21.15.112:0/4144266940 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fb17001a040 con 0x5641b6972060 2024-05-08T06:03:08.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.657+0000 7fb193eb0b80 1 -- 172.21.15.112:0/4144266940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb170003680 con 0x5641b6972060 2024-05-08T06:03:08.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.658+0000 7fb16cc00700 1 -- 172.21.15.112:0/4144266940 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb170029070 con 0x5641b6972060 2024-05-08T06:03:08.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.658+0000 7fb193eb0b80 10 client.15804 fetch_fsmap learned FSMap version 33 2024-05-08T06:03:08.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.658+0000 7fb193eb0b80 1 -- 172.21.15.112:0/4144266940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5641b6991d00 con 0x5641b6972060 2024-05-08T06:03:08.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.658+0000 7fb16cc00700 1 -- 172.21.15.112:0/4144266940 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb170003680 con 0x5641b6972060 2024-05-08T06:03:08.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.658+0000 7fb16cc00700 1 -- 172.21.15.112:0/4144266940 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7fb170031410 con 0x5641b6972060 2024-05-08T06:03:08.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.658+0000 7fb193eb0b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:03:08.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.658+0000 7fb193eb0b80 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:03:08.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.658+0000 7fb193eb0b80 20 client.15804 populate_metadata read hostname 'smithi112' 2024-05-08T06:03:08.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.658+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/4144266940 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5641b69941d0 0x5641b69965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.658+0000 7fb193eb0b80 4 client.15804 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:03:08.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.658+0000 7fb193eb0b80 1 -- 172.21.15.112:0/4144266940 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641b693f0a0 con 0x5641b69941d0 2024-05-08T06:03:08.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.659+0000 7fb182a00700 1 --2- 172.21.15.112:0/4144266940 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5641b69941d0 0x5641b69965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:08.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.659+0000 7fb182a00700 1 --2- 172.21.15.112:0/4144266940 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5641b69941d0 0x5641b69965c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8dca7f239bd22f2 server_cookie=d6ee094a6f933bfd in_seq=0 out_seq=0 2024-05-08T06:03:08.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.659+0000 7fb16cc00700 10 client.15804 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:08.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.660+0000 7fb16cc00700 1 -- 172.21.15.112:0/4144266940 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5641b693f0a0 con 0x5641b69941d0 2024-05-08T06:03:08.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.660+0000 7fb16cc00700 10 client.15804 handle_command_reply: tid=0 2024-05-08T06:03:08.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.660+0000 7fb193eb0b80 1 client.15804 shutdown 2024-05-08T06:03:08.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.660+0000 7fb193eb0b80 1 -- 172.21.15.112:0/4144266940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b6972060 msgr2=0x5641b697df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:08.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.660+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/4144266940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b6972060 0x5641b697df00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb170009340 tx=0x7fb1700092d0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.661+0000 7fb193eb0b80 1 -- 172.21.15.112:0/4144266940 shutdown_connections 2024-05-08T06:03:08.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.661+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/4144266940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b69769d0 0x5641b697b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.661+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/4144266940 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5641b69941d0 0x5641b69965c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.661+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/4144266940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b6972060 0x5641b697df00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.661+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/4144266940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 0x5641b697b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.661+0000 7fb193eb0b80 1 -- 172.21.15.112:0/4144266940 >> 172.21.15.112:0/4144266940 conn(0x5641b68a0b60 msgr2=0x5641b69672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:08.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.661+0000 7fb16cc00700 0 client.15804 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:08.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.661+0000 7fb193eb0b80 1 -- 172.21.15.112:0/4144266940 shutdown_connections 2024-05-08T06:03:08.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.661+0000 7fb193eb0b80 1 -- 172.21.15.112:0/4144266940 wait complete. 2024-05-08T06:03:08.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.661+0000 7fb193eb0b80 20 client.15804 trim_cache size 0 max 16384 2024-05-08T06:03:08.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.678+0000 7fb193eb0b80 1 Processor -- start 2024-05-08T06:03:08.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.678+0000 7fb193eb0b80 1 -- start start 2024-05-08T06:03:08.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.678+0000 7fb193eb0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b6972060 0x5641b6978e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.678+0000 7fb193eb0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 0x5641b697a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.678+0000 7fb193eb0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b69769d0 0x5641b6970e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.678+0000 7fb193eb0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841ae160 con 0x5641b69769d0 2024-05-08T06:03:08.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.678+0000 7fb193eb0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841ab9f0 con 0x5641b6972060 2024-05-08T06:03:08.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.678+0000 7fb193eb0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb18410e390 con 0x5641b6972980 2024-05-08T06:03:08.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.678+0000 7fb182a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 0x5641b697a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:08.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.678+0000 7fb182000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b6972060 0x5641b6978e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:08.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.678+0000 7fb182a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 0x5641b697a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60758/0 (socket says 172.21.15.112:60758) 2024-05-08T06:03:08.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.678+0000 7fb182a00700 1 -- 172.21.15.112:0/3089161999 learned_addr learned my addr 172.21.15.112:0/3089161999 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:08.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.679+0000 7fb182000700 1 --2- 172.21.15.112:0/3089161999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b69769d0 0x5641b6970e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:08.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.679+0000 7fb182a00700 1 -- 172.21.15.112:0/3089161999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b6972060 msgr2=0x5641b6978e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:08.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.679+0000 7fb182a00700 1 --2- 172.21.15.112:0/3089161999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b6972060 0x5641b6978e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.679+0000 7fb182a00700 1 -- 172.21.15.112:0/3089161999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b69769d0 msgr2=0x5641b6970e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:08.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.679+0000 7fb182a00700 1 --2- 172.21.15.112:0/3089161999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b69769d0 0x5641b6970e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.679+0000 7fb182a00700 1 -- 172.21.15.112:0/3089161999 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b697c1e0 con 0x5641b6972980 2024-05-08T06:03:08.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.679+0000 7fb182a00700 1 --2- 172.21.15.112:0/3089161999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 0x5641b697a170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb1740092a0 tx=0x7fb174004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdb3ea1de6d2dd0b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:08.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.679+0000 7fb16f400700 1 -- 172.21.15.112:0/3089161999 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174012af0 con 0x5641b6972980 2024-05-08T06:03:08.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.680+0000 7fb16f400700 1 -- 172.21.15.112:0/3089161999 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb174012c90 con 0x5641b6972980 2024-05-08T06:03:08.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.680+0000 7fb16f400700 1 -- 172.21.15.112:0/3089161999 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174010690 con 0x5641b6972980 2024-05-08T06:03:08.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.681+0000 7fb193eb0b80 1 -- 172.21.15.112:0/3089161999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 msgr2=0x5641b697a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:08.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.681+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/3089161999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 0x5641b697a170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb1740092a0 tx=0x7fb174004750 comp rx=0 tx=0).stop 2024-05-08T06:03:08.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.681+0000 7fb193eb0b80 1 -- 172.21.15.112:0/3089161999 shutdown_connections 2024-05-08T06:03:08.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.681+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/3089161999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b69769d0 0x5641b6970e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.681+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/3089161999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b6972060 0x5641b6978e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.681+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/3089161999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 0x5641b697a170 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.681+0000 7fb193eb0b80 1 -- 172.21.15.112:0/3089161999 >> 172.21.15.112:0/3089161999 conn(0x5641b68a0b60 msgr2=0x5641b6979a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:08.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.681+0000 7fb193eb0b80 1 -- 172.21.15.112:0/3089161999 shutdown_connections 2024-05-08T06:03:08.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.681+0000 7fb193eb0b80 1 -- 172.21.15.112:0/3089161999 wait complete. 2024-05-08T06:03:08.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.682+0000 7fb193eb0b80 1 -- start start 2024-05-08T06:03:08.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.682+0000 7fb193eb0b80 1 Processor -- start 2024-05-08T06:03:08.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.683+0000 7fb193eb0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b6972060 0x5641b699acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.683+0000 7fb193eb0b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 0x5641b699b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.683+0000 7fb193eb0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b69769d0 0x5641b699b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.683+0000 7fb193eb0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841a9eb0 con 0x5641b69769d0 2024-05-08T06:03:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.683+0000 7fb193eb0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841ab320 con 0x5641b6972060 2024-05-08T06:03:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.683+0000 7fb193eb0b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841aa580 con 0x5641b6972980 2024-05-08T06:03:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.683+0000 7fb193eb0b80 10 client.0 _ll_register_callbacks cb 0x5641b686bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:03:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.683+0000 7fb182a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 0x5641b699b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.683+0000 7fb182a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 0x5641b699b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60762/0 (socket says 172.21.15.112:60762) 2024-05-08T06:03:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.683+0000 7fb182a00700 1 -- 172.21.15.112:0/2065150278 learned_addr learned my addr 172.21.15.112:0/2065150278 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:08.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.683+0000 7fb182000700 1 --2- 172.21.15.112:0/2065150278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b6972060 0x5641b699acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:08.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.684+0000 7fb182000700 1 -- 172.21.15.112:0/2065150278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 msgr2=0x5641b699b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:08.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.684+0000 7fb182000700 1 --2- 172.21.15.112:0/2065150278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 0x5641b699b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.684+0000 7fb182000700 1 -- 172.21.15.112:0/2065150278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b69769d0 msgr2=0x5641b699b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:08.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.684+0000 7fb182000700 1 --2- 172.21.15.112:0/2065150278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b69769d0 0x5641b699b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.684+0000 7fb182000700 1 -- 172.21.15.112:0/2065150278 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b698b800 con 0x5641b6972060 2024-05-08T06:03:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.684+0000 7fb182a00700 1 --2- 172.21.15.112:0/2065150278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 0x5641b699b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:03:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.684+0000 7fb182000700 1 --2- 172.21.15.112:0/2065150278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b6972060 0x5641b699acf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb170031830 tx=0x7fb170031740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69f2e267ea125efc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.684+0000 7fb16cc00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:03:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.684+0000 7fb16cc00700 1 -- 172.21.15.112:0/2065150278 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb170042060 con 0x5641b6972060 2024-05-08T06:03:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.684+0000 7fb16cc00700 1 -- 172.21.15.112:0/2065150278 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fb170031aa0 con 0x5641b6972060 2024-05-08T06:03:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.684+0000 7fb193eb0b80 1 -- 172.21.15.112:0/2065150278 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb174012c90 con 0x5641b6972060 2024-05-08T06:03:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.685+0000 7fb16cc00700 1 -- 172.21.15.112:0/2065150278 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb170023040 con 0x5641b6972060 2024-05-08T06:03:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.685+0000 7fb193eb0b80 10 client.25994 fetch_fsmap learned FSMap version 33 2024-05-08T06:03:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.685+0000 7fb193eb0b80 1 -- 172.21.15.112:0/2065150278 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5641b698cb00 con 0x5641b6972060 2024-05-08T06:03:08.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.685+0000 7fb16cc00700 1 -- 172.21.15.112:0/2065150278 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb174012c90 con 0x5641b6972060 2024-05-08T06:03:08.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.685+0000 7fb16cc00700 1 -- 172.21.15.112:0/2065150278 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7fb170030a90 con 0x5641b6972060 2024-05-08T06:03:08.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.685+0000 7fb193eb0b80 10 client.25994 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:03:08.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.685+0000 7fb193eb0b80 10 client.25994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:03:08.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.685+0000 7fb193eb0b80 20 client.25994 populate_metadata read hostname 'smithi112' 2024-05-08T06:03:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.686+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/2065150278 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5641b6a0f8c0 0x5641b6a11cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.686+0000 7fb193eb0b80 4 client.25994 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:03:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.686+0000 7fb193eb0b80 1 -- 172.21.15.112:0/2065150278 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641b68d2400 con 0x5641b6a0f8c0 2024-05-08T06:03:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.686+0000 7fb182a00700 1 --2- 172.21.15.112:0/2065150278 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5641b6a0f8c0 0x5641b6a11cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.686+0000 7fb182a00700 1 --2- 172.21.15.112:0/2065150278 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5641b6a0f8c0 0x5641b6a11cb0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56b9e7b63350bdd0 server_cookie=b842bfe86b823ce7 in_seq=0 out_seq=0 2024-05-08T06:03:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.686+0000 7fb16cc00700 10 client.25994 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.687+0000 7fb16cc00700 1 -- 172.21.15.112:0/2065150278 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641b68d2400 con 0x5641b6a0f8c0 2024-05-08T06:03:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.687+0000 7fb16cc00700 10 client.25994 handle_command_reply: tid=0 2024-05-08T06:03:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.687+0000 7fb193eb0b80 1 client.25994 shutdown 2024-05-08T06:03:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.687+0000 7fb193eb0b80 1 -- 172.21.15.112:0/2065150278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b6972060 msgr2=0x5641b699acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.687+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/2065150278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b6972060 0x5641b699acf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb170031830 tx=0x7fb170031740 comp rx=0 tx=0).stop 2024-05-08T06:03:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.687+0000 7fb193eb0b80 1 -- 172.21.15.112:0/2065150278 shutdown_connections 2024-05-08T06:03:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.687+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/2065150278 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5641b6a0f8c0 0x5641b6a11cb0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.687+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/2065150278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641b69769d0 0x5641b699b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.687+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/2065150278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641b6972060 0x5641b699acf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.687+0000 7fb193eb0b80 1 --2- 172.21.15.112:0/2065150278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5641b6972980 0x5641b699b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.687+0000 7fb193eb0b80 1 -- 172.21.15.112:0/2065150278 >> 172.21.15.112:0/2065150278 conn(0x5641b68a0b60 msgr2=0x5641b6971700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.687+0000 7fb16cc00700 0 client.25994 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.687+0000 7fb193eb0b80 1 -- 172.21.15.112:0/2065150278 shutdown_connections 2024-05-08T06:03:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.687+0000 7fb193eb0b80 1 -- 172.21.15.112:0/2065150278 wait complete. 2024-05-08T06:03:08.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.688+0000 7fb193eb0b80 20 client.25994 trim_cache size 0 max 16384 2024-05-08T06:03:08.690 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T06:03:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.688+0000 7fb189600700 1 -- 172.21.15.112:0/3320480835 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb1680b2980 msgr2=0x7fb1680b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:08.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.688+0000 7fb189600700 1 --2- 172.21.15.112:0/3320480835 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb1680b2980 0x7fb1680b4e40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0a6770 tx=0x7fb17c0095a0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.688+0000 7fb189600700 1 -- 172.21.15.112:0/3320480835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb184108850 msgr2=0x7fb18419bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:08.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.688+0000 7fb189600700 1 --2- 172.21.15.112:0/3320480835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb184108850 0x7fb18419bae0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0079a0 tx=0x7fb17c0079d0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.688+0000 7fb189600700 1 -- 172.21.15.112:0/3320480835 shutdown_connections 2024-05-08T06:03:08.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.688+0000 7fb189600700 1 --2- 172.21.15.112:0/3320480835 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb1680b2980 0x7fb1680b4e40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.688+0000 7fb189600700 1 --2- 172.21.15.112:0/3320480835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb184108850 0x7fb18419bae0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.688+0000 7fb189600700 1 --2- 172.21.15.112:0/3320480835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb184103750 0x7fb1841a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.689+0000 7fb189600700 1 --2- 172.21.15.112:0/3320480835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb184104100 0x7fb1841a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:08.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.689+0000 7fb189600700 1 -- 172.21.15.112:0/3320480835 >> 172.21.15.112:0/3320480835 conn(0x7fb18407a600 msgr2=0x7fb1841013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:08.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.689+0000 7fb189600700 1 -- 172.21.15.112:0/3320480835 shutdown_connections 2024-05-08T06:03:08.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:08.689+0000 7fb189600700 1 -- 172.21.15.112:0/3320480835 wait complete. 2024-05-08T06:03:09.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:08 smithi112 ceph-mon[30003]: pgmap v676: 97 pgs: 97 active+clean; 144 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.1 MiB/s wr, 410 op/s 2024-05-08T06:03:09.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:08 smithi150 ceph-mon[34315]: pgmap v676: 97 pgs: 97 active+clean; 144 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.1 MiB/s wr, 410 op/s 2024-05-08T06:03:09.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:08 smithi162 ceph-mon[34800]: pgmap v676: 97 pgs: 97 active+clean; 144 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.1 MiB/s wr, 410 op/s 2024-05-08T06:03:10.679 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T06:03:10.680 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T06:03:11.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:10 smithi112 ceph-mon[30003]: pgmap v677: 97 pgs: 97 active+clean; 134 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.3 MiB/s wr, 452 op/s 2024-05-08T06:03:11.177 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:03:11.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:10 smithi150 ceph-mon[34315]: pgmap v677: 97 pgs: 97 active+clean; 134 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.3 MiB/s wr, 452 op/s 2024-05-08T06:03:11.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:10 smithi162 ceph-mon[34800]: pgmap v677: 97 pgs: 97 active+clean; 134 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.3 MiB/s wr, 452 op/s 2024-05-08T06:03:13.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:12 smithi112 ceph-mon[30003]: pgmap v678: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.3 MiB/s wr, 453 op/s 2024-05-08T06:03:13.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:12 smithi150 ceph-mon[34315]: pgmap v678: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.3 MiB/s wr, 453 op/s 2024-05-08T06:03:13.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:12 smithi162 ceph-mon[34800]: pgmap v678: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.3 MiB/s wr, 453 op/s 2024-05-08T06:03:13.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.635+0000 7f44ef000700 1 -- 172.21.15.112:0/1278383385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e8075ff0 msgr2=0x7f44e810f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:13.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.635+0000 7f44ef000700 1 --2- 172.21.15.112:0/1278383385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e8075ff0 0x7f44e810f960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009990 tx=0x7f44d4009ca0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.635+0000 7f44ef000700 1 -- 172.21.15.112:0/1278383385 shutdown_connections 2024-05-08T06:03:13.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.635+0000 7f44ef000700 1 --2- 172.21.15.112:0/1278383385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44e810fea0 0x7f44e8112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.635+0000 7f44ef000700 1 --2- 172.21.15.112:0/1278383385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e80756d0 0x7f44e8075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.635+0000 7f44ef000700 1 --2- 172.21.15.112:0/1278383385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e8075ff0 0x7f44e810f960 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.635+0000 7f44ef000700 1 -- 172.21.15.112:0/1278383385 >> 172.21.15.112:0/1278383385 conn(0x7f44e807a600 msgr2=0x7f44e807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:13.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.635+0000 7f44ef000700 1 -- 172.21.15.112:0/1278383385 shutdown_connections 2024-05-08T06:03:13.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.635+0000 7f44ef000700 1 -- 172.21.15.112:0/1278383385 wait complete. 2024-05-08T06:03:13.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.636+0000 7f44ef000700 1 Processor -- start 2024-05-08T06:03:13.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.636+0000 7f44ef000700 1 -- start start 2024-05-08T06:03:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.637+0000 7f44ef000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e80756d0 0x7f44e819be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.637+0000 7f44ef000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e8075ff0 0x7f44e819c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.637+0000 7f44ef000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44e810fea0 0x7f44e81a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.637+0000 7f44ef000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44e81158b0 con 0x7f44e8075ff0 2024-05-08T06:03:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.637+0000 7f44ef000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8115730 con 0x7f44e80756d0 2024-05-08T06:03:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.637+0000 7f44ef000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8115a30 con 0x7f44e810fea0 2024-05-08T06:03:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.637+0000 7f44e7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e80756d0 0x7f44e819be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.637+0000 7f44e7400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e8075ff0 0x7f44e819c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.637+0000 7f44e7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e80756d0 0x7f44e819be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:39632/0 (socket says 172.21.15.112:39632) 2024-05-08T06:03:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.637+0000 7f44e7e00700 1 -- 172.21.15.112:0/477739852 learned_addr learned my addr 172.21.15.112:0/477739852 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:13.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.637+0000 7f44ed000700 1 --2- 172.21.15.112:0/477739852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44e810fea0 0x7f44e81a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.638+0000 7f44e7e00700 1 -- 172.21.15.112:0/477739852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44e810fea0 msgr2=0x7f44e81a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.638+0000 7f44e7e00700 1 --2- 172.21.15.112:0/477739852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44e810fea0 0x7f44e81a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.638+0000 7f44e7e00700 1 -- 172.21.15.112:0/477739852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e8075ff0 msgr2=0x7f44e819c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.638+0000 7f44e7e00700 1 --2- 172.21.15.112:0/477739852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e8075ff0 0x7f44e819c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.638+0000 7f44e7e00700 1 -- 172.21.15.112:0/477739852 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e81a0de0 con 0x7f44e80756d0 2024-05-08T06:03:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.638+0000 7f44e7e00700 1 --2- 172.21.15.112:0/477739852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e80756d0 0x7f44e819be20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f44d800d950 tx=0x7f44d800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:13.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.638+0000 7f44e4c00700 1 -- 172.21.15.112:0/477739852 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8016070 con 0x7f44e80756d0 2024-05-08T06:03:13.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.638+0000 7f44e4c00700 1 -- 172.21.15.112:0/477739852 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44d800fd40 con 0x7f44e80756d0 2024-05-08T06:03:13.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.638+0000 7f44e4c00700 1 -- 172.21.15.112:0/477739852 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d800e3b0 con 0x7f44e80756d0 2024-05-08T06:03:13.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.638+0000 7f44ef000700 1 -- 172.21.15.112:0/477739852 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e81a10d0 con 0x7f44e80756d0 2024-05-08T06:03:13.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.639+0000 7f44ef000700 1 -- 172.21.15.112:0/477739852 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e81a8950 con 0x7f44e80756d0 2024-05-08T06:03:13.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.642+0000 7f44e4c00700 1 -- 172.21.15.112:0/477739852 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f44d8019070 con 0x7f44e80756d0 2024-05-08T06:03:13.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.643+0000 7f44f9893b80 1 Processor -- start 2024-05-08T06:03:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.644+0000 7f44f9893b80 1 -- start start 2024-05-08T06:03:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.644+0000 7f44f9893b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x55670900c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.644+0000 7f44f9893b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55670900ca00 0x556709010510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.644+0000 7f44f9893b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556709010a50 0x556709010eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.644+0000 7f44f9893b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556708f5eaa0 con 0x55670900ca00 2024-05-08T06:03:13.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.644+0000 7f44f9893b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556708efc7e0 con 0x556709010a50 2024-05-08T06:03:13.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.644+0000 7f44f9893b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x556708f5aa60 con 0x55670900c0e0 2024-05-08T06:03:13.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.644+0000 7f44ed000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x55670900c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.644+0000 7f44ed000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x55670900c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60804/0 (socket says 172.21.15.112:60804) 2024-05-08T06:03:13.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.644+0000 7f44ed000700 1 -- 172.21.15.112:0/1680226224 learned_addr learned my addr 172.21.15.112:0/1680226224 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:13.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.644+0000 7f44e7e00700 1 --2- 172.21.15.112:0/1680226224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55670900ca00 0x556709010510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.645+0000 7f44e4c00700 1 --2- 172.21.15.112:0/477739852 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f44cc0b2880 0x7f44cc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.645+0000 7f44e7400700 1 --2- 172.21.15.112:0/1680226224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556709010a50 0x556709010eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.645+0000 7f44e4c00700 1 -- 172.21.15.112:0/477739852 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f44d80e1d30 con 0x7f44e80756d0 2024-05-08T06:03:13.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.645+0000 7f44ed000700 1 -- 172.21.15.112:0/1680226224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556709010a50 msgr2=0x556709010eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:13.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.645+0000 7f44ed000700 1 --2- 172.21.15.112:0/1680226224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556709010a50 0x556709010eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.645+0000 7f44ed000700 1 -- 172.21.15.112:0/1680226224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55670900ca00 msgr2=0x556709010510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.645+0000 7f44ed000700 1 --2- 172.21.15.112:0/1680226224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55670900ca00 0x556709010510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.645+0000 7f44ed000700 1 -- 172.21.15.112:0/1680226224 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556708eab950 con 0x55670900c0e0 2024-05-08T06:03:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.645+0000 7f44ed000700 1 --2- 172.21.15.112:0/477739852 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f44cc0b2880 0x7f44cc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.646+0000 7f44ed000700 1 --2- 172.21.15.112:0/1680226224 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x55670900c4c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f44e0011500 tx=0x7f44e0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a863afd72fce9310 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.646+0000 7f44dd600700 1 -- 172.21.15.112:0/1680226224 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0019070 con 0x55670900c0e0 2024-05-08T06:03:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.646+0000 7f44dd600700 1 -- 172.21.15.112:0/1680226224 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e001e430 con 0x55670900c0e0 2024-05-08T06:03:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.646+0000 7f44dd600700 1 -- 172.21.15.112:0/1680226224 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e00145d0 con 0x55670900c0e0 2024-05-08T06:03:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.647+0000 7f44f9893b80 1 -- 172.21.15.112:0/1680226224 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 msgr2=0x55670900c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.647+0000 7f44f9893b80 1 --2- 172.21.15.112:0/1680226224 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x55670900c4c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f44e0011500 tx=0x7f44e0011810 comp rx=0 tx=0).stop 2024-05-08T06:03:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.647+0000 7f44ed000700 1 --2- 172.21.15.112:0/477739852 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f44cc0b2880 0x7f44cc0b4d40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f44e0020000 tx=0x7f44e001d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.647+0000 7f44f9893b80 1 -- 172.21.15.112:0/1680226224 shutdown_connections 2024-05-08T06:03:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.647+0000 7f44f9893b80 1 --2- 172.21.15.112:0/1680226224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556709010a50 0x556709010eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.647+0000 7f44f9893b80 1 --2- 172.21.15.112:0/1680226224 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x55670900c4c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.647+0000 7f44f9893b80 1 --2- 172.21.15.112:0/1680226224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55670900ca00 0x556709010510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.647+0000 7f44f9893b80 1 -- 172.21.15.112:0/1680226224 >> 172.21.15.112:0/1680226224 conn(0x556708f3abe0 msgr2=0x556709006820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.647+0000 7f44f9893b80 1 -- 172.21.15.112:0/1680226224 shutdown_connections 2024-05-08T06:03:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.647+0000 7f44f9893b80 1 -- 172.21.15.112:0/1680226224 wait complete. 2024-05-08T06:03:13.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.648+0000 7f44f9893b80 1 -- start start 2024-05-08T06:03:13.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.648+0000 7f44f9893b80 1 Processor -- start 2024-05-08T06:03:13.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44f9893b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55670900c0e0 0x556709017ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44f9893b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900ca00 0x5567090152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44f9893b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556709010a50 0x556709015820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44f9893b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556709018c60 con 0x556709010a50 2024-05-08T06:03:13.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44f9893b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556708ec4ba0 con 0x55670900c0e0 2024-05-08T06:03:13.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44f9893b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x556709018de0 con 0x55670900ca00 2024-05-08T06:03:13.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44f9893b80 10 client.0 _ll_register_callbacks cb 0x556708f05cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:03:13.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44e7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55670900c0e0 0x556709017ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44ed000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556709010a50 0x556709015820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44e7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55670900c0e0 0x556709017ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:39642/0 (socket says 172.21.15.112:39642) 2024-05-08T06:03:13.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44e7e00700 1 -- 172.21.15.112:0/4272759982 learned_addr learned my addr 172.21.15.112:0/4272759982 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:13.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44e7400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900ca00 0x5567090152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44e7e00700 1 -- 172.21.15.112:0/4272759982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900ca00 msgr2=0x5567090152e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44e7e00700 1 --2- 172.21.15.112:0/4272759982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900ca00 0x5567090152e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44e7e00700 1 -- 172.21.15.112:0/4272759982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556709010a50 msgr2=0x556709015820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.649+0000 7f44e7e00700 1 --2- 172.21.15.112:0/4272759982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556709010a50 0x556709015820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.650+0000 7f44e7e00700 1 -- 172.21.15.112:0/4272759982 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556708f244a0 con 0x55670900c0e0 2024-05-08T06:03:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.650+0000 7f44e7e00700 1 --2- 172.21.15.112:0/4272759982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55670900c0e0 0x556709017ed0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f44d80a9020 tx=0x7f44d8010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb24aa21dbdda5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.650+0000 7f44d2000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:03:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.650+0000 7f44d2000700 1 -- 172.21.15.112:0/4272759982 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8020040 con 0x55670900c0e0 2024-05-08T06:03:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.650+0000 7f44d2000700 1 -- 172.21.15.112:0/4272759982 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f44d80a9580 con 0x55670900c0e0 2024-05-08T06:03:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.650+0000 7f44f9893b80 1 -- 172.21.15.112:0/4272759982 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44e001e430 con 0x55670900c0e0 2024-05-08T06:03:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.650+0000 7f44d2000700 1 -- 172.21.15.112:0/4272759982 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f44d80a9320 con 0x55670900c0e0 2024-05-08T06:03:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.650+0000 7f44f9893b80 10 client.26006 fetch_fsmap learned FSMap version 33 2024-05-08T06:03:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.650+0000 7f44f9893b80 1 -- 172.21.15.112:0/4272759982 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5567090235c0 con 0x55670900c0e0 2024-05-08T06:03:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.651+0000 7f44d2000700 1 -- 172.21.15.112:0/4272759982 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e001e430 con 0x55670900c0e0 2024-05-08T06:03:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.651+0000 7f44d2000700 1 -- 172.21.15.112:0/4272759982 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f44d80b1df0 con 0x55670900c0e0 2024-05-08T06:03:13.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.651+0000 7f44f9893b80 10 client.26006 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:03:13.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.651+0000 7f44f9893b80 10 client.26006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:03:13.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.651+0000 7f44f9893b80 20 client.26006 populate_metadata read hostname 'smithi112' 2024-05-08T06:03:13.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.651+0000 7f44f9893b80 1 --2- 172.21.15.112:0/4272759982 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556709025a90 0x556709027e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.651+0000 7f44f9893b80 4 client.26006 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:03:13.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.651+0000 7f44f9893b80 1 -- 172.21.15.112:0/4272759982 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556708fd9120 con 0x556709025a90 2024-05-08T06:03:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.651+0000 7f44e7400700 1 --2- 172.21.15.112:0/4272759982 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556709025a90 0x556709027e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.652+0000 7f44e7400700 1 --2- 172.21.15.112:0/4272759982 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556709025a90 0x556709027e80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59b52b0e2368404f server_cookie=b6d18da61411acb3 in_seq=0 out_seq=0 2024-05-08T06:03:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.652+0000 7f44d2000700 10 client.26006 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.653+0000 7f44d2000700 1 -- 172.21.15.112:0/4272759982 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556708fd9120 con 0x556709025a90 2024-05-08T06:03:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.653+0000 7f44d2000700 10 client.26006 handle_command_reply: tid=0 2024-05-08T06:03:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.653+0000 7f44f9893b80 1 client.26006 shutdown 2024-05-08T06:03:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.653+0000 7f44f9893b80 1 -- 172.21.15.112:0/4272759982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55670900c0e0 msgr2=0x556709017ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.653+0000 7f44f9893b80 1 --2- 172.21.15.112:0/4272759982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55670900c0e0 0x556709017ed0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f44d80a9020 tx=0x7f44d8010000 comp rx=0 tx=0).stop 2024-05-08T06:03:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.653+0000 7f44f9893b80 1 -- 172.21.15.112:0/4272759982 shutdown_connections 2024-05-08T06:03:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.653+0000 7f44f9893b80 1 --2- 172.21.15.112:0/4272759982 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556709025a90 0x556709027e80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.653+0000 7f44f9893b80 1 --2- 172.21.15.112:0/4272759982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556709010a50 0x556709015820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.653+0000 7f44d2000700 0 client.26006 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.653+0000 7f44f9893b80 1 --2- 172.21.15.112:0/4272759982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55670900c0e0 0x556709017ed0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.653+0000 7f44f9893b80 1 --2- 172.21.15.112:0/4272759982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900ca00 0x5567090152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.653+0000 7f44f9893b80 1 -- 172.21.15.112:0/4272759982 >> 172.21.15.112:0/4272759982 conn(0x556708f3abe0 msgr2=0x5567090067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.654+0000 7f44f9893b80 1 -- 172.21.15.112:0/4272759982 shutdown_connections 2024-05-08T06:03:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.654+0000 7f44f9893b80 1 -- 172.21.15.112:0/4272759982 wait complete. 2024-05-08T06:03:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.654+0000 7f44f9893b80 20 client.26006 trim_cache size 0 max 16384 2024-05-08T06:03:13.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.681+0000 7f44f9893b80 1 Processor -- start 2024-05-08T06:03:13.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.682+0000 7f44f9893b80 1 -- start start 2024-05-08T06:03:13.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.682+0000 7f44f9893b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x556709021d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.682+0000 7f44f9893b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55670900ca00 0x55670900fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.682+0000 7f44f9893b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556709010a50 0x55670902d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.682+0000 7f44f9893b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44e81ae310 con 0x556709010a50 2024-05-08T06:03:13.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.682+0000 7f44f9893b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44e81abba0 con 0x55670900ca00 2024-05-08T06:03:13.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.682+0000 7f44f9893b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f44e81075b0 con 0x55670900c0e0 2024-05-08T06:03:13.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.682+0000 7f44ed000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556709010a50 0x55670902d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.682+0000 7f44e7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x556709021d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.682+0000 7f44e7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55670900ca00 0x55670900fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.682+0000 7f44e7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x556709021d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60808/0 (socket says 172.21.15.112:60808) 2024-05-08T06:03:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.683+0000 7f44e7e00700 1 -- 172.21.15.112:0/1704866882 learned_addr learned my addr 172.21.15.112:0/1704866882 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.683+0000 7f44e7e00700 1 -- 172.21.15.112:0/1704866882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55670900ca00 msgr2=0x55670900fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.683+0000 7f44e7e00700 1 --2- 172.21.15.112:0/1704866882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55670900ca00 0x55670900fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.683+0000 7f44e7e00700 1 -- 172.21.15.112:0/1704866882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556709010a50 msgr2=0x55670902d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.683+0000 7f44e7e00700 1 --2- 172.21.15.112:0/1704866882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556709010a50 0x55670902d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.683+0000 7f44e7e00700 1 -- 172.21.15.112:0/1704866882 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556709017910 con 0x55670900c0e0 2024-05-08T06:03:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.683+0000 7f44e7e00700 1 --2- 172.21.15.112:0/1704866882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x556709021d20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f44d80a8750 tx=0x7f44d8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cb3137aeebbf1a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.683+0000 7f44dcc00700 1 -- 172.21.15.112:0/1704866882 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8020070 con 0x55670900c0e0 2024-05-08T06:03:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.684+0000 7f44dcc00700 1 -- 172.21.15.112:0/1704866882 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44d806e040 con 0x55670900c0e0 2024-05-08T06:03:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.684+0000 7f44dcc00700 1 -- 172.21.15.112:0/1704866882 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d80b1f50 con 0x55670900c0e0 2024-05-08T06:03:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.684+0000 7f44f9893b80 1 -- 172.21.15.112:0/1704866882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 msgr2=0x556709021d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.684+0000 7f44f9893b80 1 --2- 172.21.15.112:0/1704866882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x556709021d20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f44d80a8750 tx=0x7f44d8000bc0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.685+0000 7f44f9893b80 1 -- 172.21.15.112:0/1704866882 shutdown_connections 2024-05-08T06:03:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.685+0000 7f44f9893b80 1 --2- 172.21.15.112:0/1704866882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556709010a50 0x55670902d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.685+0000 7f44f9893b80 1 --2- 172.21.15.112:0/1704866882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x556709021d20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.685+0000 7f44f9893b80 1 --2- 172.21.15.112:0/1704866882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55670900ca00 0x55670900fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.685+0000 7f44f9893b80 1 -- 172.21.15.112:0/1704866882 >> 172.21.15.112:0/1704866882 conn(0x556708f3abe0 msgr2=0x55670900b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.685+0000 7f44f9893b80 1 -- 172.21.15.112:0/1704866882 shutdown_connections 2024-05-08T06:03:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.685+0000 7f44f9893b80 1 -- 172.21.15.112:0/1704866882 wait complete. 2024-05-08T06:03:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.685+0000 7f44f9893b80 1 -- start start 2024-05-08T06:03:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.685+0000 7f44f9893b80 1 Processor -- start 2024-05-08T06:03:13.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.686+0000 7f44f9893b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x55670901cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.686+0000 7f44f9893b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55670900ca00 0x55670901d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.686+0000 7f44f9893b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556709010a50 0x55670901d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.686+0000 7f44f9893b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44e81aa060 con 0x55670900ca00 2024-05-08T06:03:13.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.686+0000 7f44f9893b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44e81ab4d0 con 0x556709010a50 2024-05-08T06:03:13.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.686+0000 7f44f9893b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f44e81aa730 con 0x55670900c0e0 2024-05-08T06:03:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.686+0000 7f44f9893b80 10 client.0 _ll_register_callbacks cb 0x556708f05cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:03:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.686+0000 7f44e7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x55670901cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.686+0000 7f44ed000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556709010a50 0x55670901d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.686+0000 7f44e7e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x55670901cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:60814/0 (socket says 172.21.15.112:60814) 2024-05-08T06:03:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.686+0000 7f44e7e00700 1 -- 172.21.15.112:0/3020297728 learned_addr learned my addr 172.21.15.112:0/3020297728 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.686+0000 7f44e7400700 1 --2- 172.21.15.112:0/3020297728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55670900ca00 0x55670901d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.687+0000 7f44ed000700 1 -- 172.21.15.112:0/3020297728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 msgr2=0x55670901cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.687+0000 7f44ed000700 1 --2- 172.21.15.112:0/3020297728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x55670901cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.687+0000 7f44ed000700 1 -- 172.21.15.112:0/3020297728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55670900ca00 msgr2=0x55670901d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:13.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.687+0000 7f44ed000700 1 --2- 172.21.15.112:0/3020297728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55670900ca00 0x55670901d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.687+0000 7f44ed000700 1 -- 172.21.15.112:0/3020297728 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556709090500 con 0x556709010a50 2024-05-08T06:03:13.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.687+0000 7f44e7400700 1 --2- 172.21.15.112:0/3020297728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55670900ca00 0x55670901d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:03:13.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.687+0000 7f44ed000700 1 --2- 172.21.15.112:0/3020297728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556709010a50 0x55670901d960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f44e00079b0 tx=0x7f44e001ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dafef92347349fe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:13.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.687+0000 7f44d2000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:03:13.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.687+0000 7f44d2000700 1 -- 172.21.15.112:0/3020297728 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0019040 con 0x556709010a50 2024-05-08T06:03:13.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.687+0000 7f44d2000700 1 -- 172.21.15.112:0/3020297728 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f44e0007480 con 0x556709010a50 2024-05-08T06:03:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.687+0000 7f44f9893b80 1 -- 172.21.15.112:0/3020297728 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44d806e040 con 0x556709010a50 2024-05-08T06:03:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.688+0000 7f44d2000700 1 -- 172.21.15.112:0/3020297728 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f44e0038030 con 0x556709010a50 2024-05-08T06:03:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.688+0000 7f44f9893b80 10 client.26012 fetch_fsmap learned FSMap version 33 2024-05-08T06:03:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.688+0000 7f44f9893b80 1 -- 172.21.15.112:0/3020297728 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556709091820 con 0x556709010a50 2024-05-08T06:03:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.688+0000 7f44d2000700 1 -- 172.21.15.112:0/3020297728 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44d806e040 con 0x556709010a50 2024-05-08T06:03:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.688+0000 7f44d2000700 1 -- 172.21.15.112:0/3020297728 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f44e0030020 con 0x556709010a50 2024-05-08T06:03:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.688+0000 7f44f9893b80 10 client.26012 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:03:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.688+0000 7f44f9893b80 10 client.26012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:03:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.688+0000 7f44f9893b80 20 client.26012 populate_metadata read hostname 'smithi112' 2024-05-08T06:03:13.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.688+0000 7f44f9893b80 1 --2- 172.21.15.112:0/3020297728 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556709091bf0 0x556709093fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.688+0000 7f44f9893b80 4 client.26012 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T06:03:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.688+0000 7f44f9893b80 1 -- 172.21.15.112:0/3020297728 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556708f6c480 con 0x556709091bf0 2024-05-08T06:03:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.688+0000 7f44e7e00700 1 --2- 172.21.15.112:0/3020297728 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556709091bf0 0x556709093fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.689+0000 7f44e7e00700 1 --2- 172.21.15.112:0/3020297728 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556709091bf0 0x556709093fe0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ac192a88a3c58f server_cookie=499ecbe60a3ef499 in_seq=0 out_seq=0 2024-05-08T06:03:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.689+0000 7f44d2000700 10 client.26012 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.690+0000 7f44d2000700 1 -- 172.21.15.112:0/3020297728 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556708f6c480 con 0x556709091bf0 2024-05-08T06:03:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.690+0000 7f44d2000700 10 client.26012 handle_command_reply: tid=0 2024-05-08T06:03:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.690+0000 7f44f9893b80 1 client.26012 shutdown 2024-05-08T06:03:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.691+0000 7f44f9893b80 1 -- 172.21.15.112:0/3020297728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556709010a50 msgr2=0x55670901d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:13.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.691+0000 7f44f9893b80 1 --2- 172.21.15.112:0/3020297728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556709010a50 0x55670901d960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f44e00079b0 tx=0x7f44e001ecc0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.691+0000 7f44f9893b80 1 -- 172.21.15.112:0/3020297728 shutdown_connections 2024-05-08T06:03:13.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.691+0000 7f44f9893b80 1 --2- 172.21.15.112:0/3020297728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556709010a50 0x55670901d960 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.691+0000 7f44f9893b80 1 --2- 172.21.15.112:0/3020297728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670900c0e0 0x55670901cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.691+0000 7f44f9893b80 1 --2- 172.21.15.112:0/3020297728 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x556709091bf0 0x556709093fe0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.691+0000 7f44f9893b80 1 --2- 172.21.15.112:0/3020297728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55670900ca00 0x55670901d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.691+0000 7f44f9893b80 1 -- 172.21.15.112:0/3020297728 >> 172.21.15.112:0/3020297728 conn(0x556708f3abe0 msgr2=0x556709017120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:13.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.691+0000 7f44d2000700 0 client.26012 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:13.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.692+0000 7f44f9893b80 1 -- 172.21.15.112:0/3020297728 shutdown_connections 2024-05-08T06:03:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.692+0000 7f44f9893b80 1 -- 172.21.15.112:0/3020297728 wait complete. 2024-05-08T06:03:13.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.692+0000 7f44f9893b80 20 client.26012 trim_cache size 0 max 16384 2024-05-08T06:03:13.702 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:03:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-05-08T06:03:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "ae815242-035b-4772-b278-46256c6eb21d", 2024-05-08T06:03:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-05-08T06:03:13.703 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:03:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.693+0000 7f44ef000700 1 -- 172.21.15.112:0/477739852 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f44cc0b2880 msgr2=0x7f44cc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.693+0000 7f44ef000700 1 --2- 172.21.15.112:0/477739852 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f44cc0b2880 0x7f44cc0b4d40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f44e0020000 tx=0x7f44e001d000 comp rx=0 tx=0).stop 2024-05-08T06:03:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.693+0000 7f44ef000700 1 -- 172.21.15.112:0/477739852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e80756d0 msgr2=0x7f44e819be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:13.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.693+0000 7f44ef000700 1 --2- 172.21.15.112:0/477739852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e80756d0 0x7f44e819be20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f44d800d950 tx=0x7f44d800dc60 comp rx=0 tx=0).stop 2024-05-08T06:03:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.693+0000 7f44ef000700 1 -- 172.21.15.112:0/477739852 shutdown_connections 2024-05-08T06:03:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.693+0000 7f44ef000700 1 --2- 172.21.15.112:0/477739852 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f44cc0b2880 0x7f44cc0b4d40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.693+0000 7f44ef000700 1 --2- 172.21.15.112:0/477739852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44e810fea0 0x7f44e81a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.693+0000 7f44ef000700 1 --2- 172.21.15.112:0/477739852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44e80756d0 0x7f44e819be20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.693+0000 7f44ef000700 1 --2- 172.21.15.112:0/477739852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44e8075ff0 0x7f44e819c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.693+0000 7f44ef000700 1 -- 172.21.15.112:0/477739852 >> 172.21.15.112:0/477739852 conn(0x7f44e807a600 msgr2=0x7f44e8101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.693+0000 7f44ef000700 1 -- 172.21.15.112:0/477739852 shutdown_connections 2024-05-08T06:03:13.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:13.693+0000 7f44ef000700 1 -- 172.21.15.112:0/477739852 wait complete. 2024-05-08T06:03:14.673 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T06:03:15.152 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:03:15.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:14 smithi150 ceph-mon[34315]: scrub queued for path: / 2024-05-08T06:03:15.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:14 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:03:15.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:14 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T06:03:15.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:14 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T06:03:15.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:14 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:03:15.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:14 smithi150 ceph-mon[34315]: scrub summary: active paths [/] 2024-05-08T06:03:15.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:14 smithi150 ceph-mon[34315]: pgmap v679: 97 pgs: 97 active+clean; 130 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.1 MiB/s wr, 360 op/s 2024-05-08T06:03:15.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:14 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:03:15.305 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:14 smithi112 ceph-mon[30003]: scrub queued for path: / 2024-05-08T06:03:15.305 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:14 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:03:15.305 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:14 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T06:03:15.305 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:14 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T06:03:15.305 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:14 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:03:15.305 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:14 smithi112 ceph-mon[30003]: scrub summary: active paths [/] 2024-05-08T06:03:15.305 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:14 smithi112 ceph-mon[30003]: pgmap v679: 97 pgs: 97 active+clean; 130 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.1 MiB/s wr, 360 op/s 2024-05-08T06:03:15.305 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:14 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:03:15.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:14 smithi162 ceph-mon[34800]: scrub queued for path: / 2024-05-08T06:03:15.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:14 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:03:15.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:14 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T06:03:15.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:14 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T06:03:15.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:14 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:03:15.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:14 smithi162 ceph-mon[34800]: scrub summary: active paths [/] 2024-05-08T06:03:15.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:14 smithi162 ceph-mon[34800]: pgmap v679: 97 pgs: 97 active+clean; 130 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.1 MiB/s wr, 360 op/s 2024-05-08T06:03:15.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:14 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:03:16.165 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:15 smithi112 ceph-mon[30003]: scrub summary: idle+waiting paths [/] 2024-05-08T06:03:16.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:15 smithi150 ceph-mon[34315]: scrub summary: idle+waiting paths [/] 2024-05-08T06:03:16.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:15 smithi162 ceph-mon[34800]: scrub summary: idle+waiting paths [/] 2024-05-08T06:03:17.134 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:16 smithi112 ceph-mon[30003]: pgmap v680: 97 pgs: 97 active+clean; 131 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.3 MiB/s wr, 464 op/s 2024-05-08T06:03:17.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:16 smithi150 ceph-mon[34315]: pgmap v680: 97 pgs: 97 active+clean; 131 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.3 MiB/s wr, 464 op/s 2024-05-08T06:03:17.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:16 smithi162 ceph-mon[34800]: pgmap v680: 97 pgs: 97 active+clean; 131 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.3 MiB/s wr, 464 op/s 2024-05-08T06:03:17.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.467+0000 7f8f2f200700 1 -- 172.21.15.112:0/517659039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f30104fa0 msgr2=0x7f8f3010d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:17.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.467+0000 7f8f2f200700 1 --2- 172.21.15.112:0/517659039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f30104fa0 0x7f8f3010d490 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0098f0 tx=0x7f8f1c009c00 comp rx=0 tx=0).stop 2024-05-08T06:03:17.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.468+0000 7f8f2f200700 1 -- 172.21.15.112:0/517659039 shutdown_connections 2024-05-08T06:03:17.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.468+0000 7f8f2f200700 1 --2- 172.21.15.112:0/517659039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3010da60 0x7f8f3010fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.468+0000 7f8f2f200700 1 --2- 172.21.15.112:0/517659039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f30104680 0x7f8f30104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.468+0000 7f8f2f200700 1 --2- 172.21.15.112:0/517659039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f30104fa0 0x7f8f3010d490 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.468+0000 7f8f2f200700 1 -- 172.21.15.112:0/517659039 >> 172.21.15.112:0/517659039 conn(0x7f8f300fde30 msgr2=0x7f8f30100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:17.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.468+0000 7f8f2f200700 1 -- 172.21.15.112:0/517659039 shutdown_connections 2024-05-08T06:03:17.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.468+0000 7f8f2f200700 1 -- 172.21.15.112:0/517659039 wait complete. 2024-05-08T06:03:17.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.469+0000 7f8f2f200700 1 Processor -- start 2024-05-08T06:03:17.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.469+0000 7f8f2f200700 1 -- start start 2024-05-08T06:03:17.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.470+0000 7f8f2f200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f30104680 0x7f8f3019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.470+0000 7f8f2f200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f30104fa0 0x7f8f3019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.470+0000 7f8f2f200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f3010da60 0x7f8f301977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.470+0000 7f8f2f200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301114b0 con 0x7f8f30104fa0 2024-05-08T06:03:17.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.470+0000 7f8f2f200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30111330 con 0x7f8f3010da60 2024-05-08T06:03:17.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.470+0000 7f8f2f200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30111630 con 0x7f8f30104680 2024-05-08T06:03:17.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.470+0000 7f8f2d400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f30104fa0 0x7f8f3019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.470+0000 7f8f2de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f30104680 0x7f8f3019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.470+0000 7f8f2de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f30104680 0x7f8f3019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:36186/0 (socket says 172.21.15.112:36186) 2024-05-08T06:03:17.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.470+0000 7f8f2de00700 1 -- 172.21.15.112:0/2946974129 learned_addr learned my addr 172.21.15.112:0/2946974129 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:17.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.471+0000 7f8f2e800700 1 --2- 172.21.15.112:0/2946974129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f3010da60 0x7f8f301977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.471+0000 7f8f2d400700 1 -- 172.21.15.112:0/2946974129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f30104680 msgr2=0x7f8f3019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:17.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.471+0000 7f8f2d400700 1 --2- 172.21.15.112:0/2946974129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f30104680 0x7f8f3019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.471+0000 7f8f2d400700 1 -- 172.21.15.112:0/2946974129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f3010da60 msgr2=0x7f8f301977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:17.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.471+0000 7f8f2d400700 1 --2- 172.21.15.112:0/2946974129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f3010da60 0x7f8f301977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.471+0000 7f8f2d400700 1 -- 172.21.15.112:0/2946974129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f30197fd0 con 0x7f8f30104fa0 2024-05-08T06:03:17.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.471+0000 7f8f2d400700 1 --2- 172.21.15.112:0/2946974129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f30104fa0 0x7f8f3019dbb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c000c00 tx=0x7f8f1c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:17.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.472+0000 7f8f26a00700 1 -- 172.21.15.112:0/2946974129 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c005280 con 0x7f8f30104fa0 2024-05-08T06:03:17.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.472+0000 7f8f26a00700 1 -- 172.21.15.112:0/2946974129 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f1c00ba10 con 0x7f8f30104fa0 2024-05-08T06:03:17.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.472+0000 7f8f26a00700 1 -- 172.21.15.112:0/2946974129 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c003ae0 con 0x7f8f30104fa0 2024-05-08T06:03:17.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.472+0000 7f8f2f200700 1 -- 172.21.15.112:0/2946974129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f30198260 con 0x7f8f30104fa0 2024-05-08T06:03:17.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.472+0000 7f8f2f200700 1 -- 172.21.15.112:0/2946974129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f301a44d0 con 0x7f8f30104fa0 2024-05-08T06:03:17.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.477+0000 7f8f3f519b80 1 Processor -- start 2024-05-08T06:03:17.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.477+0000 7f8f26a00700 1 -- 172.21.15.112:0/2946974129 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8f1c003c80 con 0x7f8f30104fa0 2024-05-08T06:03:17.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.477+0000 7f8f3f519b80 1 -- start start 2024-05-08T06:03:17.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.478+0000 7f8f3f519b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f04539da0 0x563f0453a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.478+0000 7f8f3f519b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f0453a6c0 0x563f04534430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.478+0000 7f8f3f519b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04534a50 0x563f04534ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.478+0000 7f8f3f519b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563f04486a20 con 0x563f04539da0 2024-05-08T06:03:17.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.478+0000 7f8f3f519b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563f04424760 con 0x563f04534a50 2024-05-08T06:03:17.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.478+0000 7f8f3f519b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563f044829e0 con 0x563f0453a6c0 2024-05-08T06:03:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.478+0000 7f8f2e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f04539da0 0x563f0453a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.478+0000 7f8f2de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f0453a6c0 0x563f04534430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.478+0000 7f8f2e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f04539da0 0x563f0453a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58898/0 (socket says 172.21.15.112:58898) 2024-05-08T06:03:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.478+0000 7f8f2e800700 1 -- 172.21.15.112:0/3386198772 learned_addr learned my addr 172.21.15.112:0/3386198772 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.478+0000 7f8f2d400700 1 --2- 172.21.15.112:0/3386198772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04534a50 0x563f04534ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.479+0000 7f8f2e800700 1 -- 172.21.15.112:0/3386198772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f0453a6c0 msgr2=0x563f04534430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.479+0000 7f8f2e800700 1 --2- 172.21.15.112:0/3386198772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f0453a6c0 0x563f04534430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.479+0000 7f8f2e800700 1 -- 172.21.15.112:0/3386198772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04534a50 msgr2=0x563f04534ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.479+0000 7f8f2e800700 1 --2- 172.21.15.112:0/3386198772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04534a50 0x563f04534ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.479+0000 7f8f2e800700 1 -- 172.21.15.112:0/3386198772 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f043d38d0 con 0x563f04539da0 2024-05-08T06:03:17.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.479+0000 7f8f2de00700 1 --2- 172.21.15.112:0/3386198772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f0453a6c0 0x563f04534430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:03:17.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.479+0000 7f8f2e800700 1 --2- 172.21.15.112:0/3386198772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f04539da0 0x563f0453a180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009440 tx=0x7f8f28009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcefe141dda51b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:17.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.479+0000 7f8f1b400700 1 -- 172.21.15.112:0/3386198772 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28013810 con 0x563f04539da0 2024-05-08T06:03:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.480+0000 7f8f26a00700 1 --2- 172.21.15.112:0/2946974129 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8f140b2880 0x7f8f140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.480+0000 7f8f1b400700 1 -- 172.21.15.112:0/3386198772 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f28015440 con 0x563f04539da0 2024-05-08T06:03:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.480+0000 7f8f1b400700 1 -- 172.21.15.112:0/3386198772 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28013810 con 0x563f04539da0 2024-05-08T06:03:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.480+0000 7f8f26a00700 1 -- 172.21.15.112:0/2946974129 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f8f1c0e3300 con 0x7f8f30104fa0 2024-05-08T06:03:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.480+0000 7f8f2de00700 1 --2- 172.21.15.112:0/2946974129 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8f140b2880 0x7f8f140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.481+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3386198772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f04539da0 msgr2=0x563f0453a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:17.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.481+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/3386198772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f04539da0 0x563f0453a180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009440 tx=0x7f8f28009920 comp rx=0 tx=0).stop 2024-05-08T06:03:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.481+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3386198772 shutdown_connections 2024-05-08T06:03:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.481+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/3386198772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04534a50 0x563f04534ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.481+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/3386198772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f04539da0 0x563f0453a180 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.481+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/3386198772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f0453a6c0 0x563f04534430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.481+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3386198772 >> 172.21.15.112:0/3386198772 conn(0x563f044ac6d0 msgr2=0x563f0452eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:17.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.482+0000 7f8f2de00700 1 --2- 172.21.15.112:0/2946974129 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8f140b2880 0x7f8f140b4d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x563f04530970 tx=0x7f8f2000f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:17.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.482+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3386198772 shutdown_connections 2024-05-08T06:03:17.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.483+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3386198772 wait complete. 2024-05-08T06:03:17.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.483+0000 7f8f3f519b80 1 -- start start 2024-05-08T06:03:17.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.484+0000 7f8f3f519b80 1 Processor -- start 2024-05-08T06:03:17.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.484+0000 7f8f3f519b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f04534a50 0x563f045403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.484+0000 7f8f3f519b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f0453d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.484+0000 7f8f3f519b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f0453a6c0 0x563f0453dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.485+0000 7f8f3f519b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563f04541150 con 0x563f0453a6c0 2024-05-08T06:03:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.485+0000 7f8f3f519b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563f043ecb20 con 0x563f04539da0 2024-05-08T06:03:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.485+0000 7f8f3f519b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563f045412d0 con 0x563f04534a50 2024-05-08T06:03:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.485+0000 7f8f2d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f0453d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.485+0000 7f8f2d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f0453d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:52950/0 (socket says 172.21.15.112:52950) 2024-05-08T06:03:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.485+0000 7f8f2d400700 1 -- 172.21.15.112:0/3339361444 learned_addr learned my addr 172.21.15.112:0/3339361444 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.485+0000 7f8f2de00700 1 --2- 172.21.15.112:0/3339361444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f04534a50 0x563f045403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.485+0000 7f8f2e800700 1 --2- 172.21.15.112:0/3339361444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f0453a6c0 0x563f0453dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.485+0000 7f8f3f519b80 10 client.0 _ll_register_callbacks cb 0x563f0442dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:03:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.486+0000 7f8f2d400700 1 -- 172.21.15.112:0/3339361444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f04534a50 msgr2=0x563f045403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.486+0000 7f8f2d400700 1 --2- 172.21.15.112:0/3339361444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f04534a50 0x563f045403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.486+0000 7f8f2d400700 1 -- 172.21.15.112:0/3339361444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f0453a6c0 msgr2=0x563f0453dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.486+0000 7f8f2d400700 1 --2- 172.21.15.112:0/3339361444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f0453a6c0 0x563f0453dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.486+0000 7f8f2d400700 1 -- 172.21.15.112:0/3339361444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f0444c420 con 0x563f04539da0 2024-05-08T06:03:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.486+0000 7f8f2d400700 1 --2- 172.21.15.112:0/3339361444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f0453d750 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0aa910 tx=0x7f8f1c00fb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90a930dc64eb552b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.486+0000 7f8f0fe00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:03:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.486+0000 7f8f0fe00700 1 -- 172.21.15.112:0/3339361444 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0aa0f0 con 0x563f04539da0 2024-05-08T06:03:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.486+0000 7f8f0fe00700 1 -- 172.21.15.112:0/3339361444 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f8f1c0b10d0 con 0x563f04539da0 2024-05-08T06:03:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.487+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3339361444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f28015440 con 0x563f04539da0 2024-05-08T06:03:17.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.487+0000 7f8f0fe00700 1 -- 172.21.15.112:0/3339361444 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8f1c0b3090 con 0x563f04539da0 2024-05-08T06:03:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.487+0000 7f8f3f519b80 10 client.26024 fetch_fsmap learned FSMap version 33 2024-05-08T06:03:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.487+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3339361444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563f04554160 con 0x563f04539da0 2024-05-08T06:03:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.487+0000 7f8f0fe00700 1 -- 172.21.15.112:0/3339361444 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f28015440 con 0x563f04539da0 2024-05-08T06:03:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.487+0000 7f8f0fe00700 1 -- 172.21.15.112:0/3339361444 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f8f1c005420 con 0x563f04539da0 2024-05-08T06:03:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.488+0000 7f8f3f519b80 10 client.26024 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:03:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.488+0000 7f8f3f519b80 10 client.26024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:03:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.488+0000 7f8f3f519b80 20 client.26024 populate_metadata read hostname 'smithi112' 2024-05-08T06:03:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.488+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/3339361444 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f04556720 0x563f04558bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.488+0000 7f8f3f519b80 4 client.26024 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:03:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.488+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3339361444 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f04501140 con 0x563f04556720 2024-05-08T06:03:17.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.488+0000 7f8f2de00700 1 --2- 172.21.15.112:0/3339361444 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f04556720 0x563f04558bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.489+0000 7f8f2de00700 1 --2- 172.21.15.112:0/3339361444 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f04556720 0x563f04558bc0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b4e3895ab92e416 server_cookie=131375077882ffe3 in_seq=0 out_seq=0 2024-05-08T06:03:17.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.489+0000 7f8f0fe00700 10 client.26024 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:17.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.490+0000 7f8f0fe00700 1 -- 172.21.15.112:0/3339361444 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f04501140 con 0x563f04556720 2024-05-08T06:03:17.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.490+0000 7f8f0fe00700 10 client.26024 handle_command_reply: tid=0 2024-05-08T06:03:17.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.490+0000 7f8f3f519b80 1 client.26024 shutdown 2024-05-08T06:03:17.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.490+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3339361444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 msgr2=0x563f0453d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:17.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.490+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/3339361444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f0453d750 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0aa910 tx=0x7f8f1c00fb20 comp rx=0 tx=0).stop 2024-05-08T06:03:17.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.491+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3339361444 shutdown_connections 2024-05-08T06:03:17.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.491+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/3339361444 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f04556720 0x563f04558bc0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.491+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/3339361444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f0453a6c0 0x563f0453dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.491+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/3339361444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f04534a50 0x563f045403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.491+0000 7f8f0fe00700 0 client.26024 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:17.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.491+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/3339361444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f0453d750 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.491+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3339361444 >> 172.21.15.112:0/3339361444 conn(0x563f044ac6d0 msgr2=0x563f04529750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:17.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.491+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3339361444 shutdown_connections 2024-05-08T06:03:17.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.491+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3339361444 wait complete. 2024-05-08T06:03:17.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.491+0000 7f8f3f519b80 20 client.26024 trim_cache size 0 max 16384 2024-05-08T06:03:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.520+0000 7f8f3f519b80 1 Processor -- start 2024-05-08T06:03:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.520+0000 7f8f3f519b80 1 -- start start 2024-05-08T06:03:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.521+0000 7f8f3f519b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f04534a50 0x563f04552a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.521+0000 7f8f3f519b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f04534430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.521+0000 7f8f3f519b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f0453a6c0 0x563f04537ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.521+0000 7f8f3f519b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301ae080 con 0x563f04534a50 2024-05-08T06:03:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.521+0000 7f8f3f519b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301ab910 con 0x563f04539da0 2024-05-08T06:03:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.521+0000 7f8f3f519b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301a9780 con 0x563f0453a6c0 2024-05-08T06:03:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.521+0000 7f8f2de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f04534a50 0x563f04552a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.521+0000 7f8f2d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f04534430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.521+0000 7f8f2de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f04534a50 0x563f04552a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58914/0 (socket says 172.21.15.112:58914) 2024-05-08T06:03:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.521+0000 7f8f2de00700 1 -- 172.21.15.112:0/3858749211 learned_addr learned my addr 172.21.15.112:0/3858749211 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.521+0000 7f8f2e800700 1 --2- 172.21.15.112:0/3858749211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f0453a6c0 0x563f04537ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.521+0000 7f8f2d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f04534430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:52960/0 (socket says 172.21.15.112:52960) 2024-05-08T06:03:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.522+0000 7f8f2d400700 1 -- 172.21.15.112:0/3858749211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f0453a6c0 msgr2=0x563f04537ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.522+0000 7f8f2d400700 1 --2- 172.21.15.112:0/3858749211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f0453a6c0 0x563f04537ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.522+0000 7f8f2d400700 1 -- 172.21.15.112:0/3858749211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f04534a50 msgr2=0x563f04552a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.522+0000 7f8f2d400700 1 --2- 172.21.15.112:0/3858749211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f04534a50 0x563f04552a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.522+0000 7f8f2d400700 1 -- 172.21.15.112:0/3858749211 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f04559240 con 0x563f04539da0 2024-05-08T06:03:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.522+0000 7f8f2de00700 1 --2- 172.21.15.112:0/3858749211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f04534a50 0x563f04552a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:03:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.522+0000 7f8f2d400700 1 --2- 172.21.15.112:0/3858749211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f04534430 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0e0e20 tx=0x7f8f1c0e6af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cb9b0ded330a32c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:17.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.522+0000 7f8f1aa00700 1 -- 172.21.15.112:0/3858749211 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0aa0f0 con 0x563f04539da0 2024-05-08T06:03:17.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.523+0000 7f8f1aa00700 1 -- 172.21.15.112:0/3858749211 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f1c0aa490 con 0x563f04539da0 2024-05-08T06:03:17.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.523+0000 7f8f1aa00700 1 -- 172.21.15.112:0/3858749211 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0b10d0 con 0x563f04539da0 2024-05-08T06:03:17.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.523+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3858749211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 msgr2=0x563f04534430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:17.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.523+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/3858749211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f04534430 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0e0e20 tx=0x7f8f1c0e6af0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.523+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3858749211 shutdown_connections 2024-05-08T06:03:17.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.523+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/3858749211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f0453a6c0 0x563f04537ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.523+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/3858749211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f04534a50 0x563f04552a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.523+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/3858749211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f04534430 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.523+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3858749211 >> 172.21.15.112:0/3858749211 conn(0x563f044ac6d0 msgr2=0x563f0453aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:17.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.524+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3858749211 shutdown_connections 2024-05-08T06:03:17.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.524+0000 7f8f3f519b80 1 -- 172.21.15.112:0/3858749211 wait complete. 2024-05-08T06:03:17.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.524+0000 7f8f3f519b80 1 -- start start 2024-05-08T06:03:17.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.524+0000 7f8f3f519b80 1 Processor -- start 2024-05-08T06:03:17.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.525+0000 7f8f3f519b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f04534a50 0x563f0455df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.525+0000 7f8f3f519b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f0455e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.525+0000 7f8f3f519b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f0453a6c0 0x563f0455e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.525+0000 7f8f3f519b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301a9dd0 con 0x563f0453a6c0 2024-05-08T06:03:17.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.525+0000 7f8f3f519b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301ab240 con 0x563f04539da0 2024-05-08T06:03:17.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.525+0000 7f8f3f519b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301aa4a0 con 0x563f04534a50 2024-05-08T06:03:17.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.525+0000 7f8f3f519b80 10 client.0 _ll_register_callbacks cb 0x563f0442dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:03:17.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.525+0000 7f8f2e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f0453a6c0 0x563f0455e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.525+0000 7f8f2de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f04534a50 0x563f0455df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.525+0000 7f8f2de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f04534a50 0x563f0455df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:36236/0 (socket says 172.21.15.112:36236) 2024-05-08T06:03:17.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.525+0000 7f8f2de00700 1 -- 172.21.15.112:0/1044926608 learned_addr learned my addr 172.21.15.112:0/1044926608 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:17.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.525+0000 7f8f2d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f0455e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.526+0000 7f8f2d400700 1 -- 172.21.15.112:0/1044926608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f04534a50 msgr2=0x563f0455df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:17.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.526+0000 7f8f2d400700 1 --2- 172.21.15.112:0/1044926608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f04534a50 0x563f0455df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.526+0000 7f8f2d400700 1 -- 172.21.15.112:0/1044926608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f0453a6c0 msgr2=0x563f0455e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:17.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.526+0000 7f8f2d400700 1 --2- 172.21.15.112:0/1044926608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f0453a6c0 0x563f0455e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.526+0000 7f8f2d400700 1 -- 172.21.15.112:0/1044926608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f045c0320 con 0x563f04539da0 2024-05-08T06:03:17.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.526+0000 7f8f2d400700 1 --2- 172.21.15.112:0/1044926608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f0455e4a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0e36f0 tx=0x7f8f1c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7455fa5da020f1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:17.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.526+0000 7f8f0fe00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:03:17.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.526+0000 7f8f0fe00700 1 -- 172.21.15.112:0/1044926608 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0aa0f0 con 0x563f04539da0 2024-05-08T06:03:17.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.526+0000 7f8f0fe00700 1 -- 172.21.15.112:0/1044926608 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f8f1c0b32c0 con 0x563f04539da0 2024-05-08T06:03:17.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.526+0000 7f8f3f519b80 1 -- 172.21.15.112:0/1044926608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f1c0aa490 con 0x563f04539da0 2024-05-08T06:03:17.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.527+0000 7f8f0fe00700 1 -- 172.21.15.112:0/1044926608 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8f1c0aa750 con 0x563f04539da0 2024-05-08T06:03:17.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.527+0000 7f8f3f519b80 10 client.26030 fetch_fsmap learned FSMap version 33 2024-05-08T06:03:17.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.527+0000 7f8f3f519b80 1 -- 172.21.15.112:0/1044926608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563f045c0320 con 0x563f04539da0 2024-05-08T06:03:17.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.527+0000 7f8f0fe00700 1 -- 172.21.15.112:0/1044926608 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f1c0aa490 con 0x563f04539da0 2024-05-08T06:03:17.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.527+0000 7f8f0fe00700 1 -- 172.21.15.112:0/1044926608 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f8f1c0aabe0 con 0x563f04539da0 2024-05-08T06:03:17.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.527+0000 7f8f3f519b80 10 client.26030 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T06:03:17.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.527+0000 7f8f3f519b80 10 client.26030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:03:17.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.527+0000 7f8f3f519b80 20 client.26030 populate_metadata read hostname 'smithi112' 2024-05-08T06:03:17.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.527+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/1044926608 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f045c1800 0x563f045c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:17.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.527+0000 7f8f3f519b80 4 client.26030 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T06:03:17.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.527+0000 7f8f3f519b80 1 -- 172.21.15.112:0/1044926608 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563f04494400 con 0x563f045c1800 2024-05-08T06:03:17.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.528+0000 7f8f2de00700 1 --2- 172.21.15.112:0/1044926608 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f045c1800 0x563f045c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:17.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.528+0000 7f8f2de00700 1 --2- 172.21.15.112:0/1044926608 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f045c1800 0x563f045c3bf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4f0dca06e89a4de server_cookie=a93be503118376a6 in_seq=0 out_seq=0 2024-05-08T06:03:17.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.528+0000 7f8f0fe00700 10 client.26030 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:17.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.529+0000 7f8f0fe00700 1 -- 172.21.15.112:0/1044926608 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563f04494400 con 0x563f045c1800 2024-05-08T06:03:17.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.529+0000 7f8f0fe00700 10 client.26030 handle_command_reply: tid=0 2024-05-08T06:03:17.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.529+0000 7f8f3f519b80 1 client.26030 shutdown 2024-05-08T06:03:17.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.529+0000 7f8f3f519b80 1 -- 172.21.15.112:0/1044926608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 msgr2=0x563f0455e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:17.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.529+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/1044926608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f0455e4a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0e36f0 tx=0x7f8f1c009c00 comp rx=0 tx=0).stop 2024-05-08T06:03:17.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.529+0000 7f8f3f519b80 1 -- 172.21.15.112:0/1044926608 shutdown_connections 2024-05-08T06:03:17.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.529+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/1044926608 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f045c1800 0x563f045c3bf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.529+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/1044926608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f0453a6c0 0x563f0455e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.529+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/1044926608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f04534a50 0x563f0455df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.529+0000 7f8f3f519b80 1 --2- 172.21.15.112:0/1044926608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f04539da0 0x563f0455e4a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.529+0000 7f8f0fe00700 0 client.26030 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:17.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.529+0000 7f8f3f519b80 1 -- 172.21.15.112:0/1044926608 >> 172.21.15.112:0/1044926608 conn(0x563f044ac6d0 msgr2=0x563f0453c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:17.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.529+0000 7f8f3f519b80 1 -- 172.21.15.112:0/1044926608 shutdown_connections 2024-05-08T06:03:17.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.530+0000 7f8f3f519b80 1 -- 172.21.15.112:0/1044926608 wait complete. 2024-05-08T06:03:17.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.530+0000 7f8f3f519b80 20 client.26030 trim_cache size 0 max 16384 2024-05-08T06:03:17.532 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:03:17.532 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T06:03:17.532 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-05-08T06:03:17.532 INFO:teuthology.orchestra.run.smithi112.stdout: "ae815242-035b-4772-b278-46256c6eb21d": { 2024-05-08T06:03:17.532 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-05-08T06:03:17.532 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "ae815242-035b-4772-b278-46256c6eb21d", 2024-05-08T06:03:17.532 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-05-08T06:03:17.532 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:03:17.532 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:03:17.532 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:03:17.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.531+0000 7f8f2f200700 1 -- 172.21.15.112:0/2946974129 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8f140b2880 msgr2=0x7f8f140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:17.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.531+0000 7f8f2f200700 1 --2- 172.21.15.112:0/2946974129 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8f140b2880 0x7f8f140b4d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x563f04530970 tx=0x7f8f2000f040 comp rx=0 tx=0).stop 2024-05-08T06:03:17.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.531+0000 7f8f2f200700 1 -- 172.21.15.112:0/2946974129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f30104fa0 msgr2=0x7f8f3019dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:17.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.531+0000 7f8f2f200700 1 --2- 172.21.15.112:0/2946974129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f30104fa0 0x7f8f3019dbb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c000c00 tx=0x7f8f1c00fa00 comp rx=0 tx=0).stop 2024-05-08T06:03:17.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.531+0000 7f8f2f200700 1 -- 172.21.15.112:0/2946974129 shutdown_connections 2024-05-08T06:03:17.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.531+0000 7f8f2f200700 1 --2- 172.21.15.112:0/2946974129 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f8f140b2880 0x7f8f140b4d40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.531+0000 7f8f2f200700 1 --2- 172.21.15.112:0/2946974129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f3010da60 0x7f8f301977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.531+0000 7f8f2f200700 1 --2- 172.21.15.112:0/2946974129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f30104680 0x7f8f3019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.531+0000 7f8f2f200700 1 --2- 172.21.15.112:0/2946974129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f30104fa0 0x7f8f3019dbb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:17.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.531+0000 7f8f2f200700 1 -- 172.21.15.112:0/2946974129 >> 172.21.15.112:0/2946974129 conn(0x7f8f300fde30 msgr2=0x7f8f300febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:17.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.531+0000 7f8f2f200700 1 -- 172.21.15.112:0/2946974129 shutdown_connections 2024-05-08T06:03:17.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:17.531+0000 7f8f2f200700 1 -- 172.21.15.112:0/2946974129 wait complete. 2024-05-08T06:03:19.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:18 smithi112 ceph-mon[30003]: pgmap v681: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.1 MiB/s wr, 401 op/s 2024-05-08T06:03:19.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:18 smithi150 ceph-mon[34315]: pgmap v681: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.1 MiB/s wr, 401 op/s 2024-05-08T06:03:19.203 INFO:tasks.cephfs.filesystem:scrub status for tag:ae815242-035b-4772-b278-46256c6eb21d - {'path': '/', 'tag': 'ae815242-035b-4772-b278-46256c6eb21d', 'options': 'recursive,force'} 2024-05-08T06:03:19.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:18 smithi162 ceph-mon[34800]: pgmap v681: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.1 MiB/s wr, 401 op/s 2024-05-08T06:03:21.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:20 smithi150 ceph-mon[34315]: pgmap v682: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 443 op/s 2024-05-08T06:03:21.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:20 smithi112 ceph-mon[30003]: pgmap v682: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 443 op/s 2024-05-08T06:03:21.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:20 smithi162 ceph-mon[34800]: pgmap v682: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 443 op/s 2024-05-08T06:03:21.834 INFO:tasks.workunit.client.0.smithi112.stdout:All 50000 operations completed A-OK! 2024-05-08T06:03:21.869 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T06:03:21.869 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T06:03:23.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:22 smithi150 ceph-mon[34315]: pgmap v683: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.1 MiB/s wr, 432 op/s 2024-05-08T06:03:23.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:22 smithi112 ceph-mon[30003]: pgmap v683: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.1 MiB/s wr, 432 op/s 2024-05-08T06:03:23.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:22 smithi162 ceph-mon[34800]: pgmap v683: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.1 MiB/s wr, 432 op/s 2024-05-08T06:03:25.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:24 smithi150 ceph-mon[34315]: pgmap v684: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 958 KiB/s wr, 369 op/s 2024-05-08T06:03:25.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:24 smithi112 ceph-mon[30003]: pgmap v684: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 958 KiB/s wr, 369 op/s 2024-05-08T06:03:25.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:24 smithi162 ceph-mon[34800]: pgmap v684: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 958 KiB/s wr, 369 op/s 2024-05-08T06:03:27.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:26 smithi150 ceph-mon[34315]: pgmap v685: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.2 MiB/s wr, 426 op/s 2024-05-08T06:03:27.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:26 smithi112 ceph-mon[30003]: pgmap v685: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.2 MiB/s wr, 426 op/s 2024-05-08T06:03:27.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:26 smithi162 ceph-mon[34800]: pgmap v685: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.2 MiB/s wr, 426 op/s 2024-05-08T06:03:29.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:28 smithi150 ceph-mon[34315]: pgmap v686: 97 pgs: 97 active+clean; 125 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 862 KiB/s wr, 331 op/s 2024-05-08T06:03:29.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:28 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T06:03:29.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:28 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T06:03:29.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:28 smithi112 ceph-mon[30003]: pgmap v686: 97 pgs: 97 active+clean; 125 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 862 KiB/s wr, 331 op/s 2024-05-08T06:03:29.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:28 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T06:03:29.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:28 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T06:03:29.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:28 smithi162 ceph-mon[34800]: pgmap v686: 97 pgs: 97 active+clean; 125 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 862 KiB/s wr, 331 op/s 2024-05-08T06:03:29.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:28 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T06:03:29.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:28 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T06:03:30.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:29 smithi150 ceph-mon[34315]: scrub summary: idle 2024-05-08T06:03:30.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:29 smithi112 ceph-mon[30003]: scrub summary: idle 2024-05-08T06:03:30.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:29 smithi162 ceph-mon[34800]: scrub summary: idle 2024-05-08T06:03:31.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:30 smithi150 ceph-mon[34315]: pgmap v687: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.1 MiB/s wr, 316 op/s 2024-05-08T06:03:31.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:30 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:03:31.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:30 smithi112 ceph-mon[30003]: pgmap v687: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.1 MiB/s wr, 316 op/s 2024-05-08T06:03:31.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:30 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:03:31.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:30 smithi162 ceph-mon[34800]: pgmap v687: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.1 MiB/s wr, 316 op/s 2024-05-08T06:03:31.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:30 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:03:33.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:32 smithi112 ceph-mon[30003]: pgmap v688: 97 pgs: 97 active+clean; 120 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.0 MiB/s wr, 297 op/s 2024-05-08T06:03:33.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:32 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:03:33.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:32 smithi162 ceph-mon[34800]: pgmap v688: 97 pgs: 97 active+clean; 120 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.0 MiB/s wr, 297 op/s 2024-05-08T06:03:33.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:32 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:03:33.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:32 smithi150 ceph-mon[34315]: pgmap v688: 97 pgs: 97 active+clean; 120 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.0 MiB/s wr, 297 op/s 2024-05-08T06:03:33.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:32 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:03:35.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:34 smithi112 ceph-mon[30003]: pgmap v689: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 849 KiB/s wr, 236 op/s 2024-05-08T06:03:35.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:34 smithi112 ceph-mon[30003]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:03:35.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:34 smithi162 ceph-mon[34800]: pgmap v689: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 849 KiB/s wr, 236 op/s 2024-05-08T06:03:35.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:34 smithi162 ceph-mon[34800]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:03:35.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:34 smithi150 ceph-mon[34315]: pgmap v689: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 849 KiB/s wr, 236 op/s 2024-05-08T06:03:35.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:34 smithi150 ceph-mon[34315]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:03:37.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:36 smithi112 ceph-mon[30003]: pgmap v690: 97 pgs: 97 active+clean; 124 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1007 KiB/s wr, 325 op/s 2024-05-08T06:03:37.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:36 smithi162 ceph-mon[34800]: pgmap v690: 97 pgs: 97 active+clean; 124 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1007 KiB/s wr, 325 op/s 2024-05-08T06:03:37.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:36 smithi150 ceph-mon[34315]: pgmap v690: 97 pgs: 97 active+clean; 124 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1007 KiB/s wr, 325 op/s 2024-05-08T06:03:38.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:37 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:38.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:37 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:38.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:37 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:38.939 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-05-08T06:03:38.939 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-08T06:03:39.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:38 smithi112 ceph-mon[30003]: pgmap v691: 97 pgs: 97 active+clean; 128 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 971 KiB/s wr, 302 op/s 2024-05-08T06:03:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:38 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:39.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:38 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:39.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:38 smithi162 ceph-mon[34800]: pgmap v691: 97 pgs: 97 active+clean; 128 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 971 KiB/s wr, 302 op/s 2024-05-08T06:03:39.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:38 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:39.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:38 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:39.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:38 smithi150 ceph-mon[34315]: pgmap v691: 97 pgs: 97 active+clean; 128 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 971 KiB/s wr, 302 op/s 2024-05-08T06:03:39.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:39 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:39.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:39 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:40.073 DEBUG:teuthology.parallel:result is None 2024-05-08T06:03:40.074 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T06:03:40.161 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T06:03:40.161 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-08T06:03:40.176 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-08T06:03:40.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:40 smithi150 ceph-mon[34315]: pgmap v692: 97 pgs: 97 active+clean; 121 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 345 op/s 2024-05-08T06:03:40.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:40 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:40.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:40 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:41.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:40 smithi112 ceph-mon[30003]: pgmap v692: 97 pgs: 97 active+clean; 121 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 345 op/s 2024-05-08T06:03:41.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:40 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:41.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:40 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:41.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:40 smithi162 ceph-mon[34800]: pgmap v692: 97 pgs: 97 active+clean; 121 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 345 op/s 2024-05-08T06:03:41.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:40 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:41.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:40 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:42.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:42 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:42.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:42 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:42.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:42 smithi150 ceph-mon[34315]: pgmap v693: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 966 KiB/s wr, 367 op/s 2024-05-08T06:03:42.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:42 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:42.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:42 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:43.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:43.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:43.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:42 smithi112 ceph-mon[30003]: pgmap v693: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 966 KiB/s wr, 367 op/s 2024-05-08T06:03:43.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:43.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:42 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:43.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:42 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:43.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:42 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:43.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:42 smithi162 ceph-mon[34800]: pgmap v693: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 966 KiB/s wr, 367 op/s 2024-05-08T06:03:43.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:42 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:43.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:42 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:43.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:43 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:43.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:43 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:43.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:43 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:44.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:43 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:44.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:43 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:44.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:43 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:44.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:43 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:44.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:43 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:44.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:43 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:03:44.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:44 smithi150 ceph-mon[34315]: pgmap v694: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 825 KiB/s wr, 303 op/s 2024-05-08T06:03:45.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:44 smithi112 ceph-mon[30003]: pgmap v694: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 825 KiB/s wr, 303 op/s 2024-05-08T06:03:45.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:44 smithi162 ceph-mon[34800]: pgmap v694: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 825 KiB/s wr, 303 op/s 2024-05-08T06:03:47.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:46 smithi150 ceph-mon[34315]: pgmap v695: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 827 KiB/s wr, 290 op/s 2024-05-08T06:03:47.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:46 smithi112 ceph-mon[30003]: pgmap v695: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 827 KiB/s wr, 290 op/s 2024-05-08T06:03:47.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:46 smithi162 ceph-mon[34800]: pgmap v695: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 827 KiB/s wr, 290 op/s 2024-05-08T06:03:49.118 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:48 smithi112 ceph-mon[30003]: pgmap v696: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 631 KiB/s wr, 191 op/s 2024-05-08T06:03:49.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:48 smithi150 ceph-mon[34315]: pgmap v696: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 631 KiB/s wr, 191 op/s 2024-05-08T06:03:49.204 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T06:03:49.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:48 smithi162 ceph-mon[34800]: pgmap v696: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 631 KiB/s wr, 191 op/s 2024-05-08T06:03:49.665 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:03:51.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:50 smithi112 ceph-mon[30003]: pgmap v697: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 401 KiB/s wr, 148 op/s 2024-05-08T06:03:51.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:50 smithi150 ceph-mon[34315]: pgmap v697: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 401 KiB/s wr, 148 op/s 2024-05-08T06:03:51.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:50 smithi162 ceph-mon[34800]: pgmap v697: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 401 KiB/s wr, 148 op/s 2024-05-08T06:03:52.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.022+0000 7fbc5f200700 1 -- 172.21.15.112:0/3989054160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc60103fc0 msgr2=0x7fbc60108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:52.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.022+0000 7fbc5f200700 1 --2- 172.21.15.112:0/3989054160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc60103fc0 0x7fbc60108010 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0099c0 tx=0x7fbc4c009cd0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.022+0000 7fbc5f200700 1 -- 172.21.15.112:0/3989054160 shutdown_connections 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.022+0000 7fbc5f200700 1 --2- 172.21.15.112:0/3989054160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc601086f0 0x7fbc601102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.022+0000 7fbc5f200700 1 --2- 172.21.15.112:0/3989054160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc60103610 0x7fbc601039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.022+0000 7fbc5f200700 1 --2- 172.21.15.112:0/3989054160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc60103fc0 0x7fbc60108010 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.022+0000 7fbc5f200700 1 -- 172.21.15.112:0/3989054160 >> 172.21.15.112:0/3989054160 conn(0x7fbc600feec0 msgr2=0x7fbc601012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.023+0000 7fbc5f200700 1 -- 172.21.15.112:0/3989054160 shutdown_connections 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.023+0000 7fbc5f200700 1 -- 172.21.15.112:0/3989054160 wait complete. 2024-05-08T06:03:52.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.024+0000 7fbc5f200700 1 Processor -- start 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.024+0000 7fbc5f200700 1 -- start start 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.024+0000 7fbc5f200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc60103610 0x7fbc600757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.024+0000 7fbc5f200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc60103fc0 0x7fbc60075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.024+0000 7fbc5f200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc601086f0 0x7fbc60076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.024+0000 7fbc5f200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc60113560 con 0x7fbc60103fc0 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.024+0000 7fbc5f200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbc601133e0 con 0x7fbc601086f0 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.024+0000 7fbc5f200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbc601136e0 con 0x7fbc60103610 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.024+0000 7fbc5d400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc60103fc0 0x7fbc60075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.025+0000 7fbc5de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc60103610 0x7fbc600757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.025+0000 7fbc5d400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc60103fc0 0x7fbc60075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42384/0 (socket says 172.21.15.112:42384) 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.025+0000 7fbc5d400700 1 -- 172.21.15.112:0/3335947614 learned_addr learned my addr 172.21.15.112:0/3335947614 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.025+0000 7fbc5e800700 1 --2- 172.21.15.112:0/3335947614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc601086f0 0x7fbc60076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.025+0000 7fbc5d400700 1 -- 172.21.15.112:0/3335947614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc60103610 msgr2=0x7fbc600757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.025+0000 7fbc5d400700 1 --2- 172.21.15.112:0/3335947614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc60103610 0x7fbc600757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.025+0000 7fbc5d400700 1 -- 172.21.15.112:0/3335947614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc601086f0 msgr2=0x7fbc60076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.025+0000 7fbc5d400700 1 --2- 172.21.15.112:0/3335947614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc601086f0 0x7fbc60076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.025+0000 7fbc5d400700 1 -- 172.21.15.112:0/3335947614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc601a7fb0 con 0x7fbc60103fc0 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.025+0000 7fbc5e800700 1 --2- 172.21.15.112:0/3335947614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc601086f0 0x7fbc60076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.025+0000 7fbc5d400700 1 --2- 172.21.15.112:0/3335947614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc60103fc0 0x7fbc60075ce0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c006010 tx=0x7fbc4c005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.026+0000 7fbc56a00700 1 -- 172.21.15.112:0/3335947614 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c026070 con 0x7fbc60103fc0 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.026+0000 7fbc56a00700 1 -- 172.21.15.112:0/3335947614 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc4c003c70 con 0x7fbc60103fc0 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.026+0000 7fbc56a00700 1 -- 172.21.15.112:0/3335947614 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c00fbd0 con 0x7fbc60103fc0 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.026+0000 7fbc5f200700 1 -- 172.21.15.112:0/3335947614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc601a8240 con 0x7fbc60103fc0 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.026+0000 7fbc5f200700 1 -- 172.21.15.112:0/3335947614 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc601a8690 con 0x7fbc60103fc0 2024-05-08T06:03:52.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.029+0000 7fbc56a00700 1 -- 172.21.15.112:0/3335947614 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbc4c029070 con 0x7fbc60103fc0 2024-05-08T06:03:52.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.030+0000 7fbc6f3acb80 1 Processor -- start 2024-05-08T06:03:52.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.031+0000 7fbc6f3acb80 1 -- start start 2024-05-08T06:03:52.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.031+0000 7fbc56a00700 1 --2- 172.21.15.112:0/3335947614 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbc440b2880 0x7fbc440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.031+0000 7fbc6f3acb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea9da0 0x5646fceaa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.031+0000 7fbc6f3acb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fceaa6c0 0x5646fcea4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.031+0000 7fbc56a00700 1 -- 172.21.15.112:0/3335947614 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fbc4c0e29b0 con 0x7fbc60103fc0 2024-05-08T06:03:52.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.031+0000 7fbc6f3acb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea4a50 0x5646fcea4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.031+0000 7fbc6f3acb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5646fcdf6a20 con 0x5646fcea9da0 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.031+0000 7fbc6f3acb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5646fcd94760 con 0x5646fcea4a50 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.031+0000 7fbc6f3acb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5646fcdf29e0 con 0x5646fceaa6c0 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.031+0000 7fbc5de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea9da0 0x5646fceaa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.031+0000 7fbc5e800700 1 --2- 172.21.15.112:0/3335947614 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbc440b2880 0x7fbc440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.031+0000 7fbc5de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea9da0 0x5646fceaa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42388/0 (socket says 172.21.15.112:42388) 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.031+0000 7fbc5de00700 1 -- 172.21.15.112:0/2293410280 learned_addr learned my addr 172.21.15.112:0/2293410280 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.032+0000 7fbc5d400700 1 --2- 172.21.15.112:0/2293410280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fceaa6c0 0x5646fcea4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.032+0000 7fbc5e800700 1 --2- 172.21.15.112:0/2293410280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea4a50 0x5646fcea4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.032+0000 7fbc5de00700 1 -- 172.21.15.112:0/2293410280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fceaa6c0 msgr2=0x5646fcea4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.032+0000 7fbc5de00700 1 --2- 172.21.15.112:0/2293410280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fceaa6c0 0x5646fcea4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.032+0000 7fbc5de00700 1 -- 172.21.15.112:0/2293410280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea4a50 msgr2=0x5646fcea4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.032+0000 7fbc5de00700 1 --2- 172.21.15.112:0/2293410280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea4a50 0x5646fcea4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.032+0000 7fbc5de00700 1 -- 172.21.15.112:0/2293410280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646fcd438d0 con 0x5646fcea9da0 2024-05-08T06:03:52.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.032+0000 7fbc5de00700 1 --2- 172.21.15.112:0/2293410280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea9da0 0x5646fceaa180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009d80 tx=0x7fbc50011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329b01d472befc22 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.033+0000 7fbc5e800700 1 --2- 172.21.15.112:0/3335947614 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbc440b2880 0x7fbc440b4d40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5646fcea5410 tx=0x7fbc58009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.033+0000 7fbc4b400700 1 -- 172.21.15.112:0/2293410280 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50004230 con 0x5646fcea9da0 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.033+0000 7fbc4b400700 1 -- 172.21.15.112:0/2293410280 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc500043d0 con 0x5646fcea9da0 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.033+0000 7fbc4b400700 1 -- 172.21.15.112:0/2293410280 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50019920 con 0x5646fcea9da0 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.033+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/2293410280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea9da0 msgr2=0x5646fceaa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.033+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/2293410280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea9da0 0x5646fceaa180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009d80 tx=0x7fbc50011720 comp rx=0 tx=0).stop 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.034+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/2293410280 shutdown_connections 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.034+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/2293410280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea4a50 0x5646fcea4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.034+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/2293410280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea9da0 0x5646fceaa180 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.034+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/2293410280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fceaa6c0 0x5646fcea4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.034+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/2293410280 >> 172.21.15.112:0/2293410280 conn(0x5646fce1c6d0 msgr2=0x5646fce9eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.034+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/2293410280 shutdown_connections 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.034+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/2293410280 wait complete. 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.034+0000 7fbc6f3acb80 1 -- start start 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.035+0000 7fbc6f3acb80 1 Processor -- start 2024-05-08T06:03:52.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.035+0000 7fbc6f3acb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fcea4a50 0x5646fceb0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.035+0000 7fbc6f3acb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea9da0 0x5646fcead7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.035+0000 7fbc6f3acb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fceaa6c0 0x5646fceadd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.035+0000 7fbc6f3acb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5646fceb1150 con 0x5646fceaa6c0 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.035+0000 7fbc6f3acb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5646fcd5cb20 con 0x5646fcea9da0 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.035+0000 7fbc6f3acb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5646fceb12d0 con 0x5646fcea4a50 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.035+0000 7fbc6f3acb80 10 client.0 _ll_register_callbacks cb 0x5646fcd9dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.035+0000 7fbc5d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea9da0 0x5646fcead7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.035+0000 7fbc5de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fcea4a50 0x5646fceb0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.035+0000 7fbc5d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea9da0 0x5646fcead7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:48284/0 (socket says 172.21.15.112:48284) 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.035+0000 7fbc5d400700 1 -- 172.21.15.112:0/856628720 learned_addr learned my addr 172.21.15.112:0/856628720 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.036+0000 7fbc5e800700 1 --2- 172.21.15.112:0/856628720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fceaa6c0 0x5646fceadd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.036+0000 7fbc5d400700 1 -- 172.21.15.112:0/856628720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fcea4a50 msgr2=0x5646fceb0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.036+0000 7fbc5d400700 1 --2- 172.21.15.112:0/856628720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fcea4a50 0x5646fceb0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.036+0000 7fbc5d400700 1 -- 172.21.15.112:0/856628720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fceaa6c0 msgr2=0x5646fceadd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.036+0000 7fbc5d400700 1 --2- 172.21.15.112:0/856628720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fceaa6c0 0x5646fceadd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.036+0000 7fbc5d400700 1 -- 172.21.15.112:0/856628720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646fcdbc420 con 0x5646fcea9da0 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.036+0000 7fbc5de00700 1 --2- 172.21.15.112:0/856628720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fcea4a50 0x5646fceb0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.036+0000 7fbc5d400700 1 --2- 172.21.15.112:0/856628720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea9da0 0x5646fcead7d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0e0420 tx=0x7fbc4c0a9740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ef24e617027f4ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.036+0000 7fbc3fe00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.036+0000 7fbc3fe00700 1 -- 172.21.15.112:0/856628720 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0a9ae0 con 0x5646fcea9da0 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.037+0000 7fbc3fe00700 1 -- 172.21.15.112:0/856628720 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fbc4c0b2910 con 0x5646fcea9da0 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.037+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/856628720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc500043d0 con 0x5646fcea9da0 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.037+0000 7fbc3fe00700 1 -- 172.21.15.112:0/856628720 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbc4c0aa6d0 con 0x5646fcea9da0 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.037+0000 7fbc6f3acb80 10 client.26042 fetch_fsmap learned FSMap version 36 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.037+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/856628720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5646fcebb970 con 0x5646fcea9da0 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.037+0000 7fbc3fe00700 1 -- 172.21.15.112:0/856628720 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc500043d0 con 0x5646fcea9da0 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.037+0000 7fbc3fe00700 1 -- 172.21.15.112:0/856628720 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7fbc4c0a9230 con 0x5646fcea9da0 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.037+0000 7fbc6f3acb80 10 client.26042 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.037+0000 7fbc6f3acb80 10 client.26042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.037+0000 7fbc6f3acb80 20 client.26042 populate_metadata read hostname 'smithi112' 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.037+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/856628720 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5646fcebdf30 0x5646fcec03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.037+0000 7fbc6f3acb80 4 client.26042 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.037+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/856628720 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646fce71140 con 0x5646fcebdf30 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.038+0000 7fbc5de00700 1 --2- 172.21.15.112:0/856628720 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5646fcebdf30 0x5646fcec03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.038+0000 7fbc5de00700 1 --2- 172.21.15.112:0/856628720 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5646fcebdf30 0x5646fcec03b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8bedbee91ba6fba server_cookie=89869ad9a22867fb in_seq=0 out_seq=0 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.038+0000 7fbc3fe00700 10 client.26042 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.039+0000 7fbc3fe00700 1 -- 172.21.15.112:0/856628720 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646fce71140 con 0x5646fcebdf30 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.039+0000 7fbc3fe00700 10 client.26042 handle_command_reply: tid=0 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.039+0000 7fbc6f3acb80 1 client.26042 shutdown 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.039+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/856628720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea9da0 msgr2=0x5646fcead7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.039+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/856628720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea9da0 0x5646fcead7d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0e0420 tx=0x7fbc4c0a9740 comp rx=0 tx=0).stop 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.040+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/856628720 shutdown_connections 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.040+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/856628720 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5646fcebdf30 0x5646fcec03b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.040+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/856628720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fceaa6c0 0x5646fceadd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.040+0000 7fbc3fe00700 0 client.26042 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.040+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/856628720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fcea4a50 0x5646fceb0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.040+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/856628720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea9da0 0x5646fcead7d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.040+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/856628720 >> 172.21.15.112:0/856628720 conn(0x5646fce1c6d0 msgr2=0x5646fce99750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.040+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/856628720 shutdown_connections 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.040+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/856628720 wait complete. 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.040+0000 7fbc6f3acb80 20 client.26042 trim_cache size 0 max 16384 2024-05-08T06:03:52.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.062+0000 7fbc6f3acb80 1 Processor -- start 2024-05-08T06:03:52.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.063+0000 7fbc6f3acb80 1 -- start start 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.063+0000 7fbc6f3acb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea4a50 0x5646fcea4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.063+0000 7fbc6f3acb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea9da0 0x5646fcea7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.063+0000 7fbc6f3acb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fceaa6c0 0x5646fceba1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.063+0000 7fbc6f3acb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc601b24f0 con 0x5646fcea4a50 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.063+0000 7fbc6f3acb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbc601afd20 con 0x5646fcea9da0 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.063+0000 7fbc6f3acb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbc601adb70 con 0x5646fceaa6c0 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.063+0000 7fbc5de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea4a50 0x5646fcea4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.063+0000 7fbc5de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea4a50 0x5646fcea4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42406/0 (socket says 172.21.15.112:42406) 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.063+0000 7fbc5de00700 1 -- 172.21.15.112:0/1380915898 learned_addr learned my addr 172.21.15.112:0/1380915898 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.064+0000 7fbc5d400700 1 --2- 172.21.15.112:0/1380915898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea9da0 0x5646fcea7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.064+0000 7fbc5e800700 1 --2- 172.21.15.112:0/1380915898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fceaa6c0 0x5646fceba1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.064+0000 7fbc5de00700 1 -- 172.21.15.112:0/1380915898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fceaa6c0 msgr2=0x5646fceba1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.064+0000 7fbc5de00700 1 --2- 172.21.15.112:0/1380915898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fceaa6c0 0x5646fceba1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.064+0000 7fbc5de00700 1 -- 172.21.15.112:0/1380915898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea9da0 msgr2=0x5646fcea7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.064+0000 7fbc5de00700 1 --2- 172.21.15.112:0/1380915898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea9da0 0x5646fcea7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.064+0000 7fbc5de00700 1 -- 172.21.15.112:0/1380915898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646fceae6b0 con 0x5646fcea4a50 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.064+0000 7fbc5e800700 1 --2- 172.21.15.112:0/1380915898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fceaa6c0 0x5646fceba1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.064+0000 7fbc5de00700 1 --2- 172.21.15.112:0/1380915898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea4a50 0x5646fcea4430 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009cd0 tx=0x7fbc50011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea98b0d272a2971f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.064+0000 7fbc4aa00700 1 -- 172.21.15.112:0/1380915898 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc500056e0 con 0x5646fcea4a50 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.065+0000 7fbc4aa00700 1 -- 172.21.15.112:0/1380915898 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc50031070 con 0x5646fcea4a50 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.065+0000 7fbc4aa00700 1 -- 172.21.15.112:0/1380915898 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50004a20 con 0x5646fcea4a50 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.065+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/1380915898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea4a50 msgr2=0x5646fcea4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.065+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/1380915898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea4a50 0x5646fcea4430 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009cd0 tx=0x7fbc50011720 comp rx=0 tx=0).stop 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.065+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/1380915898 shutdown_connections 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.065+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/1380915898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fceaa6c0 0x5646fceba1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.065+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/1380915898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea4a50 0x5646fcea4430 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.065+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/1380915898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fcea9da0 0x5646fcea7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.065+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/1380915898 >> 172.21.15.112:0/1380915898 conn(0x5646fce1c6d0 msgr2=0x5646fcea7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.065+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/1380915898 shutdown_connections 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.066+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/1380915898 wait complete. 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.066+0000 7fbc6f3acb80 1 -- start start 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.066+0000 7fbc6f3acb80 1 Processor -- start 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc6f3acb80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fcea4a50 0x5646fceb6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc6f3acb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea9da0 0x5646fcec5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc6f3acb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fceaa6c0 0x5646fcec5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc6f3acb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc601ae1e0 con 0x5646fcea9da0 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc6f3acb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbc601af650 con 0x5646fceaa6c0 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc6f3acb80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbc601ae8b0 con 0x5646fcea4a50 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc6f3acb80 10 client.0 _ll_register_callbacks cb 0x5646fcd9dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc5d400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea9da0 0x5646fcec5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc5de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fcea4a50 0x5646fceb6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc5d400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea9da0 0x5646fcec5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42408/0 (socket says 172.21.15.112:42408) 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc5d400700 1 -- 172.21.15.112:0/498400337 learned_addr learned my addr 172.21.15.112:0/498400337 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc5e800700 1 --2- 172.21.15.112:0/498400337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fceaa6c0 0x5646fcec5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc5d400700 1 -- 172.21.15.112:0/498400337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fcea4a50 msgr2=0x5646fceb6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc5d400700 1 --2- 172.21.15.112:0/498400337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fcea4a50 0x5646fceb6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc5d400700 1 -- 172.21.15.112:0/498400337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fceaa6c0 msgr2=0x5646fcec5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc5d400700 1 --2- 172.21.15.112:0/498400337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fceaa6c0 0x5646fcec5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.067+0000 7fbc5d400700 1 -- 172.21.15.112:0/498400337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646fcec6000 con 0x5646fcea9da0 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.068+0000 7fbc5d400700 1 --2- 172.21.15.112:0/498400337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea9da0 0x5646fcec5250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0a6570 tx=0x7fbc4c0a94c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc07844232e3fd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.068+0000 7fbc3fe00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.068+0000 7fbc3fe00700 1 -- 172.21.15.112:0/498400337 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0a9ae0 con 0x5646fcea9da0 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.068+0000 7fbc3fe00700 1 -- 172.21.15.112:0/498400337 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fbc4c0039d0 con 0x5646fcea9da0 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.068+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/498400337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc50031070 con 0x5646fcea9da0 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.068+0000 7fbc3fe00700 1 -- 172.21.15.112:0/498400337 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbc4c003e20 con 0x5646fcea9da0 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.068+0000 7fbc6f3acb80 10 client.15861 fetch_fsmap learned FSMap version 36 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.068+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/498400337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5646fcf288c0 con 0x5646fcea9da0 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.069+0000 7fbc3fe00700 1 -- 172.21.15.112:0/498400337 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc50031070 con 0x5646fcea9da0 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.069+0000 7fbc3fe00700 1 -- 172.21.15.112:0/498400337 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7fbc4c090080 con 0x5646fcea9da0 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.069+0000 7fbc6f3acb80 10 client.15861 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.069+0000 7fbc6f3acb80 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.069+0000 7fbc6f3acb80 20 client.15861 populate_metadata read hostname 'smithi112' 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.069+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/498400337 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5646fcf28e70 0x5646fcf2b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.069+0000 7fbc6f3acb80 4 client.15861 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.069+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/498400337 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646fce04400 con 0x5646fcf28e70 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.069+0000 7fbc5de00700 1 --2- 172.21.15.112:0/498400337 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5646fcf28e70 0x5646fcf2b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.070+0000 7fbc3fe00700 10 client.15861 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.070+0000 7fbc5de00700 1 --2- 172.21.15.112:0/498400337 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5646fcf28e70 0x5646fcf2b290 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2316c98ecdec543 server_cookie=7f0b0d9f612c592a in_seq=0 out_seq=0 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.070+0000 7fbc3fe00700 1 -- 172.21.15.112:0/498400337 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5646fce04400 con 0x5646fcf28e70 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.070+0000 7fbc3fe00700 10 client.15861 handle_command_reply: tid=0 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.070+0000 7fbc6f3acb80 1 client.15861 shutdown 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.070+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/498400337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea9da0 msgr2=0x5646fcec5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.070+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/498400337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea9da0 0x5646fcec5250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0a6570 tx=0x7fbc4c0a94c0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.070+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/498400337 shutdown_connections 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.070+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/498400337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646fceaa6c0 0x5646fcec5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.070+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/498400337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5646fcea4a50 0x5646fceb6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.070+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/498400337 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x5646fcf28e70 0x5646fcf2b290 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.070+0000 7fbc6f3acb80 1 --2- 172.21.15.112:0/498400337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646fcea9da0 0x5646fcec5250 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.070+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/498400337 >> 172.21.15.112:0/498400337 conn(0x5646fce1c6d0 msgr2=0x5646fceaaaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.070+0000 7fbc3fe00700 0 client.15861 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.071+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/498400337 shutdown_connections 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.071+0000 7fbc6f3acb80 1 -- 172.21.15.112:0/498400337 wait complete. 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.071+0000 7fbc6f3acb80 20 client.15861 trim_cache size 0 max 16384 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.072+0000 7fbc5f200700 1 -- 172.21.15.112:0/3335947614 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbc440b2880 msgr2=0x7fbc440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.072+0000 7fbc5f200700 1 --2- 172.21.15.112:0/3335947614 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbc440b2880 0x7fbc440b4d40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5646fcea5410 tx=0x7fbc58009920 comp rx=0 tx=0).stop 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.072+0000 7fbc5f200700 1 -- 172.21.15.112:0/3335947614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc60103fc0 msgr2=0x7fbc60075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.072+0000 7fbc5f200700 1 --2- 172.21.15.112:0/3335947614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc60103fc0 0x7fbc60075ce0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c006010 tx=0x7fbc4c005280 comp rx=0 tx=0).stop 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.072+0000 7fbc5f200700 1 -- 172.21.15.112:0/3335947614 shutdown_connections 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.072+0000 7fbc5f200700 1 --2- 172.21.15.112:0/3335947614 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbc440b2880 0x7fbc440b4d40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.072+0000 7fbc5f200700 1 --2- 172.21.15.112:0/3335947614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc601086f0 0x7fbc60076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.072+0000 7fbc5f200700 1 --2- 172.21.15.112:0/3335947614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc60103610 0x7fbc600757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.072+0000 7fbc5f200700 1 --2- 172.21.15.112:0/3335947614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc60103fc0 0x7fbc60075ce0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.072+0000 7fbc5f200700 1 -- 172.21.15.112:0/3335947614 >> 172.21.15.112:0/3335947614 conn(0x7fbc600feec0 msgr2=0x7fbc60101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:52.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.072+0000 7fbc5f200700 1 -- 172.21.15.112:0/3335947614 shutdown_connections 2024-05-08T06:03:52.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:52.072+0000 7fbc5f200700 1 -- 172.21.15.112:0/3335947614 wait complete. 2024-05-08T06:03:52.870 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T06:03:52.870 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:03:53.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:52 smithi112 ceph-mon[30003]: pgmap v698: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 130 KiB/s wr, 81 op/s 2024-05-08T06:03:53.119 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:52 smithi162 ceph-mon[34800]: pgmap v698: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 130 KiB/s wr, 81 op/s 2024-05-08T06:03:53.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:52 smithi150 ceph-mon[34315]: pgmap v698: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 130 KiB/s wr, 81 op/s 2024-05-08T06:03:53.324 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:03:55.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:55 smithi112 ceph-mon[30003]: pgmap v699: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-05-08T06:03:55.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:55 smithi162 ceph-mon[34800]: pgmap v699: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-05-08T06:03:55.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:55 smithi150 ceph-mon[34315]: pgmap v699: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-05-08T06:03:55.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.772+0000 7fa6b7800700 1 -- 172.21.15.112:0/1311358606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6b0103750 msgr2=0x7fa6b0103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:55.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.772+0000 7fa6b7800700 1 --2- 172.21.15.112:0/1311358606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6b0103750 0x7fa6b0103b30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009920 tx=0x7fa6a0009c30 comp rx=0 tx=0).stop 2024-05-08T06:03:55.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.772+0000 7fa6b7800700 1 -- 172.21.15.112:0/1311358606 shutdown_connections 2024-05-08T06:03:55.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.772+0000 7fa6b7800700 1 --2- 172.21.15.112:0/1311358606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6b0108850 0x7fa6b01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:55.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.772+0000 7fa6b7800700 1 --2- 172.21.15.112:0/1311358606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6b0103750 0x7fa6b0103b30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:55.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.772+0000 7fa6b7800700 1 --2- 172.21.15.112:0/1311358606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b0104100 0x7fa6b0108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:55.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.772+0000 7fa6b7800700 1 -- 172.21.15.112:0/1311358606 >> 172.21.15.112:0/1311358606 conn(0x7fa6b007a600 msgr2=0x7fa6b007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:55.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.772+0000 7fa6b7800700 1 -- 172.21.15.112:0/1311358606 shutdown_connections 2024-05-08T06:03:55.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.772+0000 7fa6b7800700 1 -- 172.21.15.112:0/1311358606 wait complete. 2024-05-08T06:03:55.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.774+0000 7fa6b7800700 1 Processor -- start 2024-05-08T06:03:55.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.774+0000 7fa6b7800700 1 -- start start 2024-05-08T06:03:55.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.774+0000 7fa6b7800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6b0103750 0x7fa6b00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:55.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.774+0000 7fa6b7800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6b0104100 0x7fa6b0075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:55.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.774+0000 7fa6b7800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b0108850 0x7fa6b0076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:55.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.774+0000 7fa6b7800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b0113690 con 0x7fa6b0103750 2024-05-08T06:03:55.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.774+0000 7fa6b7800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b0113510 con 0x7fa6b0104100 2024-05-08T06:03:55.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.774+0000 7fa6b7800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b0113810 con 0x7fa6b0108850 2024-05-08T06:03:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.775+0000 7fa6b4e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6b0103750 0x7fa6b00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.775+0000 7fa6b5800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b0108850 0x7fa6b0076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.775+0000 7fa6b4e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6b0103750 0x7fa6b00757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51362/0 (socket says 172.21.15.112:51362) 2024-05-08T06:03:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.775+0000 7fa6a7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6b0104100 0x7fa6b0075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.775+0000 7fa6b4e00700 1 -- 172.21.15.112:0/1524195431 learned_addr learned my addr 172.21.15.112:0/1524195431 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.775+0000 7fa6b5800700 1 -- 172.21.15.112:0/1524195431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6b0104100 msgr2=0x7fa6b0075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.775+0000 7fa6b5800700 1 --2- 172.21.15.112:0/1524195431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6b0104100 0x7fa6b0075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.775+0000 7fa6b5800700 1 -- 172.21.15.112:0/1524195431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6b0103750 msgr2=0x7fa6b00757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.775+0000 7fa6b5800700 1 --2- 172.21.15.112:0/1524195431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6b0103750 0x7fa6b00757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.775+0000 7fa6b5800700 1 -- 172.21.15.112:0/1524195431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b01a8130 con 0x7fa6b0108850 2024-05-08T06:03:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.776+0000 7fa6b5800700 1 --2- 172.21.15.112:0/1524195431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b0108850 0x7fa6b0076220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009cd0 tx=0x7fa6a800c620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.776+0000 7fa6ae000700 1 -- 172.21.15.112:0/1524195431 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800cd10 con 0x7fa6b0108850 2024-05-08T06:03:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.776+0000 7fa6ae000700 1 -- 172.21.15.112:0/1524195431 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6a8003a00 con 0x7fa6b0108850 2024-05-08T06:03:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.776+0000 7fa6ae000700 1 -- 172.21.15.112:0/1524195431 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8004520 con 0x7fa6b0108850 2024-05-08T06:03:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.776+0000 7fa6b7800700 1 -- 172.21.15.112:0/1524195431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b01a83c0 con 0x7fa6b0108850 2024-05-08T06:03:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.776+0000 7fa6b7800700 1 -- 172.21.15.112:0/1524195431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b01a88b0 con 0x7fa6b0108850 2024-05-08T06:03:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.777+0000 7fa6b7800700 1 -- 172.21.15.112:0/1524195431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b010ba90 con 0x7fa6b0108850 2024-05-08T06:03:55.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.779+0000 7fa6ae000700 1 -- 172.21.15.112:0/1524195431 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa6a801a020 con 0x7fa6b0108850 2024-05-08T06:03:55.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.781+0000 7fa6ae000700 1 --2- 172.21.15.112:0/1524195431 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa6940b2900 0x7fa6940b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:55.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.781+0000 7fa6ae000700 1 -- 172.21.15.112:0/1524195431 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fa6a80e2530 con 0x7fa6b0108850 2024-05-08T06:03:55.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.781+0000 7fa6b4e00700 1 --2- 172.21.15.112:0/1524195431 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa6940b2900 0x7fa6940b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.782+0000 7fa6b4e00700 1 --2- 172.21.15.112:0/1524195431 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa6940b2900 0x7fa6940b4dc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00098f0 tx=0x7fa6a0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:55.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:55.783+0000 7fa6ae000700 1 -- 172.21.15.112:0/1524195431 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa6a8010040 con 0x7fa6b0108850 2024-05-08T06:03:56.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:56.066+0000 7fa6b7800700 1 -- 172.21.15.112:0/1524195431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6b002e510 con 0x7fa6b0108850 2024-05-08T06:03:56.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:56.067+0000 7fa6ae000700 1 -- 172.21.15.112:0/1524195431 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11894 (secure 0 0 0) 0x7fa6a80aa1c0 con 0x7fa6b0108850 2024-05-08T06:03:56.068 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T06:03:56.068 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T06:03:33.314029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24511,24518]},"id":1}]} 2024-05-08T06:03:56.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:56.070+0000 7fa6b7800700 1 -- 172.21.15.112:0/1524195431 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa6940b2900 msgr2=0x7fa6940b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:56.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:56.070+0000 7fa6b7800700 1 --2- 172.21.15.112:0/1524195431 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa6940b2900 0x7fa6940b4dc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00098f0 tx=0x7fa6a0019040 comp rx=0 tx=0).stop 2024-05-08T06:03:56.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:56.070+0000 7fa6b7800700 1 -- 172.21.15.112:0/1524195431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b0108850 msgr2=0x7fa6b0076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:56.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:56.070+0000 7fa6b7800700 1 --2- 172.21.15.112:0/1524195431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b0108850 0x7fa6b0076220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009cd0 tx=0x7fa6a800c620 comp rx=0 tx=0).stop 2024-05-08T06:03:56.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:56.070+0000 7fa6b7800700 1 -- 172.21.15.112:0/1524195431 shutdown_connections 2024-05-08T06:03:56.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:56.070+0000 7fa6b7800700 1 --2- 172.21.15.112:0/1524195431 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fa6940b2900 0x7fa6940b4dc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:56.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:56.070+0000 7fa6b7800700 1 --2- 172.21.15.112:0/1524195431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b0108850 0x7fa6b0076220 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:56.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:56.070+0000 7fa6b7800700 1 --2- 172.21.15.112:0/1524195431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6b0103750 0x7fa6b00757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:56.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:56.070+0000 7fa6b7800700 1 --2- 172.21.15.112:0/1524195431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6b0104100 0x7fa6b0075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:56.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:56.070+0000 7fa6b7800700 1 -- 172.21.15.112:0/1524195431 >> 172.21.15.112:0/1524195431 conn(0x7fa6b007a600 msgr2=0x7fa6b0101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:56.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:56.071+0000 7fa6b7800700 1 -- 172.21.15.112:0/1524195431 shutdown_connections 2024-05-08T06:03:56.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:56.071+0000 7fa6b7800700 1 -- 172.21.15.112:0/1524195431 wait complete. 2024-05-08T06:03:56.072 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 36 2024-05-08T06:03:56.415 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:56 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1524195431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:03:56.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:56 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1524195431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:03:56.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:56 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1524195431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:03:56.941 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-08T06:03:57.393 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:03:57.545 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:57 smithi112 ceph-mon[30003]: pgmap v700: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-05-08T06:03:57.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:57 smithi162 ceph-mon[34800]: pgmap v700: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-05-08T06:03:57.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:57 smithi150 ceph-mon[34315]: pgmap v700: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-05-08T06:03:59.472 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:03:59 smithi112 ceph-mon[30003]: pgmap v701: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-05-08T06:03:59.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:03:59 smithi162 ceph-mon[34800]: pgmap v701: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-05-08T06:03:59.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:03:59 smithi150 ceph-mon[34315]: pgmap v701: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-05-08T06:03:59.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.766+0000 7fb8f1a00700 1 -- 172.21.15.112:0/2042354723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ec1060a0 msgr2=0x7fb8ec1125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:59.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.766+0000 7fb8f1a00700 1 --2- 172.21.15.112:0/2042354723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ec1060a0 0x7fb8ec1125b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009f90 tx=0x7fb8e400b330 comp rx=0 tx=0).stop 2024-05-08T06:03:59.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.766+0000 7fb8f1a00700 1 -- 172.21.15.112:0/2042354723 shutdown_connections 2024-05-08T06:03:59.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.766+0000 7fb8f1a00700 1 --2- 172.21.15.112:0/2042354723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ec1060a0 0x7fb8ec1125b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.766+0000 7fb8f1a00700 1 --2- 172.21.15.112:0/2042354723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec10b6e0 0x7fb8ec10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.766+0000 7fb8f1a00700 1 --2- 172.21.15.112:0/2042354723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8ec1056e0 0x7fb8ec105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.766+0000 7fb8f1a00700 1 -- 172.21.15.112:0/2042354723 >> 172.21.15.112:0/2042354723 conn(0x7fb8ec1011d0 msgr2=0x7fb8ec1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:59.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.766+0000 7fb8f1a00700 1 -- 172.21.15.112:0/2042354723 shutdown_connections 2024-05-08T06:03:59.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.766+0000 7fb8f1a00700 1 -- 172.21.15.112:0/2042354723 wait complete. 2024-05-08T06:03:59.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.767+0000 7fb8f1a00700 1 Processor -- start 2024-05-08T06:03:59.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.767+0000 7fb8f1a00700 1 -- start start 2024-05-08T06:03:59.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.767+0000 7fb8f1a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec1056e0 0x7fb8ec19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.767+0000 7fb8f1a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8ec1060a0 0x7fb8ec19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.767+0000 7fb8f1a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ec10b6e0 0x7fb8ec1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.767+0000 7fb8f1a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec115930 con 0x7fb8ec10b6e0 2024-05-08T06:03:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.767+0000 7fb8f1a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1157b0 con 0x7fb8ec1060a0 2024-05-08T06:03:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.767+0000 7fb8f1a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec115ab0 con 0x7fb8ec1056e0 2024-05-08T06:03:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.768+0000 7fb8eaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec1056e0 0x7fb8ec19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.768+0000 7fb8eaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec1056e0 0x7fb8ec19bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:44910/0 (socket says 172.21.15.112:44910) 2024-05-08T06:03:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.768+0000 7fb8eaa00700 1 -- 172.21.15.112:0/375558396 learned_addr learned my addr 172.21.15.112:0/375558396 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.768+0000 7fb8ea000700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8ec1060a0 0x7fb8ec19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.768+0000 7fb8eb400700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ec10b6e0 0x7fb8ec1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.768+0000 7fb8eaa00700 1 -- 172.21.15.112:0/375558396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8ec1060a0 msgr2=0x7fb8ec19c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.768+0000 7fb8eaa00700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8ec1060a0 0x7fb8ec19c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.768+0000 7fb8eaa00700 1 -- 172.21.15.112:0/375558396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ec10b6e0 msgr2=0x7fb8ec1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.768+0000 7fb8eaa00700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ec10b6e0 0x7fb8ec1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.768+0000 7fb8eaa00700 1 -- 172.21.15.112:0/375558396 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ec1a0d90 con 0x7fb8ec1056e0 2024-05-08T06:03:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.768+0000 7fb8ea000700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8ec1060a0 0x7fb8ec19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:03:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.768+0000 7fb8eb400700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ec10b6e0 0x7fb8ec1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:03:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.768+0000 7fb8eaa00700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec1056e0 0x7fb8ec19bd80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00e9c0 tx=0x7fb8dc00ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.769+0000 7fb8e3400700 1 -- 172.21.15.112:0/375558396 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00cbd0 con 0x7fb8ec1056e0 2024-05-08T06:03:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.769+0000 7fb8e3400700 1 -- 172.21.15.112:0/375558396 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8dc00cd70 con 0x7fb8ec1056e0 2024-05-08T06:03:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.769+0000 7fb8e3400700 1 -- 172.21.15.112:0/375558396 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc010430 con 0x7fb8ec1056e0 2024-05-08T06:03:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.769+0000 7fb8f1a00700 1 -- 172.21.15.112:0/375558396 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ec1a1080 con 0x7fb8ec1056e0 2024-05-08T06:03:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.769+0000 7fb8f1a00700 1 -- 172.21.15.112:0/375558396 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ec1a8900 con 0x7fb8ec1056e0 2024-05-08T06:03:59.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.772+0000 7fb8e3400700 1 -- 172.21.15.112:0/375558396 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb8dc004750 con 0x7fb8ec1056e0 2024-05-08T06:03:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.773+0000 7fb8fc429b80 1 Processor -- start 2024-05-08T06:03:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.774+0000 7fb8fc429b80 1 -- start start 2024-05-08T06:03:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.774+0000 7fb8fc429b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 0x5593bcfe5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.774+0000 7fb8fc429b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe64e0 0x5593bcfe02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.774+0000 7fb8fc429b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe08f0 0x5593bcfe0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.774+0000 7fb8fc429b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5593bcf32a20 con 0x5593bcfe5bc0 2024-05-08T06:03:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.774+0000 7fb8fc429b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5593bced0760 con 0x5593bcfe08f0 2024-05-08T06:03:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.774+0000 7fb8fc429b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5593bcf2e9e0 con 0x5593bcfe64e0 2024-05-08T06:03:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.774+0000 7fb8e3400700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb8d00b2880 0x7fb8d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.774+0000 7fb8eaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe64e0 0x5593bcfe02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.774+0000 7fb8eaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe64e0 0x5593bcfe02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:44922/0 (socket says 172.21.15.112:44922) 2024-05-08T06:03:59.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.774+0000 7fb8eaa00700 1 -- 172.21.15.112:0/1598962597 learned_addr learned my addr 172.21.15.112:0/1598962597 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:59.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.775+0000 7fb8eb400700 1 --2- 172.21.15.112:0/1598962597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 0x5593bcfe5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.775+0000 7fb8e3400700 1 -- 172.21.15.112:0/375558396 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fb8dc0e2610 con 0x7fb8ec1056e0 2024-05-08T06:03:59.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.775+0000 7fb8ea000700 1 --2- 172.21.15.112:0/1598962597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe08f0 0x5593bcfe0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.775+0000 7fb8eaa00700 1 -- 172.21.15.112:0/1598962597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe08f0 msgr2=0x5593bcfe0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:59.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.775+0000 7fb8eaa00700 1 --2- 172.21.15.112:0/1598962597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe08f0 0x5593bcfe0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.775+0000 7fb8eaa00700 1 -- 172.21.15.112:0/1598962597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 msgr2=0x5593bcfe5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:59.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.775+0000 7fb8eaa00700 1 --2- 172.21.15.112:0/1598962597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 0x5593bcfe5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.775+0000 7fb8eaa00700 1 -- 172.21.15.112:0/1598962597 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593bce7f8d0 con 0x5593bcfe64e0 2024-05-08T06:03:59.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.775+0000 7fb8eaa00700 1 --2- 172.21.15.112:0/1598962597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe64e0 0x5593bcfe02d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0a9a20 tx=0x7fb8dc005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0637d0e45c40859 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.775+0000 7fb8eb400700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb8d00b2880 0x7fb8d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.775+0000 7fb8d7e00700 1 -- 172.21.15.112:0/1598962597 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc005740 con 0x5593bcfe64e0 2024-05-08T06:03:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.776+0000 7fb8d7e00700 1 -- 172.21.15.112:0/1598962597 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8dc0cae50 con 0x5593bcfe64e0 2024-05-08T06:03:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.776+0000 7fb8d7e00700 1 -- 172.21.15.112:0/1598962597 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc0b2a00 con 0x5593bcfe64e0 2024-05-08T06:03:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.776+0000 7fb8fc429b80 1 -- 172.21.15.112:0/1598962597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe64e0 msgr2=0x5593bcfe02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.776+0000 7fb8eb400700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb8d00b2880 0x7fb8d00b4d40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400b6c0 tx=0x7fb8e400b330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.776+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/1598962597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe64e0 0x5593bcfe02d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0a9a20 tx=0x7fb8dc005020 comp rx=0 tx=0).stop 2024-05-08T06:03:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.776+0000 7fb8fc429b80 1 -- 172.21.15.112:0/1598962597 shutdown_connections 2024-05-08T06:03:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.776+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/1598962597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe08f0 0x5593bcfe0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.776+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/1598962597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 0x5593bcfe5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.776+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/1598962597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe64e0 0x5593bcfe02d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.776+0000 7fb8fc429b80 1 -- 172.21.15.112:0/1598962597 >> 172.21.15.112:0/1598962597 conn(0x5593bcf586d0 msgr2=0x5593bcfdab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.777+0000 7fb8fc429b80 1 -- 172.21.15.112:0/1598962597 shutdown_connections 2024-05-08T06:03:59.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.777+0000 7fb8fc429b80 1 -- 172.21.15.112:0/1598962597 wait complete. 2024-05-08T06:03:59.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.777+0000 7fb8fc429b80 1 -- start start 2024-05-08T06:03:59.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.777+0000 7fb8fc429b80 1 Processor -- start 2024-05-08T06:03:59.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.778+0000 7fb8fc429b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe08f0 0x5593bcfec1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.778+0000 7fb8fc429b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 0x5593bcfe9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.778+0000 7fb8fc429b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe64e0 0x5593bcfe9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.778+0000 7fb8fc429b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5593bcfece60 con 0x5593bcfe5bc0 2024-05-08T06:03:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.778+0000 7fb8fc429b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5593bce98b20 con 0x5593bcfe64e0 2024-05-08T06:03:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.778+0000 7fb8fc429b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5593bcfecfe0 con 0x5593bcfe08f0 2024-05-08T06:03:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.778+0000 7fb8fc429b80 10 client.0 _ll_register_callbacks cb 0x5593bced9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:03:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.778+0000 7fb8ea000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 0x5593bcfe9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.778+0000 7fb8eaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe08f0 0x5593bcfec1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.778+0000 7fb8ea000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 0x5593bcfe9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51402/0 (socket says 172.21.15.112:51402) 2024-05-08T06:03:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.778+0000 7fb8ea000700 1 -- 172.21.15.112:0/1515087431 learned_addr learned my addr 172.21.15.112:0/1515087431 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.778+0000 7fb8eaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe08f0 0x5593bcfec1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:44938/0 (socket says 172.21.15.112:44938) 2024-05-08T06:03:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.779+0000 7fb8eb400700 1 --2- 172.21.15.112:0/1515087431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe64e0 0x5593bcfe9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.779+0000 7fb8eaa00700 1 -- 172.21.15.112:0/1515087431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe64e0 msgr2=0x5593bcfe9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.779+0000 7fb8eaa00700 1 --2- 172.21.15.112:0/1515087431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe64e0 0x5593bcfe9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.779+0000 7fb8eaa00700 1 -- 172.21.15.112:0/1515087431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 msgr2=0x5593bcfe9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:59.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.779+0000 7fb8eaa00700 1 --2- 172.21.15.112:0/1515087431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 0x5593bcfe9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.779+0000 7fb8eaa00700 1 -- 172.21.15.112:0/1515087431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593bcef8420 con 0x5593bcfe08f0 2024-05-08T06:03:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.779+0000 7fb8ea000700 1 --2- 172.21.15.112:0/1515087431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 0x5593bcfe9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:03:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.779+0000 7fb8eaa00700 1 --2- 172.21.15.112:0/1515087431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe08f0 0x5593bcfec1c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x5593bcfe68e0 tx=0x7fb8dc0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7d319c449381c1d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.779+0000 7fb8d4c00700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:03:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.779+0000 7fb8d4c00700 1 -- 172.21.15.112:0/1515087431 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc0a9ba0 con 0x5593bcfe08f0 2024-05-08T06:03:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.779+0000 7fb8fc429b80 1 -- 172.21.15.112:0/1515087431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8dc0cae50 con 0x5593bcfe08f0 2024-05-08T06:03:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.779+0000 7fb8d4c00700 1 -- 172.21.15.112:0/1515087431 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fb8dc0a9e00 con 0x5593bcfe08f0 2024-05-08T06:03:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.780+0000 7fb8d4c00700 1 -- 172.21.15.112:0/1515087431 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb8dc0a96b0 con 0x5593bcfe08f0 2024-05-08T06:03:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.780+0000 7fb8fc429b80 10 client.26101 fetch_fsmap learned FSMap version 36 2024-05-08T06:03:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.780+0000 7fb8fc429b80 1 -- 172.21.15.112:0/1515087431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5593bcfece60 con 0x5593bcfe08f0 2024-05-08T06:03:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.780+0000 7fb8d4c00700 1 -- 172.21.15.112:0/1515087431 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8dc0cae50 con 0x5593bcfe08f0 2024-05-08T06:03:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.780+0000 7fb8d4c00700 1 -- 172.21.15.112:0/1515087431 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7fb8dc0b2570 con 0x5593bcfe08f0 2024-05-08T06:03:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.780+0000 7fb8fc429b80 10 client.26101 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T06:03:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.780+0000 7fb8fc429b80 10 client.26101 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:03:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.780+0000 7fb8fc429b80 20 client.26101 populate_metadata read hostname 'smithi112' 2024-05-08T06:03:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.780+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/1515087431 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5593bcff99c0 0x5593bcffbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.780+0000 7fb8fc429b80 4 client.26101 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:03:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.780+0000 7fb8fc429b80 1 -- 172.21.15.112:0/1515087431 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593bcfad140 con 0x5593bcff99c0 2024-05-08T06:03:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.781+0000 7fb8ea000700 1 --2- 172.21.15.112:0/1515087431 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5593bcff99c0 0x5593bcffbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.781+0000 7fb8ea000700 1 --2- 172.21.15.112:0/1515087431 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5593bcff99c0 0x5593bcffbdb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=43142028b047509c server_cookie=fd31aac21ac0b993 in_seq=0 out_seq=0 2024-05-08T06:03:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.781+0000 7fb8d4c00700 10 client.26101 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T06:03:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.782+0000 7fb8d4c00700 1 -- 172.21.15.112:0/1515087431 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5593bcfad140 con 0x5593bcff99c0 2024-05-08T06:03:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.782+0000 7fb8d4c00700 10 client.26101 handle_command_reply: tid=0 2024-05-08T06:03:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.782+0000 7fb8fc429b80 1 client.26101 shutdown 2024-05-08T06:03:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.783+0000 7fb8fc429b80 1 -- 172.21.15.112:0/1515087431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe08f0 msgr2=0x5593bcfec1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.783+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/1515087431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe08f0 0x5593bcfec1c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x5593bcfe68e0 tx=0x7fb8dc0057f0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.783+0000 7fb8fc429b80 1 -- 172.21.15.112:0/1515087431 shutdown_connections 2024-05-08T06:03:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.783+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/1515087431 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5593bcff99c0 0x5593bcffbdb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.783+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/1515087431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe64e0 0x5593bcfe9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.783+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/1515087431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe08f0 0x5593bcfec1c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.783+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/1515087431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 0x5593bcfe9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.783+0000 7fb8fc429b80 1 -- 172.21.15.112:0/1515087431 >> 172.21.15.112:0/1515087431 conn(0x5593bcf586d0 msgr2=0x5593bcfda190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.783+0000 7fb8d4c00700 0 client.26101 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T06:03:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.783+0000 7fb8fc429b80 1 -- 172.21.15.112:0/1515087431 shutdown_connections 2024-05-08T06:03:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.783+0000 7fb8fc429b80 1 -- 172.21.15.112:0/1515087431 wait complete. 2024-05-08T06:03:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.783+0000 7fb8fc429b80 20 client.26101 trim_cache size 0 max 16384 2024-05-08T06:03:59.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.800+0000 7fb8fc429b80 1 Processor -- start 2024-05-08T06:03:59.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.800+0000 7fb8fc429b80 1 -- start start 2024-05-08T06:03:59.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.800+0000 7fb8fc429b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe08f0 0x5593bcfe02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.800+0000 7fb8fc429b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe5bc0 0x5593bcff5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.800+0000 7fb8fc429b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe64e0 0x5593bcfe8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.800+0000 7fb8fc429b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1b27f0 con 0x5593bcfe64e0 2024-05-08T06:03:59.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.800+0000 7fb8fc429b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1b0020 con 0x5593bcfe08f0 2024-05-08T06:03:59.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.800+0000 7fb8fc429b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1ade70 con 0x5593bcfe5bc0 2024-05-08T06:03:59.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.801+0000 7fb8eb400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe64e0 0x5593bcfe8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.801+0000 7fb8eaa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe08f0 0x5593bcfe02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.801+0000 7fb8eb400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe64e0 0x5593bcfe8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51404/0 (socket says 172.21.15.112:51404) 2024-05-08T06:03:59.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.801+0000 7fb8eaa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe08f0 0x5593bcfe02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:45366/0 (socket says 172.21.15.112:45366) 2024-05-08T06:03:59.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.801+0000 7fb8eb400700 1 -- 172.21.15.112:0/2454018976 learned_addr learned my addr 172.21.15.112:0/2454018976 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:59.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.801+0000 7fb8ea000700 1 --2- 172.21.15.112:0/2454018976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe5bc0 0x5593bcff5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.801+0000 7fb8eaa00700 1 -- 172.21.15.112:0/2454018976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe5bc0 msgr2=0x5593bcff5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:59.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.801+0000 7fb8eaa00700 1 --2- 172.21.15.112:0/2454018976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe5bc0 0x5593bcff5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.801+0000 7fb8eaa00700 1 -- 172.21.15.112:0/2454018976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe64e0 msgr2=0x5593bcfe8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:59.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.801+0000 7fb8eaa00700 1 --2- 172.21.15.112:0/2454018976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe64e0 0x5593bcfe8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.801+0000 7fb8eaa00700 1 -- 172.21.15.112:0/2454018976 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593bcfe0f20 con 0x5593bcfe08f0 2024-05-08T06:03:59.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.801+0000 7fb8ea000700 1 --2- 172.21.15.112:0/2454018976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe5bc0 0x5593bcff5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:03:59.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.801+0000 7fb8eaa00700 1 --2- 172.21.15.112:0/2454018976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe08f0 0x5593bcfe02d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc004d20 tx=0x7fb8dc005020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3583018581ce3ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:59.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.801+0000 7fb8d7400700 1 -- 172.21.15.112:0/2454018976 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc0aa2a0 con 0x5593bcfe08f0 2024-05-08T06:03:59.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.802+0000 7fb8d7400700 1 -- 172.21.15.112:0/2454018976 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8dc0cae50 con 0x5593bcfe08f0 2024-05-08T06:03:59.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.802+0000 7fb8d7400700 1 -- 172.21.15.112:0/2454018976 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc0a9ba0 con 0x5593bcfe08f0 2024-05-08T06:03:59.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.802+0000 7fb8fc429b80 1 -- 172.21.15.112:0/2454018976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe08f0 msgr2=0x5593bcfe02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:59.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.802+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/2454018976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe08f0 0x5593bcfe02d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc004d20 tx=0x7fb8dc005020 comp rx=0 tx=0).stop 2024-05-08T06:03:59.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.802+0000 7fb8fc429b80 1 -- 172.21.15.112:0/2454018976 shutdown_connections 2024-05-08T06:03:59.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.802+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/2454018976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe64e0 0x5593bcfe8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.802+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/2454018976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe08f0 0x5593bcfe02d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.802+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/2454018976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe5bc0 0x5593bcff5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.802+0000 7fb8fc429b80 1 -- 172.21.15.112:0/2454018976 >> 172.21.15.112:0/2454018976 conn(0x5593bcf586d0 msgr2=0x5593bcfe4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.802+0000 7fb8fc429b80 1 -- 172.21.15.112:0/2454018976 shutdown_connections 2024-05-08T06:03:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.802+0000 7fb8fc429b80 1 -- 172.21.15.112:0/2454018976 wait complete. 2024-05-08T06:03:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.803+0000 7fb8fc429b80 1 -- start start 2024-05-08T06:03:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.803+0000 7fb8fc429b80 1 Processor -- start 2024-05-08T06:03:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8fc429b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe08f0 0x5593bcff2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8fc429b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 0x5593bcff27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8fc429b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe64e0 0x5593bd000400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8fc429b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1ae4e0 con 0x5593bcfe5bc0 2024-05-08T06:03:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8fc429b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1af950 con 0x5593bcfe64e0 2024-05-08T06:03:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8fc429b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1aebb0 con 0x5593bcfe08f0 2024-05-08T06:03:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8fc429b80 10 client.0 _ll_register_callbacks cb 0x5593bced9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:03:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8eaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe08f0 0x5593bcff2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8ea000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 0x5593bcff27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8eaa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe08f0 0x5593bcff2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:44966/0 (socket says 172.21.15.112:44966) 2024-05-08T06:03:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8eaa00700 1 -- 172.21.15.112:0/3648844621 learned_addr learned my addr 172.21.15.112:0/3648844621 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:03:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8eb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe64e0 0x5593bd000400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8ea000700 1 -- 172.21.15.112:0/3648844621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe08f0 msgr2=0x5593bcff2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:59.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8ea000700 1 --2- 172.21.15.112:0/3648844621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe08f0 0x5593bcff2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8ea000700 1 -- 172.21.15.112:0/3648844621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe64e0 msgr2=0x5593bd000400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:03:59.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8ea000700 1 --2- 172.21.15.112:0/3648844621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe64e0 0x5593bd000400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8ea000700 1 -- 172.21.15.112:0/3648844621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593bd000c40 con 0x5593bcfe5bc0 2024-05-08T06:03:59.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.804+0000 7fb8eb400700 1 --2- 172.21.15.112:0/3648844621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe64e0 0x5593bd000400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:03:59.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.805+0000 7fb8ea000700 1 --2- 172.21.15.112:0/3648844621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 0x5593bcff27b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x5593bcffc3b0 tx=0x7fb8d8013400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc50d5c86ce0af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:59.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.805+0000 7fb8d4c00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T06:03:59.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.805+0000 7fb8d4c00700 1 -- 172.21.15.112:0/3648844621 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8026060 con 0x5593bcfe5bc0 2024-05-08T06:03:59.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.805+0000 7fb8d4c00700 1 -- 172.21.15.112:0/3648844621 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fb8d8004230 con 0x5593bcfe5bc0 2024-05-08T06:03:59.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.805+0000 7fb8fc429b80 1 -- 172.21.15.112:0/3648844621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8dc0cae50 con 0x5593bcfe5bc0 2024-05-08T06:03:59.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.805+0000 7fb8d4c00700 1 -- 172.21.15.112:0/3648844621 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb8d8013e20 con 0x5593bcfe5bc0 2024-05-08T06:03:59.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.805+0000 7fb8fc429b80 10 client.15885 fetch_fsmap learned FSMap version 36 2024-05-08T06:03:59.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.805+0000 7fb8fc429b80 1 -- 172.21.15.112:0/3648844621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5593bd063b00 con 0x5593bcfe5bc0 2024-05-08T06:03:59.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.806+0000 7fb8d4c00700 1 -- 172.21.15.112:0/3648844621 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8dc0cae50 con 0x5593bcfe5bc0 2024-05-08T06:03:59.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.806+0000 7fb8d4c00700 1 -- 172.21.15.112:0/3648844621 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7fb8d8003860 con 0x5593bcfe5bc0 2024-05-08T06:03:59.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.806+0000 7fb8fc429b80 10 client.15885 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T06:03:59.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.806+0000 7fb8fc429b80 10 client.15885 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:03:59.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.806+0000 7fb8fc429b80 20 client.15885 populate_metadata read hostname 'smithi112' 2024-05-08T06:03:59.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.806+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/3648844621 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5593bd0641b0 0x5593bd0665d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:59.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.806+0000 7fb8fc429b80 4 client.15885 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:03:59.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.806+0000 7fb8fc429b80 1 -- 172.21.15.112:0/3648844621 --> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593bcf40400 con 0x5593bd0641b0 2024-05-08T06:03:59.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.806+0000 7fb8eaa00700 1 --2- 172.21.15.112:0/3648844621 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5593bd0641b0 0x5593bd0665d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:59.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.807+0000 7fb8eaa00700 1 --2- 172.21.15.112:0/3648844621 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5593bd0641b0 0x5593bd0665d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b20409db865a047 server_cookie=1220b688fadd32d9 in_seq=0 out_seq=0 2024-05-08T06:03:59.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.807+0000 7fb8d4c00700 10 client.15885 ms_handle_connect on v2:172.21.15.162:6838/539986796 2024-05-08T06:03:59.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.807+0000 7fb8d4c00700 1 -- 172.21.15.112:0/3648844621 <== mds.2 v2:172.21.15.162:6838/539986796 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593bcf40400 con 0x5593bd0641b0 2024-05-08T06:03:59.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.807+0000 7fb8d4c00700 10 client.15885 handle_command_reply: tid=0 2024-05-08T06:03:59.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.807+0000 7fb8fc429b80 1 client.15885 shutdown 2024-05-08T06:03:59.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.807+0000 7fb8fc429b80 1 -- 172.21.15.112:0/3648844621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 msgr2=0x5593bcff27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:59.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.807+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/3648844621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 0x5593bcff27b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x5593bcffc3b0 tx=0x7fb8d8013400 comp rx=0 tx=0).stop 2024-05-08T06:03:59.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.807+0000 7fb8fc429b80 1 -- 172.21.15.112:0/3648844621 shutdown_connections 2024-05-08T06:03:59.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.807+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/3648844621 >> [v2:172.21.15.162:6838/539986796,v1:172.21.15.162:6839/539986796] conn(0x5593bd0641b0 0x5593bd0665d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.808+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/3648844621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593bcfe64e0 0x5593bd000400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.808+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/3648844621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5593bcfe08f0 0x5593bcff2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.808+0000 7fb8fc429b80 1 --2- 172.21.15.112:0/3648844621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593bcfe5bc0 0x5593bcff27b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.808+0000 7fb8fc429b80 1 -- 172.21.15.112:0/3648844621 >> 172.21.15.112:0/3648844621 conn(0x5593bcf586d0 msgr2=0x5593bcfe4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:59.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.808+0000 7fb8d4c00700 0 client.15885 ms_handle_reset on v2:172.21.15.162:6838/539986796 2024-05-08T06:03:59.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.808+0000 7fb8fc429b80 1 -- 172.21.15.112:0/3648844621 shutdown_connections 2024-05-08T06:03:59.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.808+0000 7fb8fc429b80 1 -- 172.21.15.112:0/3648844621 wait complete. 2024-05-08T06:03:59.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.808+0000 7fb8fc429b80 20 client.15885 trim_cache size 0 max 16384 2024-05-08T06:03:59.816 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T06:03:59.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.808+0000 7fb8f1a00700 1 -- 172.21.15.112:0/375558396 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb8d00b2880 msgr2=0x7fb8d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.809+0000 7fb8f1a00700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb8d00b2880 0x7fb8d00b4d40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400b6c0 tx=0x7fb8e400b330 comp rx=0 tx=0).stop 2024-05-08T06:03:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.809+0000 7fb8f1a00700 1 -- 172.21.15.112:0/375558396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec1056e0 msgr2=0x7fb8ec19bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.809+0000 7fb8f1a00700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec1056e0 0x7fb8ec19bd80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00e9c0 tx=0x7fb8dc00ed80 comp rx=0 tx=0).stop 2024-05-08T06:03:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.809+0000 7fb8f1a00700 1 -- 172.21.15.112:0/375558396 shutdown_connections 2024-05-08T06:03:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.809+0000 7fb8f1a00700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb8d00b2880 0x7fb8d00b4d40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.809+0000 7fb8f1a00700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8ec10b6e0 0x7fb8ec1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.809+0000 7fb8f1a00700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec1056e0 0x7fb8ec19bd80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.809+0000 7fb8f1a00700 1 --2- 172.21.15.112:0/375558396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8ec1060a0 0x7fb8ec19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.809+0000 7fb8f1a00700 1 -- 172.21.15.112:0/375558396 >> 172.21.15.112:0/375558396 conn(0x7fb8ec1011d0 msgr2=0x7fb8ec101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.809+0000 7fb8f1a00700 1 -- 172.21.15.112:0/375558396 shutdown_connections 2024-05-08T06:03:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:03:59.809+0000 7fb8f1a00700 1 -- 172.21.15.112:0/375558396 wait complete. 2024-05-08T06:04:00.781 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-08T06:04:01.229 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:01.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:01 smithi112 ceph-mon[30003]: pgmap v702: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-05-08T06:04:01.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:01 smithi162 ceph-mon[34800]: pgmap v702: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-05-08T06:04:01.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:01 smithi150 ceph-mon[34315]: pgmap v702: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-05-08T06:04:03.417 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:03 smithi112 ceph-mon[30003]: pgmap v703: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-08T06:04:03.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:03 smithi162 ceph-mon[34800]: pgmap v703: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-08T06:04:03.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:03 smithi150 ceph-mon[34315]: pgmap v703: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.696+0000 7fd5c0200700 1 -- 172.21.15.112:0/380671566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b8103550 msgr2=0x7fd5b81039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.696+0000 7fd5c0200700 1 --2- 172.21.15.112:0/380671566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b8103550 0x7fd5b81039b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4009990 tx=0x7fd5a4009ca0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.696+0000 7fd5c0200700 1 -- 172.21.15.112:0/380671566 shutdown_connections 2024-05-08T06:04:03.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.696+0000 7fd5c0200700 1 --2- 172.21.15.112:0/380671566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5b8103ef0 0x7fd5b81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.696+0000 7fd5c0200700 1 --2- 172.21.15.112:0/380671566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5b8108460 0x7fd5b8108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.696+0000 7fd5c0200700 1 --2- 172.21.15.112:0/380671566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b8103550 0x7fd5b81039b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.696+0000 7fd5c0200700 1 -- 172.21.15.112:0/380671566 >> 172.21.15.112:0/380671566 conn(0x7fd5b807a600 msgr2=0x7fd5b807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.696+0000 7fd5c0200700 1 -- 172.21.15.112:0/380671566 shutdown_connections 2024-05-08T06:04:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.696+0000 7fd5c0200700 1 -- 172.21.15.112:0/380671566 wait complete. 2024-05-08T06:04:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.697+0000 7fd5c0200700 1 Processor -- start 2024-05-08T06:04:03.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.697+0000 7fd5c0200700 1 -- start start 2024-05-08T06:04:03.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.698+0000 7fd5c0200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b8103550 0x7fd5b8199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.698+0000 7fd5c0200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5b8103ef0 0x7fd5b819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.698+0000 7fd5c0200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5b8108460 0x7fd5b819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.698+0000 7fd5c0200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b8113710 con 0x7fd5b8108460 2024-05-08T06:04:03.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.698+0000 7fd5c0200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b8113590 con 0x7fd5b8103550 2024-05-08T06:04:03.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.698+0000 7fd5c0200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b8113890 con 0x7fd5b8103ef0 2024-05-08T06:04:03.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.698+0000 7fd5bd800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b8103550 0x7fd5b8199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.698+0000 7fd5bd800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b8103550 0x7fd5b8199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:45394/0 (socket says 172.21.15.112:45394) 2024-05-08T06:04:03.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.698+0000 7fd5bd800700 1 -- 172.21.15.112:0/1598341862 learned_addr learned my addr 172.21.15.112:0/1598341862 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:03.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.698+0000 7fd5bce00700 1 --2- 172.21.15.112:0/1598341862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5b8103ef0 0x7fd5b819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.698+0000 7fd5be200700 1 --2- 172.21.15.112:0/1598341862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5b8108460 0x7fd5b819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.699+0000 7fd5bd800700 1 -- 172.21.15.112:0/1598341862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5b8103ef0 msgr2=0x7fd5b819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:03.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.699+0000 7fd5bd800700 1 --2- 172.21.15.112:0/1598341862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5b8103ef0 0x7fd5b819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.699+0000 7fd5bd800700 1 -- 172.21.15.112:0/1598341862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5b8108460 msgr2=0x7fd5b819e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:03.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.699+0000 7fd5bd800700 1 --2- 172.21.15.112:0/1598341862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5b8108460 0x7fd5b819e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.699+0000 7fd5bd800700 1 -- 172.21.15.112:0/1598341862 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b819ec20 con 0x7fd5b8103550 2024-05-08T06:04:03.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.699+0000 7fd5bd800700 1 --2- 172.21.15.112:0/1598341862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b8103550 0x7fd5b8199c20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00ea50 tx=0x7fd5ac00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:03.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.699+0000 7fd5b6000700 1 -- 172.21.15.112:0/1598341862 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac00cbd0 con 0x7fd5b8103550 2024-05-08T06:04:03.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.699+0000 7fd5b6000700 1 -- 172.21.15.112:0/1598341862 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ac00cd70 con 0x7fd5b8103550 2024-05-08T06:04:03.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.699+0000 7fd5b6000700 1 -- 172.21.15.112:0/1598341862 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac018750 con 0x7fd5b8103550 2024-05-08T06:04:03.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.699+0000 7fd5c0200700 1 -- 172.21.15.112:0/1598341862 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b819ef10 con 0x7fd5b8103550 2024-05-08T06:04:03.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.700+0000 7fd5c0200700 1 -- 172.21.15.112:0/1598341862 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b81a67d0 con 0x7fd5b8103550 2024-05-08T06:04:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.703+0000 7fd5b6000700 1 -- 172.21.15.112:0/1598341862 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd5ac01f020 con 0x7fd5b8103550 2024-05-08T06:04:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.704+0000 7fd5b6000700 1 --2- 172.21.15.112:0/1598341862 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd5a00b2900 0x7fd5a00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.704+0000 7fd5cab72b80 1 Processor -- start 2024-05-08T06:04:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.705+0000 7fd5cab72b80 1 -- start start 2024-05-08T06:04:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.705+0000 7fd5b6000700 1 -- 172.21.15.112:0/1598341862 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd5ac0e2360 con 0x7fd5b8103550 2024-05-08T06:04:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.705+0000 7fd5bce00700 1 --2- 172.21.15.112:0/1598341862 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd5a00b2900 0x7fd5a00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.705+0000 7fd5cab72b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 0x55ed1311afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.705+0000 7fd5cab72b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed131172d0 0x55ed13117750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.705+0000 7fd5cab72b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed13117c90 0x55ed1312bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.705+0000 7fd5cab72b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ed13069a20 con 0x55ed13117c90 2024-05-08T06:04:03.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.705+0000 7fd5cab72b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ed13007760 con 0x55ed131172d0 2024-05-08T06:04:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.705+0000 7fd5cab72b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55ed130659e0 con 0x55ed1311abd0 2024-05-08T06:04:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.705+0000 7fd5bd800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 0x55ed1311afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.705+0000 7fd5bd800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 0x55ed1311afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:44998/0 (socket says 172.21.15.112:44998) 2024-05-08T06:04:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.705+0000 7fd5bd800700 1 -- 172.21.15.112:0/2286828792 learned_addr learned my addr 172.21.15.112:0/2286828792 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.706+0000 7fd5bce00700 1 --2- 172.21.15.112:0/2286828792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed131172d0 0x55ed13117750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.706+0000 7fd5be200700 1 --2- 172.21.15.112:0/2286828792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed13117c90 0x55ed1312bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.706+0000 7fd5bd800700 1 -- 172.21.15.112:0/2286828792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed131172d0 msgr2=0x55ed13117750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.706+0000 7fd5bd800700 1 --2- 172.21.15.112:0/2286828792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed131172d0 0x55ed13117750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.706+0000 7fd5bd800700 1 -- 172.21.15.112:0/2286828792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed13117c90 msgr2=0x55ed1312bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.706+0000 7fd5bd800700 1 --2- 172.21.15.112:0/2286828792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed13117c90 0x55ed1312bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.706+0000 7fd5bd800700 1 -- 172.21.15.112:0/2286828792 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed12fb68d0 con 0x55ed1311abd0 2024-05-08T06:04:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.706+0000 7fd5be200700 1 --2- 172.21.15.112:0/2286828792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed13117c90 0x55ed1312bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:04:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.706+0000 7fd5bce00700 1 --2- 172.21.15.112:0/1598341862 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd5a00b2900 0x7fd5a00b4dc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8109c70 tx=0x7fd5a400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.706+0000 7fd5bd800700 1 --2- 172.21.15.112:0/2286828792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 0x55ed1311afb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0a9880 tx=0x7fd5ac0a90e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c006ba90736cf315 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:03.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.707+0000 7fd5aaa00700 1 -- 172.21.15.112:0/2286828792 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac08fe60 con 0x55ed1311abd0 2024-05-08T06:04:03.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.707+0000 7fd5aaa00700 1 -- 172.21.15.112:0/2286828792 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ac0b20a0 con 0x55ed1311abd0 2024-05-08T06:04:03.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.707+0000 7fd5aaa00700 1 -- 172.21.15.112:0/2286828792 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac0cc0b0 con 0x55ed1311abd0 2024-05-08T06:04:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.707+0000 7fd5cab72b80 1 -- 172.21.15.112:0/2286828792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 msgr2=0x55ed1311afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.707+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/2286828792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 0x55ed1311afb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0a9880 tx=0x7fd5ac0a90e0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.707+0000 7fd5cab72b80 1 -- 172.21.15.112:0/2286828792 shutdown_connections 2024-05-08T06:04:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.707+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/2286828792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed13117c90 0x55ed1312bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.707+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/2286828792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 0x55ed1311afb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.707+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/2286828792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed131172d0 0x55ed13117750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.707+0000 7fd5cab72b80 1 -- 172.21.15.112:0/2286828792 >> 172.21.15.112:0/2286828792 conn(0x55ed1308f6d0 msgr2=0x55ed13111b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.708+0000 7fd5cab72b80 1 -- 172.21.15.112:0/2286828792 shutdown_connections 2024-05-08T06:04:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.708+0000 7fd5cab72b80 1 -- 172.21.15.112:0/2286828792 wait complete. 2024-05-08T06:04:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.708+0000 7fd5cab72b80 1 -- start start 2024-05-08T06:04:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.708+0000 7fd5cab72b80 1 Processor -- start 2024-05-08T06:04:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.709+0000 7fd5cab72b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed131172d0 0x55ed1312bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.709+0000 7fd5cab72b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed13117c90 0x55ed13128f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.709+0000 7fd5cab72b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 0x55ed131294b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.709+0000 7fd5cab72b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ed1312c6c0 con 0x55ed131172d0 2024-05-08T06:04:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.709+0000 7fd5cab72b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ed12fcfb20 con 0x55ed13117c90 2024-05-08T06:04:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.709+0000 7fd5cab72b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55ed1312c840 con 0x55ed1311abd0 2024-05-08T06:04:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.709+0000 7fd5cab72b80 10 client.0 _ll_register_callbacks cb 0x55ed13010c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:04:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.709+0000 7fd5bd800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed131172d0 0x55ed1312bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.709+0000 7fd5bd800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed131172d0 0x55ed1312bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51446/0 (socket says 172.21.15.112:51446) 2024-05-08T06:04:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.709+0000 7fd5bd800700 1 -- 172.21.15.112:0/4020905605 learned_addr learned my addr 172.21.15.112:0/4020905605 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.709+0000 7fd5bce00700 1 --2- 172.21.15.112:0/4020905605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed13117c90 0x55ed13128f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.710+0000 7fd5be200700 1 --2- 172.21.15.112:0/4020905605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 0x55ed131294b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.710+0000 7fd5bd800700 1 -- 172.21.15.112:0/4020905605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 msgr2=0x55ed131294b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.710+0000 7fd5bd800700 1 --2- 172.21.15.112:0/4020905605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 0x55ed131294b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.710+0000 7fd5bd800700 1 -- 172.21.15.112:0/4020905605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed13117c90 msgr2=0x55ed13128f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.710+0000 7fd5bd800700 1 --2- 172.21.15.112:0/4020905605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed13117c90 0x55ed13128f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.710+0000 7fd5bd800700 1 -- 172.21.15.112:0/4020905605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed1302f420 con 0x55ed131172d0 2024-05-08T06:04:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.710+0000 7fd5bce00700 1 --2- 172.21.15.112:0/4020905605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed13117c90 0x55ed13128f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:04:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.710+0000 7fd5bd800700 1 --2- 172.21.15.112:0/4020905605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed131172d0 0x55ed1312bbd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac000c00 tx=0x7fd5ac010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ae67c08381d709 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.710+0000 7fd59f400700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T06:04:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.710+0000 7fd59f400700 1 -- 172.21.15.112:0/4020905605 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac0c2030 con 0x55ed131172d0 2024-05-08T06:04:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.710+0000 7fd59f400700 1 -- 172.21.15.112:0/4020905605 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd5ac0a5ee0 con 0x55ed131172d0 2024-05-08T06:04:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.710+0000 7fd5cab72b80 1 -- 172.21.15.112:0/4020905605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5ac0b20a0 con 0x55ed131172d0 2024-05-08T06:04:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.711+0000 7fd59f400700 1 -- 172.21.15.112:0/4020905605 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd5ac088050 con 0x55ed131172d0 2024-05-08T06:04:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.711+0000 7fd5cab72b80 10 client.15894 fetch_fsmap learned FSMap version 36 2024-05-08T06:04:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.711+0000 7fd5cab72b80 1 -- 172.21.15.112:0/4020905605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ed13136eb0 con 0x55ed131172d0 2024-05-08T06:04:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.711+0000 7fd59f400700 1 -- 172.21.15.112:0/4020905605 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ac0b20a0 con 0x55ed131172d0 2024-05-08T06:04:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.711+0000 7fd59f400700 1 -- 172.21.15.112:0/4020905605 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7fd5ac010430 con 0x55ed131172d0 2024-05-08T06:04:03.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.711+0000 7fd5cab72b80 10 client.15894 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T06:04:03.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.711+0000 7fd5cab72b80 10 client.15894 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T06:04:03.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.711+0000 7fd5cab72b80 20 client.15894 populate_metadata read hostname 'smithi112' 2024-05-08T06:04:03.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.711+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/4020905605 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ed13139380 0x55ed1313b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.711+0000 7fd5cab72b80 4 client.15894 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:04:03.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.711+0000 7fd5cab72b80 1 -- 172.21.15.112:0/4020905605 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed130e4140 con 0x55ed13139380 2024-05-08T06:04:03.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.712+0000 7fd5bce00700 1 --2- 172.21.15.112:0/4020905605 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ed13139380 0x55ed1313b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.712+0000 7fd5bce00700 1 --2- 172.21.15.112:0/4020905605 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ed13139380 0x55ed1313b770 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9bb5f5df9f6d1e04 server_cookie=3ad2a8534d082246 in_seq=0 out_seq=0 2024-05-08T06:04:03.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.712+0000 7fd59f400700 10 client.15894 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T06:04:03.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.713+0000 7fd59f400700 1 -- 172.21.15.112:0/4020905605 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ed130e4140 con 0x55ed13139380 2024-05-08T06:04:03.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.713+0000 7fd59f400700 10 client.15894 handle_command_reply: tid=0 2024-05-08T06:04:03.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.713+0000 7fd5cab72b80 1 client.15894 shutdown 2024-05-08T06:04:03.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.713+0000 7fd5cab72b80 1 -- 172.21.15.112:0/4020905605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed131172d0 msgr2=0x55ed1312bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:03.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.713+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/4020905605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed131172d0 0x55ed1312bbd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac000c00 tx=0x7fd5ac010ba0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.713+0000 7fd5cab72b80 1 -- 172.21.15.112:0/4020905605 shutdown_connections 2024-05-08T06:04:03.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.713+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/4020905605 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ed13139380 0x55ed1313b770 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.713+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/4020905605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 0x55ed131294b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.713+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/4020905605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed131172d0 0x55ed1312bbd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.713+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/4020905605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed13117c90 0x55ed13128f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.713+0000 7fd5cab72b80 1 -- 172.21.15.112:0/4020905605 >> 172.21.15.112:0/4020905605 conn(0x55ed1308f6d0 msgr2=0x55ed13111190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:03.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.714+0000 7fd59f400700 0 client.15894 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T06:04:03.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.714+0000 7fd5cab72b80 1 -- 172.21.15.112:0/4020905605 shutdown_connections 2024-05-08T06:04:03.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.714+0000 7fd5cab72b80 1 -- 172.21.15.112:0/4020905605 wait complete. 2024-05-08T06:04:03.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.714+0000 7fd5cab72b80 20 client.15894 trim_cache size 0 max 16384 2024-05-08T06:04:03.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.730+0000 7fd5cab72b80 1 Processor -- start 2024-05-08T06:04:03.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.730+0000 7fd5cab72b80 1 -- start start 2024-05-08T06:04:03.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5cab72b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed131172d0 0x55ed13124560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5cab72b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed13117c90 0x55ed13111910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5cab72b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed1311abd0 0x55ed1313e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5cab72b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81ae3d0 con 0x55ed131172d0 2024-05-08T06:04:03.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5cab72b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81abc00 con 0x55ed1311abd0 2024-05-08T06:04:03.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5cab72b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81a9a30 con 0x55ed13117c90 2024-05-08T06:04:03.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5bd800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed131172d0 0x55ed13124560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5bce00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed13117c90 0x55ed13111910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5bd800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed131172d0 0x55ed13124560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51448/0 (socket says 172.21.15.112:51448) 2024-05-08T06:04:03.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5bd800700 1 -- 172.21.15.112:0/2018394036 learned_addr learned my addr 172.21.15.112:0/2018394036 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:03.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5be200700 1 --2- 172.21.15.112:0/2018394036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed1311abd0 0x55ed1313e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5be200700 1 -- 172.21.15.112:0/2018394036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed13117c90 msgr2=0x55ed13111910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:03.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5be200700 1 --2- 172.21.15.112:0/2018394036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed13117c90 0x55ed13111910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5be200700 1 -- 172.21.15.112:0/2018394036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed131172d0 msgr2=0x55ed13124560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:03.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5be200700 1 --2- 172.21.15.112:0/2018394036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed131172d0 0x55ed13124560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5be200700 1 -- 172.21.15.112:0/2018394036 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed131128e0 con 0x55ed1311abd0 2024-05-08T06:04:03.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.731+0000 7fd5bd800700 1 --2- 172.21.15.112:0/2018394036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed131172d0 0x55ed13124560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:04:03.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.732+0000 7fd5be200700 1 --2- 172.21.15.112:0/2018394036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed1311abd0 0x55ed1313e3b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009bb0 tx=0x7fd5b0009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5ce30d5509aca04 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:03.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.732+0000 7fd5aa000700 1 -- 172.21.15.112:0/2018394036 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b001a070 con 0x55ed1311abd0 2024-05-08T06:04:03.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.732+0000 7fd5aa000700 1 -- 172.21.15.112:0/2018394036 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5b0007ce0 con 0x55ed1311abd0 2024-05-08T06:04:03.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.732+0000 7fd5aa000700 1 -- 172.21.15.112:0/2018394036 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0015620 con 0x55ed1311abd0 2024-05-08T06:04:03.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.733+0000 7fd5cab72b80 1 -- 172.21.15.112:0/2018394036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed1311abd0 msgr2=0x55ed1313e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:03.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.733+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/2018394036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed1311abd0 0x55ed1313e3b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009bb0 tx=0x7fd5b0009ec0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.733+0000 7fd5cab72b80 1 -- 172.21.15.112:0/2018394036 shutdown_connections 2024-05-08T06:04:03.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.733+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/2018394036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed1311abd0 0x55ed1313e3b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.733+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/2018394036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed131172d0 0x55ed13124560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.733+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/2018394036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed13117c90 0x55ed13111910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.733+0000 7fd5cab72b80 1 -- 172.21.15.112:0/2018394036 >> 172.21.15.112:0/2018394036 conn(0x55ed1308f6d0 msgr2=0x55ed13116b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:03.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.733+0000 7fd5cab72b80 1 -- 172.21.15.112:0/2018394036 shutdown_connections 2024-05-08T06:04:03.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.733+0000 7fd5cab72b80 1 -- 172.21.15.112:0/2018394036 wait complete. 2024-05-08T06:04:03.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.733+0000 7fd5cab72b80 1 -- start start 2024-05-08T06:04:03.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.734+0000 7fd5cab72b80 1 Processor -- start 2024-05-08T06:04:03.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.734+0000 7fd5cab72b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed131172d0 0x55ed1313e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.734+0000 7fd5cab72b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed13117c90 0x55ed1313bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.734+0000 7fd5cab72b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 0x55ed1313c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.734+0000 7fd5cab72b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81aa0c0 con 0x55ed13117c90 2024-05-08T06:04:03.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.734+0000 7fd5cab72b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81ab530 con 0x55ed131172d0 2024-05-08T06:04:03.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.734+0000 7fd5cab72b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81aa790 con 0x55ed1311abd0 2024-05-08T06:04:03.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.734+0000 7fd5cab72b80 10 client.0 _ll_register_callbacks cb 0x55ed13010c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:04:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.734+0000 7fd5bd800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed131172d0 0x55ed1313e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.734+0000 7fd5bce00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed13117c90 0x55ed1313bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.734+0000 7fd5bd800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed131172d0 0x55ed1313e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:45422/0 (socket says 172.21.15.112:45422) 2024-05-08T06:04:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.734+0000 7fd5bd800700 1 -- 172.21.15.112:0/809822215 learned_addr learned my addr 172.21.15.112:0/809822215 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.735+0000 7fd5be200700 1 --2- 172.21.15.112:0/809822215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 0x55ed1313c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.735+0000 7fd5bd800700 1 -- 172.21.15.112:0/809822215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 msgr2=0x55ed1313c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.735+0000 7fd5bd800700 1 --2- 172.21.15.112:0/809822215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 0x55ed1313c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.735+0000 7fd5bd800700 1 -- 172.21.15.112:0/809822215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed13117c90 msgr2=0x55ed1313bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.735+0000 7fd5bd800700 1 --2- 172.21.15.112:0/809822215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed13117c90 0x55ed1313bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.735+0000 7fd5bd800700 1 -- 172.21.15.112:0/809822215 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed1313c750 con 0x55ed131172d0 2024-05-08T06:04:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.735+0000 7fd5bd800700 1 --2- 172.21.15.112:0/809822215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed131172d0 0x55ed1313e7b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac010820 tx=0x7fd5ac0a9ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=110eca21ffb626fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.735+0000 7fd59f400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-08T06:04:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.735+0000 7fd59f400700 1 -- 172.21.15.112:0/809822215 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac0c2060 con 0x55ed131172d0 2024-05-08T06:04:03.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.735+0000 7fd59f400700 1 -- 172.21.15.112:0/809822215 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fd5ac0caba0 con 0x55ed131172d0 2024-05-08T06:04:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.735+0000 7fd5cab72b80 1 -- 172.21.15.112:0/809822215 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5b0007ce0 con 0x55ed131172d0 2024-05-08T06:04:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.735+0000 7fd59f400700 1 -- 172.21.15.112:0/809822215 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd5ac08fe60 con 0x55ed131172d0 2024-05-08T06:04:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.735+0000 7fd5cab72b80 10 client.26090 fetch_fsmap learned FSMap version 36 2024-05-08T06:04:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.736+0000 7fd5cab72b80 1 -- 172.21.15.112:0/809822215 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ed131a3770 con 0x55ed131172d0 2024-05-08T06:04:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.736+0000 7fd59f400700 1 -- 172.21.15.112:0/809822215 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5b0007ce0 con 0x55ed131172d0 2024-05-08T06:04:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.736+0000 7fd59f400700 1 -- 172.21.15.112:0/809822215 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7fd5ac0a85c0 con 0x55ed131172d0 2024-05-08T06:04:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.736+0000 7fd5cab72b80 10 client.26090 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T06:04:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.736+0000 7fd5cab72b80 10 client.26090 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-08T06:04:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.736+0000 7fd5cab72b80 20 client.26090 populate_metadata read hostname 'smithi112' 2024-05-08T06:04:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.736+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/809822215 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ed131a3c10 0x55ed131a6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.736+0000 7fd5cab72b80 4 client.26090 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:04:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.736+0000 7fd5cab72b80 1 -- 172.21.15.112:0/809822215 --> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed13077400 con 0x55ed131a3c10 2024-05-08T06:04:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.736+0000 7fd5bce00700 1 --2- 172.21.15.112:0/809822215 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ed131a3c10 0x55ed131a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.737+0000 7fd5bce00700 1 --2- 172.21.15.112:0/809822215 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ed131a3c10 0x55ed131a6030 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fad5dfc8735a1ce3 server_cookie=96835c76ba4c0555 in_seq=0 out_seq=0 2024-05-08T06:04:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.737+0000 7fd59f400700 10 client.26090 ms_handle_connect on v2:172.21.15.150:6838/1284437777 2024-05-08T06:04:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.737+0000 7fd59f400700 1 -- 172.21.15.112:0/809822215 <== mds.1 v2:172.21.15.150:6838/1284437777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed13077400 con 0x55ed131a3c10 2024-05-08T06:04:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.737+0000 7fd59f400700 10 client.26090 handle_command_reply: tid=0 2024-05-08T06:04:03.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.737+0000 7fd5cab72b80 1 client.26090 shutdown 2024-05-08T06:04:03.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.737+0000 7fd5cab72b80 1 -- 172.21.15.112:0/809822215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed131172d0 msgr2=0x55ed1313e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:03.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.737+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/809822215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed131172d0 0x55ed1313e7b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac010820 tx=0x7fd5ac0a9ef0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.738+0000 7fd5cab72b80 1 -- 172.21.15.112:0/809822215 shutdown_connections 2024-05-08T06:04:03.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.738+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/809822215 >> [v2:172.21.15.150:6838/1284437777,v1:172.21.15.150:6839/1284437777] conn(0x55ed131a3c10 0x55ed131a6030 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.738+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/809822215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ed1311abd0 0x55ed1313c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.738+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/809822215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed131172d0 0x55ed1313e7b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.738+0000 7fd59f400700 0 client.26090 ms_handle_reset on v2:172.21.15.150:6838/1284437777 2024-05-08T06:04:03.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.738+0000 7fd5cab72b80 1 --2- 172.21.15.112:0/809822215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed13117c90 0x55ed1313bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.738+0000 7fd5cab72b80 1 -- 172.21.15.112:0/809822215 >> 172.21.15.112:0/809822215 conn(0x55ed1308f6d0 msgr2=0x55ed13116b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:03.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.738+0000 7fd5cab72b80 1 -- 172.21.15.112:0/809822215 shutdown_connections 2024-05-08T06:04:03.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.738+0000 7fd5cab72b80 1 -- 172.21.15.112:0/809822215 wait complete. 2024-05-08T06:04:03.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.738+0000 7fd5cab72b80 20 client.26090 trim_cache size 0 max 16384 2024-05-08T06:04:03.742 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T06:04:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.739+0000 7fd5c0200700 1 -- 172.21.15.112:0/1598341862 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd5a00b2900 msgr2=0x7fd5a00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.739+0000 7fd5c0200700 1 --2- 172.21.15.112:0/1598341862 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd5a00b2900 0x7fd5a00b4dc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8109c70 tx=0x7fd5a400b260 comp rx=0 tx=0).stop 2024-05-08T06:04:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.739+0000 7fd5c0200700 1 -- 172.21.15.112:0/1598341862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b8103550 msgr2=0x7fd5b8199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.739+0000 7fd5c0200700 1 --2- 172.21.15.112:0/1598341862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b8103550 0x7fd5b8199c20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00ea50 tx=0x7fd5ac00ed60 comp rx=0 tx=0).stop 2024-05-08T06:04:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.739+0000 7fd5c0200700 1 -- 172.21.15.112:0/1598341862 shutdown_connections 2024-05-08T06:04:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.739+0000 7fd5c0200700 1 --2- 172.21.15.112:0/1598341862 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fd5a00b2900 0x7fd5a00b4dc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.739+0000 7fd5c0200700 1 --2- 172.21.15.112:0/1598341862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5b8108460 0x7fd5b819e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.739+0000 7fd5c0200700 1 --2- 172.21.15.112:0/1598341862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5b8103550 0x7fd5b8199c20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.739+0000 7fd5c0200700 1 --2- 172.21.15.112:0/1598341862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5b8103ef0 0x7fd5b819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.739+0000 7fd5c0200700 1 -- 172.21.15.112:0/1598341862 >> 172.21.15.112:0/1598341862 conn(0x7fd5b807a600 msgr2=0x7fd5b81015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.739+0000 7fd5c0200700 1 -- 172.21.15.112:0/1598341862 shutdown_connections 2024-05-08T06:04:03.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:03.739+0000 7fd5c0200700 1 -- 172.21.15.112:0/1598341862 wait complete. 2024-05-08T06:04:04.661 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-08T06:04:05.107 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:05.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:05 smithi112 ceph-mon[30003]: pgmap v704: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T06:04:05.584 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:05 smithi162 ceph-mon[34800]: pgmap v704: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T06:04:05.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:05 smithi150 ceph-mon[34315]: pgmap v704: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T06:04:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.411+0000 7f44a9e00700 1 -- 172.21.15.112:0/4155246762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a4075ff0 msgr2=0x7f44a4114130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.411+0000 7f44a9e00700 1 --2- 172.21.15.112:0/4155246762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a4075ff0 0x7f44a4114130 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f448c009990 tx=0x7f448c009ca0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.412+0000 7f44a9e00700 1 -- 172.21.15.112:0/4155246762 shutdown_connections 2024-05-08T06:04:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.412+0000 7f44a9e00700 1 --2- 172.21.15.112:0/4155246762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a4114670 0x7f44a4116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.412+0000 7f44a9e00700 1 --2- 172.21.15.112:0/4155246762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44a40756d0 0x7f44a4075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.412+0000 7f44a9e00700 1 --2- 172.21.15.112:0/4155246762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a4075ff0 0x7f44a4114130 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.412+0000 7f44a9e00700 1 -- 172.21.15.112:0/4155246762 >> 172.21.15.112:0/4155246762 conn(0x7f44a407a600 msgr2=0x7f44a407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.412+0000 7f44a9e00700 1 -- 172.21.15.112:0/4155246762 shutdown_connections 2024-05-08T06:04:07.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.412+0000 7f44a9e00700 1 -- 172.21.15.112:0/4155246762 wait complete. 2024-05-08T06:04:07.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:07 smithi112 ceph-mon[30003]: pgmap v705: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-08T06:04:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.413+0000 7f44a9e00700 1 Processor -- start 2024-05-08T06:04:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.413+0000 7f44a9e00700 1 -- start start 2024-05-08T06:04:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.413+0000 7f44a9e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44a40756d0 0x7f44a41a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.413+0000 7f44a9e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a4075ff0 0x7f44a41a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.414+0000 7f44a9e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a4114670 0x7f44a41a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.414+0000 7f44a9e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44a4119c60 con 0x7f44a4075ff0 2024-05-08T06:04:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.414+0000 7f44a9e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44a4119ae0 con 0x7f44a4114670 2024-05-08T06:04:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.414+0000 7f44a9e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f44a4119de0 con 0x7f44a40756d0 2024-05-08T06:04:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.414+0000 7f44a2a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44a40756d0 0x7f44a41a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.414+0000 7f449a000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a4075ff0 0x7f44a41a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.414+0000 7f44a2a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44a40756d0 0x7f44a41a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:44828/0 (socket says 172.21.15.112:44828) 2024-05-08T06:04:07.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.414+0000 7f44a2a00700 1 -- 172.21.15.112:0/2815766375 learned_addr learned my addr 172.21.15.112:0/2815766375 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:07.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.414+0000 7f449a000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a4075ff0 0x7f44a41a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48012/0 (socket says 172.21.15.112:48012) 2024-05-08T06:04:07.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.414+0000 7f44a3400700 1 --2- 172.21.15.112:0/2815766375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a4114670 0x7f44a41a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.414+0000 7f44a2a00700 1 -- 172.21.15.112:0/2815766375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a4114670 msgr2=0x7f44a41a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:07.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.414+0000 7f44a2a00700 1 --2- 172.21.15.112:0/2815766375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a4114670 0x7f44a41a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.414+0000 7f44a2a00700 1 -- 172.21.15.112:0/2815766375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a4075ff0 msgr2=0x7f44a41a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:07.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.414+0000 7f44a2a00700 1 --2- 172.21.15.112:0/2815766375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a4075ff0 0x7f44a41a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.414+0000 7f44a2a00700 1 -- 172.21.15.112:0/2815766375 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a41a5220 con 0x7f44a40756d0 2024-05-08T06:04:07.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.415+0000 7f44a2a00700 1 --2- 172.21.15.112:0/2815766375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44a40756d0 0x7f44a41a0210 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f449400e9a0 tx=0x7f449400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:07.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.415+0000 7f44a8a00700 1 -- 172.21.15.112:0/2815766375 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449400ca10 con 0x7f44a40756d0 2024-05-08T06:04:07.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.415+0000 7f44a8a00700 1 -- 172.21.15.112:0/2815766375 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f449400cbb0 con 0x7f44a40756d0 2024-05-08T06:04:07.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.415+0000 7f44a9e00700 1 -- 172.21.15.112:0/2815766375 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a41a5510 con 0x7f44a40756d0 2024-05-08T06:04:07.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.415+0000 7f44a8a00700 1 -- 172.21.15.112:0/2815766375 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494018680 con 0x7f44a40756d0 2024-05-08T06:04:07.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.415+0000 7f44a9e00700 1 -- 172.21.15.112:0/2815766375 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a41acd90 con 0x7f44a40756d0 2024-05-08T06:04:07.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.419+0000 7f44a8a00700 1 -- 172.21.15.112:0/2815766375 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4494018820 con 0x7f44a40756d0 2024-05-08T06:04:07.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.420+0000 7f44b46d8b80 1 Processor -- start 2024-05-08T06:04:07.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.420+0000 7f44a8a00700 1 --2- 172.21.15.112:0/2815766375 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f44880b2900 0x7f44880b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.420+0000 7f44b46d8b80 1 -- start start 2024-05-08T06:04:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.421+0000 7f44a8a00700 1 -- 172.21.15.112:0/2815766375 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f44940e23e0 con 0x7f44a40756d0 2024-05-08T06:04:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.421+0000 7f44a3400700 1 --2- 172.21.15.112:0/2815766375 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f44880b2900 0x7f44880b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.421+0000 7f44b46d8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f42803bc0 0x563f42803fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.421+0000 7f44b46d8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f428044e0 0x563f427fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.421+0000 7f44b46d8b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f427fe8f0 0x563f427fed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.421+0000 7f44b46d8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563f42750a20 con 0x563f428044e0 2024-05-08T06:04:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.421+0000 7f44b46d8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563f426ee760 con 0x563f42803bc0 2024-05-08T06:04:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.421+0000 7f44b46d8b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563f4274c9e0 con 0x563f427fe8f0 2024-05-08T06:04:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.421+0000 7f449a000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f428044e0 0x563f427fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.421+0000 7f44a2a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f42803bc0 0x563f42803fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.421+0000 7f44a2a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f42803bc0 0x563f42803fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:32902/0 (socket says 172.21.15.112:32902) 2024-05-08T06:04:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.421+0000 7f449a000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f428044e0 0x563f427fe2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48020/0 (socket says 172.21.15.112:48020) 2024-05-08T06:04:07.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.421+0000 7f44a2a00700 1 -- 172.21.15.112:0/1091755644 learned_addr learned my addr 172.21.15.112:0/1091755644 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:07.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.421+0000 7f44a3400700 1 --2- 172.21.15.112:0/1091755644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f427fe8f0 0x563f427fed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.422+0000 7f44a2a00700 1 -- 172.21.15.112:0/1091755644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f427fe8f0 msgr2=0x563f427fed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:07.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.422+0000 7f44a2a00700 1 --2- 172.21.15.112:0/1091755644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f427fe8f0 0x563f427fed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.422+0000 7f44a3400700 1 --2- 172.21.15.112:0/2815766375 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f44880b2900 0x7f44880b4dc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f44a406ca00 tx=0x7f449c009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:07.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.422+0000 7f44a2a00700 1 -- 172.21.15.112:0/1091755644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f428044e0 msgr2=0x563f427fe2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:07.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.422+0000 7f44a2a00700 1 --2- 172.21.15.112:0/1091755644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f428044e0 0x563f427fe2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.422+0000 7f44a2a00700 1 -- 172.21.15.112:0/1091755644 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f4269d8d0 con 0x563f42803bc0 2024-05-08T06:04:07.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.422+0000 7f44a2a00700 1 --2- 172.21.15.112:0/1091755644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f42803bc0 0x563f42803fa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f44940a9900 tx=0x7f4494009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccf98b8af28dd9ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:07.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.422+0000 7f4498c00700 1 -- 172.21.15.112:0/1091755644 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44940a9970 con 0x563f42803bc0 2024-05-08T06:04:07.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.422+0000 7f4498c00700 1 -- 172.21.15.112:0/1091755644 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44940add20 con 0x563f42803bc0 2024-05-08T06:04:07.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.422+0000 7f4498c00700 1 -- 172.21.15.112:0/1091755644 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44940a9bd0 con 0x563f42803bc0 2024-05-08T06:04:07.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.423+0000 7f44b46d8b80 1 -- 172.21.15.112:0/1091755644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f42803bc0 msgr2=0x563f42803fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:07.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.423+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/1091755644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f42803bc0 0x563f42803fa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f44940a9900 tx=0x7f4494009790 comp rx=0 tx=0).stop 2024-05-08T06:04:07.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.423+0000 7f44b46d8b80 1 -- 172.21.15.112:0/1091755644 shutdown_connections 2024-05-08T06:04:07.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.423+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/1091755644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f427fe8f0 0x563f427fed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.423+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/1091755644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f42803bc0 0x563f42803fa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.423+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/1091755644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f428044e0 0x563f427fe2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.423+0000 7f44b46d8b80 1 -- 172.21.15.112:0/1091755644 >> 172.21.15.112:0/1091755644 conn(0x563f427766d0 msgr2=0x563f427f8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:07.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.423+0000 7f44b46d8b80 1 -- 172.21.15.112:0/1091755644 shutdown_connections 2024-05-08T06:04:07.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.423+0000 7f44b46d8b80 1 -- 172.21.15.112:0/1091755644 wait complete. 2024-05-08T06:04:07.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.424+0000 7f44b46d8b80 1 -- start start 2024-05-08T06:04:07.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.424+0000 7f44b46d8b80 1 Processor -- start 2024-05-08T06:04:07.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.425+0000 7f44b46d8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f427fe8f0 0x563f4280a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.425+0000 7f44b46d8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f42803bc0 0x563f428075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.425+0000 7f44b46d8b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f428044e0 0x563f42807ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.425+0000 7f44b46d8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563f4280af20 con 0x563f42803bc0 2024-05-08T06:04:07.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.425+0000 7f44b46d8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563f426b6b20 con 0x563f427fe8f0 2024-05-08T06:04:07.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.425+0000 7f44b46d8b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563f4280b0a0 con 0x563f428044e0 2024-05-08T06:04:07.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.425+0000 7f44b46d8b80 10 client.0 _ll_register_callbacks cb 0x563f426f7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:04:07.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.425+0000 7f449a000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f42803bc0 0x563f428075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.425+0000 7f44a2a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f427fe8f0 0x563f4280a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.425+0000 7f449a000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f42803bc0 0x563f428075a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48030/0 (socket says 172.21.15.112:48030) 2024-05-08T06:04:07.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.425+0000 7f44a2a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f427fe8f0 0x563f4280a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:32908/0 (socket says 172.21.15.112:32908) 2024-05-08T06:04:07.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.425+0000 7f449a000700 1 -- 172.21.15.112:0/2024361602 learned_addr learned my addr 172.21.15.112:0/2024361602 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:07.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.425+0000 7f44a3400700 1 --2- 172.21.15.112:0/2024361602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f428044e0 0x563f42807ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.426+0000 7f44a3400700 1 -- 172.21.15.112:0/2024361602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f427fe8f0 msgr2=0x563f4280a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:07.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.426+0000 7f44a3400700 1 --2- 172.21.15.112:0/2024361602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f427fe8f0 0x563f4280a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.426+0000 7f44a3400700 1 -- 172.21.15.112:0/2024361602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f42803bc0 msgr2=0x563f428075a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:07.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.426+0000 7f44a3400700 1 --2- 172.21.15.112:0/2024361602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f42803bc0 0x563f428075a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.426+0000 7f44a3400700 1 -- 172.21.15.112:0/2024361602 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f42716420 con 0x563f428044e0 2024-05-08T06:04:07.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.426+0000 7f44a2a00700 1 --2- 172.21.15.112:0/2024361602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f427fe8f0 0x563f4280a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:04:07.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.426+0000 7f44a3400700 1 --2- 172.21.15.112:0/2024361602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f428044e0 0x563f42807ae0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f449c00f400 tx=0x7f449c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3092db6c30bc6657 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:07.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.426+0000 7f4491600700 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-05-08T06:04:07.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.426+0000 7f4491600700 1 -- 172.21.15.112:0/2024361602 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c026030 con 0x563f428044e0 2024-05-08T06:04:07.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.426+0000 7f44b46d8b80 1 -- 172.21.15.112:0/2024361602 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44940add20 con 0x563f428044e0 2024-05-08T06:04:07.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.427+0000 7f4491600700 1 -- 172.21.15.112:0/2024361602 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f449c004bb0 con 0x563f428044e0 2024-05-08T06:04:07.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.427+0000 7f4491600700 1 -- 172.21.15.112:0/2024361602 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f449c032050 con 0x563f428044e0 2024-05-08T06:04:07.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.427+0000 7f44b46d8b80 10 client.26137 fetch_fsmap learned FSMap version 36 2024-05-08T06:04:07.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.427+0000 7f44b46d8b80 1 -- 172.21.15.112:0/2024361602 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563f42815880 con 0x563f428044e0 2024-05-08T06:04:07.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.427+0000 7f4491600700 1 -- 172.21.15.112:0/2024361602 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44940add20 con 0x563f428044e0 2024-05-08T06:04:07.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.427+0000 7f4491600700 1 -- 172.21.15.112:0/2024361602 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f449c022800 con 0x563f428044e0 2024-05-08T06:04:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.428+0000 7f44b46d8b80 10 client.26137 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T06:04:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.428+0000 7f44b46d8b80 10 client.26137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:04:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.428+0000 7f44b46d8b80 20 client.26137 populate_metadata read hostname 'smithi112' 2024-05-08T06:04:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.428+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/2024361602 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f42817e60 0x563f4281a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.428+0000 7f44b46d8b80 4 client.26137 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:04:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.428+0000 7f44b46d8b80 1 -- 172.21.15.112:0/2024361602 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f427cb140 con 0x563f42817e60 2024-05-08T06:04:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.428+0000 7f44a2a00700 1 --2- 172.21.15.112:0/2024361602 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f42817e60 0x563f4281a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.428+0000 7f44a2a00700 1 --2- 172.21.15.112:0/2024361602 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f42817e60 0x563f4281a250 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=960f390eaa9b90ce server_cookie=fc53fde06b35195b in_seq=0 out_seq=0 2024-05-08T06:04:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.428+0000 7f4491600700 10 client.26137 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:04:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.429+0000 7f4491600700 1 -- 172.21.15.112:0/2024361602 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f427cb140 con 0x563f42817e60 2024-05-08T06:04:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.429+0000 7f4491600700 10 client.26137 handle_command_reply: tid=0 2024-05-08T06:04:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.429+0000 7f44b46d8b80 1 client.26137 shutdown 2024-05-08T06:04:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.430+0000 7f44b46d8b80 1 -- 172.21.15.112:0/2024361602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f428044e0 msgr2=0x563f42807ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.430+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/2024361602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f428044e0 0x563f42807ae0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f449c00f400 tx=0x7f449c004b40 comp rx=0 tx=0).stop 2024-05-08T06:04:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.430+0000 7f44b46d8b80 1 -- 172.21.15.112:0/2024361602 shutdown_connections 2024-05-08T06:04:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.430+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/2024361602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f428044e0 0x563f42807ae0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.430+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/2024361602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f427fe8f0 0x563f4280a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.430+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/2024361602 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f42817e60 0x563f4281a250 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.430+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/2024361602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f42803bc0 0x563f428075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.430+0000 7f44b46d8b80 1 -- 172.21.15.112:0/2024361602 >> 172.21.15.112:0/2024361602 conn(0x563f427766d0 msgr2=0x563f427f8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.430+0000 7f4491600700 0 client.26137 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:04:07.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.430+0000 7f44b46d8b80 1 -- 172.21.15.112:0/2024361602 shutdown_connections 2024-05-08T06:04:07.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.430+0000 7f44b46d8b80 1 -- 172.21.15.112:0/2024361602 wait complete. 2024-05-08T06:04:07.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.430+0000 7f44b46d8b80 20 client.26137 trim_cache size 0 max 16384 2024-05-08T06:04:07.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.446+0000 7f44b46d8b80 1 Processor -- start 2024-05-08T06:04:07.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.447+0000 7f44b46d8b80 1 -- start start 2024-05-08T06:04:07.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.447+0000 7f44b46d8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f427fe8f0 0x563f427fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.447+0000 7f44b46d8b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f42803bc0 0x563f427fa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.447+0000 7f44b46d8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f428044e0 0x563f42807bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.447+0000 7f44b46d8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44a41b6b70 con 0x563f427fe8f0 2024-05-08T06:04:07.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.447+0000 7f44b46d8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44a41b43a0 con 0x563f428044e0 2024-05-08T06:04:07.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.447+0000 7f44b46d8b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f44a41b21f0 con 0x563f42803bc0 2024-05-08T06:04:07.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.447+0000 7f44a2a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f427fe8f0 0x563f427fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.447+0000 7f44a2a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f427fe8f0 0x563f427fe2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48046/0 (socket says 172.21.15.112:48046) 2024-05-08T06:04:07.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.447+0000 7f44a2a00700 1 -- 172.21.15.112:0/2480942219 learned_addr learned my addr 172.21.15.112:0/2480942219 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:07.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.447+0000 7f449a000700 1 --2- 172.21.15.112:0/2480942219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f42803bc0 0x563f427fa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.447+0000 7f44a3400700 1 --2- 172.21.15.112:0/2480942219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f428044e0 0x563f42807bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.448+0000 7f44a2a00700 1 -- 172.21.15.112:0/2480942219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f42803bc0 msgr2=0x563f427fa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:07.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.448+0000 7f44a2a00700 1 --2- 172.21.15.112:0/2480942219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f42803bc0 0x563f427fa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.448+0000 7f44a2a00700 1 -- 172.21.15.112:0/2480942219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f428044e0 msgr2=0x563f42807bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:07.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.448+0000 7f44a2a00700 1 --2- 172.21.15.112:0/2480942219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f428044e0 0x563f42807bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.448+0000 7f44a2a00700 1 -- 172.21.15.112:0/2480942219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f427f9820 con 0x563f427fe8f0 2024-05-08T06:04:07.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.448+0000 7f449a000700 1 --2- 172.21.15.112:0/2480942219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f42803bc0 0x563f427fa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:04:07.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.448+0000 7f44a2a00700 1 --2- 172.21.15.112:0/2480942219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f427fe8f0 0x563f427fe2d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f449400c2d0 tx=0x7f4494091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968784225bcc3f6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:07.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.448+0000 7f4493e00700 1 -- 172.21.15.112:0/2480942219 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44940add20 con 0x563f427fe8f0 2024-05-08T06:04:07.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.448+0000 7f4493e00700 1 -- 172.21.15.112:0/2480942219 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44940a9970 con 0x563f427fe8f0 2024-05-08T06:04:07.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.448+0000 7f4493e00700 1 -- 172.21.15.112:0/2480942219 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44940b2340 con 0x563f427fe8f0 2024-05-08T06:04:07.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.449+0000 7f44b46d8b80 1 -- 172.21.15.112:0/2480942219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f427fe8f0 msgr2=0x563f427fe2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:07.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.449+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/2480942219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f427fe8f0 0x563f427fe2d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f449400c2d0 tx=0x7f4494091040 comp rx=0 tx=0).stop 2024-05-08T06:04:07.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.449+0000 7f44b46d8b80 1 -- 172.21.15.112:0/2480942219 shutdown_connections 2024-05-08T06:04:07.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.449+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/2480942219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f428044e0 0x563f42807bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.449+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/2480942219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f427fe8f0 0x563f427fe2d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.449+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/2480942219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f42803bc0 0x563f427fa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.449+0000 7f44b46d8b80 1 -- 172.21.15.112:0/2480942219 >> 172.21.15.112:0/2480942219 conn(0x563f427766d0 msgr2=0x563f42805eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:07.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.449+0000 7f44b46d8b80 1 -- 172.21.15.112:0/2480942219 shutdown_connections 2024-05-08T06:04:07.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.450+0000 7f44b46d8b80 1 -- 172.21.15.112:0/2480942219 wait complete. 2024-05-08T06:04:07.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.450+0000 7f44b46d8b80 1 -- start start 2024-05-08T06:04:07.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.450+0000 7f44b46d8b80 1 Processor -- start 2024-05-08T06:04:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44b46d8b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f427fe8f0 0x563f42810330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44b46d8b80 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f42803bc0 0x563f42810870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44b46d8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f428044e0 0x563f4281eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44b46d8b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44a41b2860 con 0x563f427fe8f0 2024-05-08T06:04:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44b46d8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44a41b3cd0 con 0x563f428044e0 2024-05-08T06:04:07.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44b46d8b80 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f44a41b2f30 con 0x563f42803bc0 2024-05-08T06:04:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44b46d8b80 10 client.0 _ll_register_callbacks cb 0x563f426f7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:04:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44a2a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f427fe8f0 0x563f42810330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44a2a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f427fe8f0 0x563f42810330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48050/0 (socket says 172.21.15.112:48050) 2024-05-08T06:04:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44a2a00700 1 -- 172.21.15.112:0/908594197 learned_addr learned my addr 172.21.15.112:0/908594197 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f449a000700 1 --2- 172.21.15.112:0/908594197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f42803bc0 0x563f42810870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44a3400700 1 --2- 172.21.15.112:0/908594197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f428044e0 0x563f4281eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44a2a00700 1 -- 172.21.15.112:0/908594197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f42803bc0 msgr2=0x563f42810870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44a2a00700 1 --2- 172.21.15.112:0/908594197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f42803bc0 0x563f42810870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44a2a00700 1 -- 172.21.15.112:0/908594197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f428044e0 msgr2=0x563f4281eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:04:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44a2a00700 1 --2- 172.21.15.112:0/908594197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f428044e0 0x563f4281eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.451+0000 7f44a2a00700 1 -- 172.21.15.112:0/908594197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f4281f250 con 0x563f427fe8f0 2024-05-08T06:04:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.452+0000 7f44a2a00700 1 --2- 172.21.15.112:0/908594197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f427fe8f0 0x563f42810330 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f44940097d0 tx=0x7f4494010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22414e3bd1e9b622 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.452+0000 7f4491600700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-05-08T06:04:07.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.452+0000 7f4491600700 1 -- 172.21.15.112:0/908594197 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494088070 con 0x563f427fe8f0 2024-05-08T06:04:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.452+0000 7f4491600700 1 -- 172.21.15.112:0/908594197 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f44940a9bd0 con 0x563f427fe8f0 2024-05-08T06:04:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.452+0000 7f44b46d8b80 1 -- 172.21.15.112:0/908594197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44940a9970 con 0x563f427fe8f0 2024-05-08T06:04:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.452+0000 7f4491600700 1 -- 172.21.15.112:0/908594197 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f44940a9480 con 0x563f427fe8f0 2024-05-08T06:04:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.452+0000 7f44b46d8b80 10 client.15918 fetch_fsmap learned FSMap version 36 2024-05-08T06:04:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.452+0000 7f44b46d8b80 1 -- 172.21.15.112:0/908594197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563f428820f0 con 0x563f427fe8f0 2024-05-08T06:04:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.453+0000 7f4491600700 1 -- 172.21.15.112:0/908594197 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44940a9970 con 0x563f427fe8f0 2024-05-08T06:04:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.453+0000 7f4491600700 1 -- 172.21.15.112:0/908594197 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f44940a5400 con 0x563f427fe8f0 2024-05-08T06:04:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.453+0000 7f44b46d8b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T06:04:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.453+0000 7f44b46d8b80 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.a 2024-05-08T06:04:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.453+0000 7f44b46d8b80 20 client.15918 populate_metadata read hostname 'smithi112' 2024-05-08T06:04:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.453+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/908594197 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f42882380 0x563f42884770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.453+0000 7f44b46d8b80 4 client.15918 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:04:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.453+0000 7f44b46d8b80 1 -- 172.21.15.112:0/908594197 --> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f4275e400 con 0x563f42882380 2024-05-08T06:04:07.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.453+0000 7f449a000700 1 --2- 172.21.15.112:0/908594197 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f42882380 0x563f42884770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:07.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.454+0000 7f449a000700 1 --2- 172.21.15.112:0/908594197 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f42882380 0x563f42884770 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6c6976c74e6d1f server_cookie=4fdbd333129f7e41 in_seq=0 out_seq=0 2024-05-08T06:04:07.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.454+0000 7f4491600700 10 client.15918 ms_handle_connect on v2:172.21.15.112:6840/2816461111 2024-05-08T06:04:07.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.454+0000 7f4491600700 1 -- 172.21.15.112:0/908594197 <== mds.0 v2:172.21.15.112:6840/2816461111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f4275e400 con 0x563f42882380 2024-05-08T06:04:07.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.454+0000 7f4491600700 10 client.15918 handle_command_reply: tid=0 2024-05-08T06:04:07.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.454+0000 7f44b46d8b80 1 client.15918 shutdown 2024-05-08T06:04:07.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.454+0000 7f44b46d8b80 1 -- 172.21.15.112:0/908594197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f427fe8f0 msgr2=0x563f42810330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:07.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.454+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/908594197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f427fe8f0 0x563f42810330 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f44940097d0 tx=0x7f4494010c10 comp rx=0 tx=0).stop 2024-05-08T06:04:07.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.454+0000 7f44b46d8b80 1 -- 172.21.15.112:0/908594197 shutdown_connections 2024-05-08T06:04:07.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.454+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/908594197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f428044e0 0x563f4281eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.454+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/908594197 >> [v2:172.21.15.112:6840/2816461111,v1:172.21.15.112:6841/2816461111] conn(0x563f42882380 0x563f42884770 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.454+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/908594197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f427fe8f0 0x563f42810330 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.454+0000 7f44b46d8b80 1 --2- 172.21.15.112:0/908594197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f42803bc0 0x563f42810870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.454+0000 7f44b46d8b80 1 -- 172.21.15.112:0/908594197 >> 172.21.15.112:0/908594197 conn(0x563f427766d0 msgr2=0x563f42806740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:07.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.454+0000 7f4491600700 0 client.15918 ms_handle_reset on v2:172.21.15.112:6840/2816461111 2024-05-08T06:04:07.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.455+0000 7f44b46d8b80 1 -- 172.21.15.112:0/908594197 shutdown_connections 2024-05-08T06:04:07.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.455+0000 7f44b46d8b80 1 -- 172.21.15.112:0/908594197 wait complete. 2024-05-08T06:04:07.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.455+0000 7f44b46d8b80 20 client.15918 trim_cache size 0 max 16384 2024-05-08T06:04:07.458 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-05-08T06:04:07.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.455+0000 7f44a9e00700 1 -- 172.21.15.112:0/2815766375 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f44880b2900 msgr2=0x7f44880b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:07.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.455+0000 7f44a9e00700 1 --2- 172.21.15.112:0/2815766375 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f44880b2900 0x7f44880b4dc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f44a406ca00 tx=0x7f449c009920 comp rx=0 tx=0).stop 2024-05-08T06:04:07.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.455+0000 7f44a9e00700 1 -- 172.21.15.112:0/2815766375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44a40756d0 msgr2=0x7f44a41a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:07.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.455+0000 7f44a9e00700 1 --2- 172.21.15.112:0/2815766375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44a40756d0 0x7f44a41a0210 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f449400e9a0 tx=0x7f449400ecb0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.456+0000 7f44a9e00700 1 -- 172.21.15.112:0/2815766375 shutdown_connections 2024-05-08T06:04:07.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.456+0000 7f44a9e00700 1 --2- 172.21.15.112:0/2815766375 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f44880b2900 0x7f44880b4dc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.456+0000 7f44a9e00700 1 --2- 172.21.15.112:0/2815766375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a4114670 0x7f44a41a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.456+0000 7f44a9e00700 1 --2- 172.21.15.112:0/2815766375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44a40756d0 0x7f44a41a0210 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.456+0000 7f44a9e00700 1 --2- 172.21.15.112:0/2815766375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44a4075ff0 0x7f44a41a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:07.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.456+0000 7f44a9e00700 1 -- 172.21.15.112:0/2815766375 >> 172.21.15.112:0/2815766375 conn(0x7f44a407a600 msgr2=0x7f44a4110110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:07.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.456+0000 7f44a9e00700 1 -- 172.21.15.112:0/2815766375 shutdown_connections 2024-05-08T06:04:07.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:07.456+0000 7f44a9e00700 1 -- 172.21.15.112:0/2815766375 wait complete. 2024-05-08T06:04:07.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:07 smithi150 ceph-mon[34315]: pgmap v705: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-08T06:04:07.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:07 smithi162 ceph-mon[34800]: pgmap v705: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-08T06:04:08.274 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-08T06:04:08.275 INFO:tasks.fwd_scrub:done joining 2024-05-08T06:04:08.275 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-08T06:04:08.285 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:04:08.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.500+0000 7f5591c00640 1 -- 172.21.15.112:0/2082485305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f558c101bc0 msgr2=0x7f558c102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:08.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.500+0000 7f5591c00640 1 --2- 172.21.15.112:0/2082485305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f558c101bc0 0x7f558c102020 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5574009860 tx=0x7f557402a340 comp rx=0 tx=0).stop 2024-05-08T06:04:08.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.500+0000 7f5591c00640 1 -- 172.21.15.112:0/2082485305 shutdown_connections 2024-05-08T06:04:08.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.500+0000 7f5591c00640 1 --2- 172.21.15.112:0/2082485305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f558c102560 0x7f558c1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:08.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.500+0000 7f5591c00640 1 --2- 172.21.15.112:0/2082485305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f558c101bc0 0x7f558c102020 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:08.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.500+0000 7f5591c00640 1 --2- 172.21.15.112:0/2082485305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f558c101260 0x7f558c101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:08.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.500+0000 7f5591c00640 1 -- 172.21.15.112:0/2082485305 >> 172.21.15.112:0/2082485305 conn(0x7f558c0fce60 msgr2=0x7f558c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:08.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.501+0000 7f5591c00640 1 -- 172.21.15.112:0/2082485305 shutdown_connections 2024-05-08T06:04:08.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.501+0000 7f5591c00640 1 -- 172.21.15.112:0/2082485305 wait complete. 2024-05-08T06:04:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.501+0000 7f5591c00640 1 Processor -- start 2024-05-08T06:04:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.502+0000 7f5591c00640 1 -- start start 2024-05-08T06:04:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.502+0000 7f5591c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f558c101260 0x7f558c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.502+0000 7f5591c00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f558c101bc0 0x7f558c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.502+0000 7f5591c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f558c102560 0x7f558c1a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.502+0000 7f5591c00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f558c1106e0 con 0x7f558c101bc0 2024-05-08T06:04:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.502+0000 7f5591c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f558c110560 con 0x7f558c102560 2024-05-08T06:04:08.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.502+0000 7f5591c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f558c110860 con 0x7f558c101260 2024-05-08T06:04:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.502+0000 7f558a000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f558c101bc0 0x7f558c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.502+0000 7f558a000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f558c101bc0 0x7f558c19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48066/0 (socket says 172.21.15.112:48066) 2024-05-08T06:04:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.502+0000 7f558a000640 1 -- 172.21.15.112:0/1158113228 learned_addr learned my addr 172.21.15.112:0/1158113228 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.502+0000 7f558aa00640 1 --2- 172.21.15.112:0/1158113228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f558c101260 0x7f558c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.502+0000 7f558b400640 1 --2- 172.21.15.112:0/1158113228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f558c102560 0x7f558c1a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.503+0000 7f558a000640 1 -- 172.21.15.112:0/1158113228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f558c101260 msgr2=0x7f558c19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.503+0000 7f558a000640 1 --2- 172.21.15.112:0/1158113228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f558c101260 0x7f558c19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.503+0000 7f558a000640 1 -- 172.21.15.112:0/1158113228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f558c102560 msgr2=0x7f558c1a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.503+0000 7f558a000640 1 --2- 172.21.15.112:0/1158113228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f558c102560 0x7f558c1a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.503+0000 7f558a000640 1 -- 172.21.15.112:0/1158113228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c1a2a30 con 0x7f558c101bc0 2024-05-08T06:04:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.503+0000 7f558aa00640 1 --2- 172.21.15.112:0/1158113228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f558c101260 0x7f558c19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:04:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.503+0000 7f558a000640 1 --2- 172.21.15.112:0/1158113228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f558c101bc0 0x7f558c19b350 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5574005940 tx=0x7f557402ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:08.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.503+0000 7f5583400640 1 -- 172.21.15.112:0/1158113228 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574040020 con 0x7f558c101bc0 2024-05-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.503+0000 7f5583400640 1 -- 172.21.15.112:0/1158113228 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5574030e50 con 0x7f558c101bc0 2024-05-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.503+0000 7f5583400640 1 -- 172.21.15.112:0/1158113228 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574037040 con 0x7f558c101bc0 2024-05-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.503+0000 7f5591c00640 1 -- 172.21.15.112:0/1158113228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c1a2cc0 con 0x7f558c101bc0 2024-05-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.504+0000 7f5591c00640 1 -- 172.21.15.112:0/1158113228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c1a3160 con 0x7f558c101bc0 2024-05-08T06:04:08.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.505+0000 7f5591c00640 1 -- 172.21.15.112:0/1158113228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558c10c230 con 0x7f558c101bc0 2024-05-08T06:04:08.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.507+0000 7f5583400640 1 -- 172.21.15.112:0/1158113228 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5574043020 con 0x7f558c101bc0 2024-05-08T06:04:08.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.509+0000 7f5583400640 1 --2- 172.21.15.112:0/1158113228 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f55700b2870 0x7f55700b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:08.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.509+0000 7f5583400640 1 -- 172.21.15.112:0/1158113228 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f5574101fd0 con 0x7f558c101bc0 2024-05-08T06:04:08.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.509+0000 7f558aa00640 1 --2- 172.21.15.112:0/1158113228 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f55700b2870 0x7f55700b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:08.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.510+0000 7f558aa00640 1 --2- 172.21.15.112:0/1158113228 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f55700b2870 0x7f55700b4d30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f557c005fd0 tx=0x7f557c010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:08.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.513+0000 7f5583400640 1 -- 172.21.15.112:0/1158113228 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f557404a020 con 0x7f558c101bc0 2024-05-08T06:04:08.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.724+0000 7f5591c00640 1 -- 172.21.15.112:0/1158113228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f558c0635b0 con 0x7f558c101bc0 2024-05-08T06:04:08.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.726+0000 7f5583400640 1 -- 172.21.15.112:0/1158113228 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22829 (secure 0 0 0) 0x7f55740c9c60 con 0x7f558c101bc0 2024-05-08T06:04:08.727 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T06:04:08.727 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":84,"fsid":"7a07ff60-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:48.498871+0000","modified":"2024-05-08T05:53:05.251932+0000","last_up_change":"2024-05-08T05:49:54.039380+0000","last_in_change":"2024-05-08T05:49:34.443850+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:47.485105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:50:58.918909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.175022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b378542-1bbe-4614-8805-8a6b875ada55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6803","nonce":3419091820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6805","nonce":3419091820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6809","nonce":3419091820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6807","nonce":3419091820}]},"public_addr":"172.21.15.112:6803/3419091820","cluster_addr":"172.21.15.112:6805/3419091820","heartbeat_back_addr":"172.21.15.112:6809/3419091820","heartbeat_front_addr":"172.21.15.112:6807/3419091820","state":["exists","up"]},{"osd":1,"uuid":"826ef036-2db7-470e-9956-d9a58b9181e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6801","nonce":2728419797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6803","nonce":2728419797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6807","nonce":2728419797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6805","nonce":2728419797}]},"public_addr":"172.21.15.150:6801/2728419797","cluster_addr":"172.21.15.150:6803/2728419797","heartbeat_back_addr":"172.21.15.150:6807/2728419797","heartbeat_front_addr":"172.21.15.150:6805/2728419797","state":["exists","up"]},{"osd":2,"uuid":"32e710e7-2a87-4863-bc7a-906d31aa4ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6801","nonce":1556203215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6803","nonce":1556203215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6807","nonce":1556203215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6805","nonce":1556203215}]},"public_addr":"172.21.15.162:6801/1556203215","cluster_addr":"172.21.15.162:6803/1556203215","heartbeat_back_addr":"172.21.15.162:6807/1556203215","heartbeat_front_addr":"172.21.15.162:6805/1556203215","state":["exists","up"]},{"osd":3,"uuid":"ba1d4a5f-6a95-4b9e-8a48-8524a899b004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6811","nonce":3325718755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6813","nonce":3325718755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6817","nonce":3325718755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6815","nonce":3325718755}]},"public_addr":"172.21.15.112:6811/3325718755","cluster_addr":"172.21.15.112:6813/3325718755","heartbeat_back_addr":"172.21.15.112:6817/3325718755","heartbeat_front_addr":"172.21.15.112:6815/3325718755","state":["exists","up"]},{"osd":4,"uuid":"cb4f1a42-8de8-4b50-b3bc-602de9f4803a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6809","nonce":1548914683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6811","nonce":1548914683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6815","nonce":1548914683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6813","nonce":1548914683}]},"public_addr":"172.21.15.150:6809/1548914683","cluster_addr":"172.21.15.150:6811/1548914683","heartbeat_back_addr":"172.21.15.150:6815/1548914683","heartbeat_front_addr":"172.21.15.150:6813/1548914683","state":["exists","up"]},{"osd":5,"uuid":"a26c03b8-050c-4a2c-8f23-5801fffb7c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6809","nonce":409836750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6811","nonce":409836750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6815","nonce":409836750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6813","nonce":409836750}]},"public_addr":"172.21.15.162:6809/409836750","cluster_addr":"172.21.15.162:6811/409836750","heartbeat_back_addr":"172.21.15.162:6815/409836750","heartbeat_front_addr":"172.21.15.162:6813/409836750","state":["exists","up"]},{"osd":6,"uuid":"8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6819","nonce":1098984186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6821","nonce":1098984186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6825","nonce":1098984186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6823","nonce":1098984186}]},"public_addr":"172.21.15.112:6819/1098984186","cluster_addr":"172.21.15.112:6821/1098984186","heartbeat_back_addr":"172.21.15.112:6825/1098984186","heartbeat_front_addr":"172.21.15.112:6823/1098984186","state":["exists","up"]},{"osd":7,"uuid":"af3629b4-9bed-4669-97a6-48deba503a96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6817","nonce":4009836564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6819","nonce":4009836564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6823","nonce":4009836564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6821","nonce":4009836564}]},"public_addr":"172.21.15.150:6817/4009836564","cluster_addr":"172.21.15.150:6819/4009836564","heartbeat_back_addr":"172.21.15.150:6823/4009836564","heartbeat_front_addr":"172.21.15.150:6821/4009836564","state":["exists","up"]},{"osd":8,"uuid":"9333e87d-dd78-4467-bbde-c2b44f4ea07f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6817","nonce":852791606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6819","nonce":852791606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6823","nonce":852791606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6821","nonce":852791606}]},"public_addr":"172.21.15.162:6817/852791606","cluster_addr":"172.21.15.162:6819/852791606","heartbeat_back_addr":"172.21.15.162:6823/852791606","heartbeat_front_addr":"172.21.15.162:6821/852791606","state":["exists","up"]},{"osd":9,"uuid":"09723581-1995-42e0-8db7-776ea98a918c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6827","nonce":2186764674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6829","nonce":2186764674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6833","nonce":2186764674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6831","nonce":2186764674}]},"public_addr":"172.21.15.112:6827/2186764674","cluster_addr":"172.21.15.112:6829/2186764674","heartbeat_back_addr":"172.21.15.112:6833/2186764674","heartbeat_front_addr":"172.21.15.112:6831/2186764674","state":["exists","up"]},{"osd":10,"uuid":"34e08e3e-1231-4899-a5b3-76d449105bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6825","nonce":2838995474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6827","nonce":2838995474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6831","nonce":2838995474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6829","nonce":2838995474}]},"public_addr":"172.21.15.150:6825/2838995474","cluster_addr":"172.21.15.150:6827/2838995474","heartbeat_back_addr":"172.21.15.150:6831/2838995474","heartbeat_front_addr":"172.21.15.150:6829/2838995474","state":["exists","up"]},{"osd":11,"uuid":"6fc34ec0-78ca-403c-acb7-b29d83ff64e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6825","nonce":3702614331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6827","nonce":3702614331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6831","nonce":3702614331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6829","nonce":3702614331}]},"public_addr":"172.21.15.162:6825/3702614331","cluster_addr":"172.21.15.162:6827/3702614331","heartbeat_back_addr":"172.21.15.162:6831/3702614331","heartbeat_front_addr":"172.21.15.162:6829/3702614331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:54.518702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:19.939543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:44.425586+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:16.126471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:41.696071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:05.517405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:38.549318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:03.799649+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:28.007282+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:01.395459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:27.963637+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:52.181212+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3667148497":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6800/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3425327158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1404611205":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3765334043":"2024-05-09T05:42:29.819307+0000","172.21.15.112:6800/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/2814303790":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1091751583":"2024-05-09T05:42:29.819307+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:04:08.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.728+0000 7f5591c00640 1 -- 172.21.15.112:0/1158113228 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f55700b2870 msgr2=0x7f55700b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:08.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.728+0000 7f5591c00640 1 --2- 172.21.15.112:0/1158113228 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f55700b2870 0x7f55700b4d30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f557c005fd0 tx=0x7f557c010040 comp rx=0 tx=0).stop 2024-05-08T06:04:08.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.728+0000 7f5591c00640 1 -- 172.21.15.112:0/1158113228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f558c101bc0 msgr2=0x7f558c19b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:08.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.728+0000 7f5591c00640 1 --2- 172.21.15.112:0/1158113228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f558c101bc0 0x7f558c19b350 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5574005940 tx=0x7f557402ae20 comp rx=0 tx=0).stop 2024-05-08T06:04:08.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.729+0000 7f5591c00640 1 -- 172.21.15.112:0/1158113228 shutdown_connections 2024-05-08T06:04:08.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.729+0000 7f5591c00640 1 --2- 172.21.15.112:0/1158113228 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f55700b2870 0x7f55700b4d30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:08.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.729+0000 7f5591c00640 1 --2- 172.21.15.112:0/1158113228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f558c102560 0x7f558c1a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:08.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.729+0000 7f5591c00640 1 --2- 172.21.15.112:0/1158113228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f558c101bc0 0x7f558c19b350 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:08.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.729+0000 7f5591c00640 1 --2- 172.21.15.112:0/1158113228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f558c101260 0x7f558c19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:08.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.729+0000 7f5591c00640 1 -- 172.21.15.112:0/1158113228 >> 172.21.15.112:0/1158113228 conn(0x7f558c0fce60 msgr2=0x7f558c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:08.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.729+0000 7f5591c00640 1 -- 172.21.15.112:0/1158113228 shutdown_connections 2024-05-08T06:04:08.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.729+0000 7f5591c00640 1 -- 172.21.15.112:0/1158113228 wait complete. 2024-05-08T06:04:08.749 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T05:45:47.485105+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T05:50:58.918909+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T05:51:03.175022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T06:04:08.750 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T06:04:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.958+0000 7f909fe00640 1 -- 172.21.15.112:0/425730814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a0104470 msgr2=0x7f90a010ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.958+0000 7f909fe00640 1 --2- 172.21.15.112:0/425730814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a0104470 0x7f90a010ad00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9098009be0 tx=0x7f909802a6d0 comp rx=0 tx=0).stop 2024-05-08T06:04:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.958+0000 7f909fe00640 1 -- 172.21.15.112:0/425730814 shutdown_connections 2024-05-08T06:04:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.958+0000 7f909fe00640 1 --2- 172.21.15.112:0/425730814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a0104470 0x7f90a010ad00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.958+0000 7f909fe00640 1 --2- 172.21.15.112:0/425730814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90a0103ab0 0x7f90a0103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.958+0000 7f909fe00640 1 --2- 172.21.15.112:0/425730814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90a01028c0 0x7f90a0102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.958+0000 7f909fe00640 1 -- 172.21.15.112:0/425730814 >> 172.21.15.112:0/425730814 conn(0x7f90a00fde50 msgr2=0x7f90a0100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.958+0000 7f909fe00640 1 -- 172.21.15.112:0/425730814 shutdown_connections 2024-05-08T06:04:08.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.958+0000 7f909fe00640 1 -- 172.21.15.112:0/425730814 wait complete. 2024-05-08T06:04:08.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.959+0000 7f909fe00640 1 Processor -- start 2024-05-08T06:04:08.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.959+0000 7f909fe00640 1 -- start start 2024-05-08T06:04:08.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.960+0000 7f909fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90a01028c0 0x7f90a019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:08.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.960+0000 7f909fe00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90a0103ab0 0x7f90a019b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:08.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.960+0000 7f909fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a0104470 0x7f90a01a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:08.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.960+0000 7f909fe00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90a010e3f0 con 0x7f90a0103ab0 2024-05-08T06:04:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.960+0000 7f909fe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f90a010e270 con 0x7f90a0104470 2024-05-08T06:04:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.960+0000 7f909fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f90a010e570 con 0x7f90a01028c0 2024-05-08T06:04:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.960+0000 7f909e000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90a0103ab0 0x7f90a019b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.960+0000 7f909e000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90a0103ab0 0x7f90a019b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48076/0 (socket says 172.21.15.112:48076) 2024-05-08T06:04:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.960+0000 7f909e000640 1 -- 172.21.15.112:0/2941221777 learned_addr learned my addr 172.21.15.112:0/2941221777 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.960+0000 7f909f400640 1 --2- 172.21.15.112:0/2941221777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a0104470 0x7f90a01a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.960+0000 7f909ea00640 1 --2- 172.21.15.112:0/2941221777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90a01028c0 0x7f90a019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.961+0000 7f909e000640 1 -- 172.21.15.112:0/2941221777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90a01028c0 msgr2=0x7f90a019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.961+0000 7f909e000640 1 --2- 172.21.15.112:0/2941221777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90a01028c0 0x7f90a019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.961+0000 7f909e000640 1 -- 172.21.15.112:0/2941221777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a0104470 msgr2=0x7f90a01a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.961+0000 7f909e000640 1 --2- 172.21.15.112:0/2941221777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a0104470 0x7f90a01a2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:08.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.961+0000 7f909e000640 1 -- 172.21.15.112:0/2941221777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a01a28c0 con 0x7f90a0103ab0 2024-05-08T06:04:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.961+0000 7f909f400640 1 --2- 172.21.15.112:0/2941221777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a0104470 0x7f90a01a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:04:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.961+0000 7f909e000640 1 --2- 172.21.15.112:0/2941221777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90a0103ab0 0x7f90a019b1c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f908c00e9f0 tx=0x7f908c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.961+0000 7f9097400640 1 -- 172.21.15.112:0/2941221777 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c00ccc0 con 0x7f90a0103ab0 2024-05-08T06:04:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.961+0000 7f9097400640 1 -- 172.21.15.112:0/2941221777 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f908c00ce60 con 0x7f90a0103ab0 2024-05-08T06:04:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.961+0000 7f9097400640 1 -- 172.21.15.112:0/2941221777 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c010430 con 0x7f90a0103ab0 2024-05-08T06:04:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.961+0000 7f909fe00640 1 -- 172.21.15.112:0/2941221777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a01a2bb0 con 0x7f90a0103ab0 2024-05-08T06:04:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.961+0000 7f909fe00640 1 -- 172.21.15.112:0/2941221777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a01a3100 con 0x7f90a0103ab0 2024-05-08T06:04:08.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.963+0000 7f909fe00640 1 -- 172.21.15.112:0/2941221777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a01a3330 con 0x7f90a0103ab0 2024-05-08T06:04:08.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.965+0000 7f9097400640 1 -- 172.21.15.112:0/2941221777 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f908c0046c0 con 0x7f90a0103ab0 2024-05-08T06:04:08.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.967+0000 7f9097400640 1 --2- 172.21.15.112:0/2941221777 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f90840b2870 0x7f90840b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:08.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.968+0000 7f9097400640 1 -- 172.21.15.112:0/2941221777 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f908c0e25f0 con 0x7f90a0103ab0 2024-05-08T06:04:08.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.968+0000 7f909ea00640 1 --2- 172.21.15.112:0/2941221777 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f90840b2870 0x7f90840b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:08.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.969+0000 7f909ea00640 1 --2- 172.21.15.112:0/2941221777 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f90840b2870 0x7f90840b4d30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f90a019bb50 tx=0x7f9090009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:08.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:08.971+0000 7f9097400640 1 -- 172.21.15.112:0/2941221777 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f908c0aa280 con 0x7f90a0103ab0 2024-05-08T06:04:09.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.178+0000 7f909fe00640 1 -- 172.21.15.112:0/2941221777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f90a01a3500 con 0x7f90a0103ab0 2024-05-08T06:04:09.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.179+0000 7f9097400640 1 -- 172.21.15.112:0/2941221777 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f908c0a9b80 con 0x7f90a0103ab0 2024-05-08T06:04:09.179 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-05-08T06:04:09.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.182+0000 7f909fe00640 1 -- 172.21.15.112:0/2941221777 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f90840b2870 msgr2=0x7f90840b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:09.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.182+0000 7f909fe00640 1 --2- 172.21.15.112:0/2941221777 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f90840b2870 0x7f90840b4d30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f90a019bb50 tx=0x7f9090009260 comp rx=0 tx=0).stop 2024-05-08T06:04:09.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.182+0000 7f909fe00640 1 -- 172.21.15.112:0/2941221777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90a0103ab0 msgr2=0x7f90a019b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:09.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.182+0000 7f909fe00640 1 --2- 172.21.15.112:0/2941221777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90a0103ab0 0x7f90a019b1c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f908c00e9f0 tx=0x7f908c00edf0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.182+0000 7f909fe00640 1 -- 172.21.15.112:0/2941221777 shutdown_connections 2024-05-08T06:04:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.182+0000 7f909fe00640 1 --2- 172.21.15.112:0/2941221777 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f90840b2870 0x7f90840b4d30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.182+0000 7f909fe00640 1 --2- 172.21.15.112:0/2941221777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a0104470 0x7f90a01a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.182+0000 7f909fe00640 1 --2- 172.21.15.112:0/2941221777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90a0103ab0 0x7f90a019b1c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.182+0000 7f909fe00640 1 --2- 172.21.15.112:0/2941221777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90a01028c0 0x7f90a019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.182+0000 7f909fe00640 1 -- 172.21.15.112:0/2941221777 >> 172.21.15.112:0/2941221777 conn(0x7f90a00fde50 msgr2=0x7f90a0074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.182+0000 7f909fe00640 1 -- 172.21.15.112:0/2941221777 shutdown_connections 2024-05-08T06:04:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.182+0000 7f909fe00640 1 -- 172.21.15.112:0/2941221777 wait complete. 2024-05-08T06:04:09.202 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T06:04:09.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.402+0000 7fbe07e00640 1 -- 172.21.15.112:0/3466002121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0810aa40 msgr2=0x7fbe08111360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:09.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.402+0000 7fbe07e00640 1 --2- 172.21.15.112:0/3466002121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0810aa40 0x7fbe08111360 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009ca0 tx=0x7fbe0002a8c0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.402+0000 7fbe07e00640 1 -- 172.21.15.112:0/3466002121 shutdown_connections 2024-05-08T06:04:09.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.402+0000 7fbe07e00640 1 --2- 172.21.15.112:0/3466002121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0810aa40 0x7fbe08111360 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.402+0000 7fbe07e00640 1 --2- 172.21.15.112:0/3466002121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe0810a080 0x7fbe0810a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.402+0000 7fbe07e00640 1 --2- 172.21.15.112:0/3466002121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe08108ea0 0x7fbe081092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.402+0000 7fbe07e00640 1 -- 172.21.15.112:0/3466002121 >> 172.21.15.112:0/3466002121 conn(0x7fbe08077e80 msgr2=0x7fbe0807a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:09.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.402+0000 7fbe07e00640 1 -- 172.21.15.112:0/3466002121 shutdown_connections 2024-05-08T06:04:09.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.402+0000 7fbe07e00640 1 -- 172.21.15.112:0/3466002121 wait complete. 2024-05-08T06:04:09.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.403+0000 7fbe07e00640 1 Processor -- start 2024-05-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.403+0000 7fbe07e00640 1 -- start start 2024-05-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.404+0000 7fbe07e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe08108ea0 0x7fbe0819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.404+0000 7fbe07e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0810a080 0x7fbe0819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.404+0000 7fbe07e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe0810aa40 0x7fbe081a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.404+0000 7fbe07e00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbe08114a20 con 0x7fbe0810a080 2024-05-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.404+0000 7fbe07e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081148a0 con 0x7fbe0810aa40 2024-05-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.404+0000 7fbe07e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbe08114ba0 con 0x7fbe08108ea0 2024-05-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.404+0000 7fbe06000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0810a080 0x7fbe0819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.404+0000 7fbe06000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0810a080 0x7fbe0819f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48096/0 (socket says 172.21.15.112:48096) 2024-05-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.404+0000 7fbe06000640 1 -- 172.21.15.112:0/2266056960 learned_addr learned my addr 172.21.15.112:0/2266056960 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.404+0000 7fbe06a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe08108ea0 0x7fbe0819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.404+0000 7fbe07400640 1 --2- 172.21.15.112:0/2266056960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe0810aa40 0x7fbe081a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:09.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.405+0000 7fbe06000640 1 -- 172.21.15.112:0/2266056960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe08108ea0 msgr2=0x7fbe0819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.405+0000 7fbe06000640 1 --2- 172.21.15.112:0/2266056960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe08108ea0 0x7fbe0819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.405+0000 7fbe06000640 1 -- 172.21.15.112:0/2266056960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe0810aa40 msgr2=0x7fbe081a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.405+0000 7fbe06000640 1 --2- 172.21.15.112:0/2266056960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe0810aa40 0x7fbe081a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.405+0000 7fbe06000640 1 -- 172.21.15.112:0/2266056960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe081a6c50 con 0x7fbe0810a080 2024-05-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.405+0000 7fbe06000640 1 --2- 172.21.15.112:0/2266056960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0810a080 0x7fbe0819f570 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400e9f0 tx=0x7fbdf400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.405+0000 7fbdff400640 1 -- 172.21.15.112:0/2266056960 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400cd80 con 0x7fbe0810a080 2024-05-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.405+0000 7fbdff400640 1 -- 172.21.15.112:0/2266056960 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdf40108e0 con 0x7fbe0810a080 2024-05-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.405+0000 7fbe07e00640 1 -- 172.21.15.112:0/2266056960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe081a6f40 con 0x7fbe0810a080 2024-05-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.405+0000 7fbdff400640 1 -- 172.21.15.112:0/2266056960 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4018980 con 0x7fbe0810a080 2024-05-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.405+0000 7fbe07e00640 1 -- 172.21.15.112:0/2266056960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe081a7400 con 0x7fbe0810a080 2024-05-08T06:04:09.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.406+0000 7fbe07e00640 1 -- 172.21.15.112:0/2266056960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe081101e0 con 0x7fbe0810a080 2024-05-08T06:04:09.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.409+0000 7fbdff400640 1 -- 172.21.15.112:0/2266056960 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbdf4010430 con 0x7fbe0810a080 2024-05-08T06:04:09.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.411+0000 7fbdff400640 1 --2- 172.21.15.112:0/2266056960 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbdec0b27f0 0x7fbdec0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:09.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.411+0000 7fbdff400640 1 -- 172.21.15.112:0/2266056960 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fbdf40e29f0 con 0x7fbe0810a080 2024-05-08T06:04:09.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.411+0000 7fbe06a00640 1 --2- 172.21.15.112:0/2266056960 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbdec0b27f0 0x7fbdec0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:09.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.412+0000 7fbe06a00640 1 --2- 172.21.15.112:0/2266056960 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbdec0b27f0 0x7fbdec0b4cb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbe08109bf0 tx=0x7fbdf8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:09.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.412+0000 7fbdff400640 1 -- 172.21.15.112:0/2266056960 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbdf40aa090 con 0x7fbe0810a080 2024-05-08T06:04:09.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:09 smithi112 ceph-mon[30003]: pgmap v706: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T06:04:09.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:09 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1158113228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:04:09.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:09 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2941221777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T06:04:09.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.621+0000 7fbe07e00640 1 -- 172.21.15.112:0/2266056960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbe0802dbf0 con 0x7fbe0810a080 2024-05-08T06:04:09.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.622+0000 7fbdff400640 1 -- 172.21.15.112:0/2266056960 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7fbdf40ae390 con 0x7fbe0810a080 2024-05-08T06:04:09.625 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-05-08T06:04:09.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.625+0000 7fbe07e00640 1 -- 172.21.15.112:0/2266056960 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbdec0b27f0 msgr2=0x7fbdec0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:09.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.625+0000 7fbe07e00640 1 --2- 172.21.15.112:0/2266056960 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbdec0b27f0 0x7fbdec0b4cb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbe08109bf0 tx=0x7fbdf8005f90 comp rx=0 tx=0).stop 2024-05-08T06:04:09.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.625+0000 7fbe07e00640 1 -- 172.21.15.112:0/2266056960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0810a080 msgr2=0x7fbe0819f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:09.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.625+0000 7fbe07e00640 1 --2- 172.21.15.112:0/2266056960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0810a080 0x7fbe0819f570 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400e9f0 tx=0x7fbdf400eeb0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.626+0000 7fbe07e00640 1 -- 172.21.15.112:0/2266056960 shutdown_connections 2024-05-08T06:04:09.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.626+0000 7fbe07e00640 1 --2- 172.21.15.112:0/2266056960 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbdec0b27f0 0x7fbdec0b4cb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.626+0000 7fbe07e00640 1 --2- 172.21.15.112:0/2266056960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe0810aa40 0x7fbe081a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.626+0000 7fbe07e00640 1 --2- 172.21.15.112:0/2266056960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0810a080 0x7fbe0819f570 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.626+0000 7fbe07e00640 1 --2- 172.21.15.112:0/2266056960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe08108ea0 0x7fbe0819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.626+0000 7fbe07e00640 1 -- 172.21.15.112:0/2266056960 >> 172.21.15.112:0/2266056960 conn(0x7fbe08077e80 msgr2=0x7fbe0810d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:09.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.626+0000 7fbe07e00640 1 -- 172.21.15.112:0/2266056960 shutdown_connections 2024-05-08T06:04:09.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.626+0000 7fbe07e00640 1 -- 172.21.15.112:0/2266056960 wait complete. 2024-05-08T06:04:09.645 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T06:04:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:09 smithi150 ceph-mon[34315]: pgmap v706: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T06:04:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:09 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1158113228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:04:09.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:09 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/2941221777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T06:04:09.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:09 smithi162 ceph-mon[34800]: pgmap v706: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T06:04:09.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:09 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1158113228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:04:09.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:09 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/2941221777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T06:04:09.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.855+0000 7f40e5e00640 1 -- 172.21.15.112:0/1722385052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0105e00 msgr2=0x7f40e0106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:09.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.855+0000 7f40e5e00640 1 --2- 172.21.15.112:0/1722385052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0105e00 0x7f40e0106280 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009900 tx=0x7f40cc02a3f0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.855+0000 7f40e5e00640 1 -- 172.21.15.112:0/1722385052 shutdown_connections 2024-05-08T06:04:09.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.855+0000 7f40e5e00640 1 --2- 172.21.15.112:0/1722385052 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40e01067c0 0x7f40e010d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.855+0000 7f40e5e00640 1 --2- 172.21.15.112:0/1722385052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0105e00 0x7f40e0106280 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.855+0000 7f40e5e00640 1 --2- 172.21.15.112:0/1722385052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40e0104c10 0x7f40e0105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.855+0000 7f40e5e00640 1 -- 172.21.15.112:0/1722385052 >> 172.21.15.112:0/1722385052 conn(0x7f40e01001a0 msgr2=0x7f40e01025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:09.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.855+0000 7f40e5e00640 1 -- 172.21.15.112:0/1722385052 shutdown_connections 2024-05-08T06:04:09.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.855+0000 7f40e5e00640 1 -- 172.21.15.112:0/1722385052 wait complete. 2024-05-08T06:04:09.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.856+0000 7f40e5e00640 1 Processor -- start 2024-05-08T06:04:09.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.856+0000 7f40e5e00640 1 -- start start 2024-05-08T06:04:09.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.856+0000 7f40e5e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40e0104c10 0x7f40e019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:09.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.856+0000 7f40e5e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0105e00 0x7f40e019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.856+0000 7f40e5e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40e01067c0 0x7f40e01a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.856+0000 7f40e5e00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40e0110720 con 0x7f40e0105e00 2024-05-08T06:04:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.856+0000 7f40e5e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f40e01105a0 con 0x7f40e01067c0 2024-05-08T06:04:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.856+0000 7f40e5e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f40e01108a0 con 0x7f40e0104c10 2024-05-08T06:04:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.857+0000 7f40de000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0105e00 0x7f40e019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.857+0000 7f40de000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0105e00 0x7f40e019b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48110/0 (socket says 172.21.15.112:48110) 2024-05-08T06:04:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.857+0000 7f40de000640 1 -- 172.21.15.112:0/3921926251 learned_addr learned my addr 172.21.15.112:0/3921926251 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.857+0000 7f40dea00640 1 --2- 172.21.15.112:0/3921926251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40e0104c10 0x7f40e019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.857+0000 7f40df400640 1 --2- 172.21.15.112:0/3921926251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40e01067c0 0x7f40e01a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.857+0000 7f40de000640 1 -- 172.21.15.112:0/3921926251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40e0104c10 msgr2=0x7f40e019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.857+0000 7f40de000640 1 --2- 172.21.15.112:0/3921926251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40e0104c10 0x7f40e019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.857+0000 7f40de000640 1 -- 172.21.15.112:0/3921926251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40e01067c0 msgr2=0x7f40e01a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.857+0000 7f40de000640 1 --2- 172.21.15.112:0/3921926251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40e01067c0 0x7f40e01a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.857+0000 7f40de000640 1 -- 172.21.15.112:0/3921926251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e01a2a80 con 0x7f40e0105e00 2024-05-08T06:04:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.858+0000 7f40de000640 1 --2- 172.21.15.112:0/3921926251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0105e00 0x7f40e019b3a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f40cc005940 tx=0x7f40cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.858+0000 7f40e4a00640 1 -- 172.21.15.112:0/3921926251 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc03f030 con 0x7f40e0105e00 2024-05-08T06:04:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.858+0000 7f40e4a00640 1 -- 172.21.15.112:0/3921926251 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40cc003cb0 con 0x7f40e0105e00 2024-05-08T06:04:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.858+0000 7f40e4a00640 1 -- 172.21.15.112:0/3921926251 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc035070 con 0x7f40e0105e00 2024-05-08T06:04:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.858+0000 7f40e5e00640 1 -- 172.21.15.112:0/3921926251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e01a2d10 con 0x7f40e0105e00 2024-05-08T06:04:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.858+0000 7f40e5e00640 1 -- 172.21.15.112:0/3921926251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e01a31b0 con 0x7f40e0105e00 2024-05-08T06:04:09.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.859+0000 7f40e5e00640 1 -- 172.21.15.112:0/3921926251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e010c050 con 0x7f40e0105e00 2024-05-08T06:04:09.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.862+0000 7f40e4a00640 1 -- 172.21.15.112:0/3921926251 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f40cc042030 con 0x7f40e0105e00 2024-05-08T06:04:09.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.864+0000 7f40e4a00640 1 --2- 172.21.15.112:0/3921926251 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f40c40b27f0 0x7f40c40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.865+0000 7f40e4a00640 1 -- 172.21.15.112:0/3921926251 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f40cc102670 con 0x7f40e0105e00 2024-05-08T06:04:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.865+0000 7f40dea00640 1 --2- 172.21.15.112:0/3921926251 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f40c40b27f0 0x7f40c40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:09.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.865+0000 7f40dea00640 1 --2- 172.21.15.112:0/3921926251 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f40c40b27f0 0x7f40c40b4cb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f40e0105c60 tx=0x7f40d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:09.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:09.869+0000 7f40e4a00640 1 -- 172.21.15.112:0/3921926251 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f40cc0ca300 con 0x7f40e0105e00 2024-05-08T06:04:10.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.079+0000 7f40e5e00640 1 -- 172.21.15.112:0/3921926251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f40e002dd30 con 0x7f40e0105e00 2024-05-08T06:04:10.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.080+0000 7f40e4a00640 1 -- 172.21.15.112:0/3921926251 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7f40cc003e60 con 0x7f40e0105e00 2024-05-08T06:04:10.081 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-05-08T06:04:10.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.083+0000 7f40e5e00640 1 -- 172.21.15.112:0/3921926251 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f40c40b27f0 msgr2=0x7f40c40b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:10.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.083+0000 7f40e5e00640 1 --2- 172.21.15.112:0/3921926251 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f40c40b27f0 0x7f40c40b4cb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f40e0105c60 tx=0x7f40d0005f90 comp rx=0 tx=0).stop 2024-05-08T06:04:10.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.083+0000 7f40e5e00640 1 -- 172.21.15.112:0/3921926251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0105e00 msgr2=0x7f40e019b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.083+0000 7f40e5e00640 1 --2- 172.21.15.112:0/3921926251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0105e00 0x7f40e019b3a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f40cc005940 tx=0x7f40cc0042f0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.083+0000 7f40e5e00640 1 -- 172.21.15.112:0/3921926251 shutdown_connections 2024-05-08T06:04:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.083+0000 7f40e5e00640 1 --2- 172.21.15.112:0/3921926251 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f40c40b27f0 0x7f40c40b4cb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.083+0000 7f40e5e00640 1 --2- 172.21.15.112:0/3921926251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40e01067c0 0x7f40e01a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.083+0000 7f40e5e00640 1 --2- 172.21.15.112:0/3921926251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0105e00 0x7f40e019b3a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.083+0000 7f40e5e00640 1 --2- 172.21.15.112:0/3921926251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40e0104c10 0x7f40e019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.083+0000 7f40e5e00640 1 -- 172.21.15.112:0/3921926251 >> 172.21.15.112:0/3921926251 conn(0x7f40e01001a0 msgr2=0x7f40e0109220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.083+0000 7f40e5e00640 1 -- 172.21.15.112:0/3921926251 shutdown_connections 2024-05-08T06:04:10.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.084+0000 7f40e5e00640 1 -- 172.21.15.112:0/3921926251 wait complete. 2024-05-08T06:04:10.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-08T06:04:10.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.303+0000 7f7277e00640 1 -- 172.21.15.112:0/2360306063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7278103a10 msgr2=0x7f7278103e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.303+0000 7f7277e00640 1 --2- 172.21.15.112:0/2360306063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7278103a10 0x7f7278103e90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7264009900 tx=0x7f726402a3f0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.303+0000 7f7277e00640 1 -- 172.21.15.112:0/2360306063 shutdown_connections 2024-05-08T06:04:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.303+0000 7f7277e00640 1 --2- 172.21.15.112:0/2360306063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72781043d0 0x7f727810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.303+0000 7f7277e00640 1 --2- 172.21.15.112:0/2360306063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7278103a10 0x7f7278103e90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.303+0000 7f7277e00640 1 --2- 172.21.15.112:0/2360306063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7278102820 0x7f7278102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.303+0000 7f7277e00640 1 -- 172.21.15.112:0/2360306063 >> 172.21.15.112:0/2360306063 conn(0x7f72780fdd90 msgr2=0x7f72781001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.303+0000 7f7277e00640 1 -- 172.21.15.112:0/2360306063 shutdown_connections 2024-05-08T06:04:10.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.303+0000 7f7277e00640 1 -- 172.21.15.112:0/2360306063 wait complete. 2024-05-08T06:04:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.304+0000 7f7277e00640 1 Processor -- start 2024-05-08T06:04:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.304+0000 7f7277e00640 1 -- start start 2024-05-08T06:04:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.305+0000 7f7277e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7278102820 0x7f7278198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:10.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.305+0000 7f7277e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7278103a10 0x7f7278199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:10.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.305+0000 7f7277e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72781043d0 0x7f72781a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:10.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.305+0000 7f7277e00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f727810e3a0 con 0x7f7278102820 2024-05-08T06:04:10.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.305+0000 7f7277e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f727810e220 con 0x7f72781043d0 2024-05-08T06:04:10.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.305+0000 7f7277e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f727810e520 con 0x7f7278103a10 2024-05-08T06:04:10.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.305+0000 7f7276a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7278102820 0x7f7278198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:10.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.305+0000 7f7276a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7278102820 0x7f7278198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48130/0 (socket says 172.21.15.112:48130) 2024-05-08T06:04:10.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.305+0000 7f7276a00640 1 -- 172.21.15.112:0/1456795926 learned_addr learned my addr 172.21.15.112:0/1456795926 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:10.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.305+0000 7f7276000640 1 --2- 172.21.15.112:0/1456795926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7278103a10 0x7f7278199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:10.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.305+0000 7f7277400640 1 --2- 172.21.15.112:0/1456795926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72781043d0 0x7f72781a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:10.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.306+0000 7f7276a00640 1 -- 172.21.15.112:0/1456795926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7278103a10 msgr2=0x7f7278199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.306+0000 7f7276a00640 1 --2- 172.21.15.112:0/1456795926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7278103a10 0x7f7278199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.306+0000 7f7276a00640 1 -- 172.21.15.112:0/1456795926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72781043d0 msgr2=0x7f72781a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.306+0000 7f7276a00640 1 --2- 172.21.15.112:0/1456795926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72781043d0 0x7f72781a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.306+0000 7f7276a00640 1 -- 172.21.15.112:0/1456795926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72781a06f0 con 0x7f7278102820 2024-05-08T06:04:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.306+0000 7f7277400640 1 --2- 172.21.15.112:0/1456795926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72781043d0 0x7f72781a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:04:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.306+0000 7f7276a00640 1 --2- 172.21.15.112:0/1456795926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7278102820 0x7f7278198ad0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f726800e9c0 tx=0x7f726800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.306+0000 7f726f400640 1 -- 172.21.15.112:0/1456795926 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726800ccc0 con 0x7f7278102820 2024-05-08T06:04:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.306+0000 7f726f400640 1 -- 172.21.15.112:0/1456795926 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72680035f0 con 0x7f7278102820 2024-05-08T06:04:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.306+0000 7f7277e00640 1 -- 172.21.15.112:0/1456795926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72781a09e0 con 0x7f7278102820 2024-05-08T06:04:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.306+0000 7f726f400640 1 -- 172.21.15.112:0/1456795926 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726800ce60 con 0x7f7278102820 2024-05-08T06:04:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.306+0000 7f7277e00640 1 -- 172.21.15.112:0/1456795926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72781a0f00 con 0x7f7278102820 2024-05-08T06:04:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.307+0000 7f7277e00640 1 -- 172.21.15.112:0/1456795926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7278109c10 con 0x7f7278102820 2024-05-08T06:04:10.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.310+0000 7f726f400640 1 -- 172.21.15.112:0/1456795926 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7268004000 con 0x7f7278102820 2024-05-08T06:04:10.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.311+0000 7f726f400640 1 --2- 172.21.15.112:0/1456795926 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f725c0b2870 0x7f725c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:10.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.312+0000 7f726f400640 1 -- 172.21.15.112:0/1456795926 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f72680e2890 con 0x7f7278102820 2024-05-08T06:04:10.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.312+0000 7f7276000640 1 --2- 172.21.15.112:0/1456795926 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f725c0b2870 0x7f725c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:10.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.313+0000 7f7276000640 1 --2- 172.21.15.112:0/1456795926 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f725c0b2870 0x7f725c0b4d30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7264031500 tx=0x7f7264005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:10.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.314+0000 7f726f400640 1 -- 172.21.15.112:0/1456795926 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f72680aa520 con 0x7f7278102820 2024-05-08T06:04:10.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:10 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2266056960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T06:04:10.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:10 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3921926251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T06:04:10.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.569+0000 7f7277e00640 1 -- 172.21.15.112:0/1456795926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f72780635b0 con 0x7f7278102820 2024-05-08T06:04:10.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.586+0000 7f726f400640 1 -- 172.21.15.112:0/1456795926 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+815648 (secure 0 0 0) 0x7f72680a9e20 con 0x7f7278102820 2024-05-08T06:04:10.598 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T06:04:10.598 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:04:10.598 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 18, 2024-05-08T06:04:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: "active_gid": 14152, 2024-05-08T06:04:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: "active_name": "x", 2024-05-08T06:04:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: "active_addrs": { 2024-05-08T06:04:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-05-08T06:04:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-05-08T06:04:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6800", 2024-05-08T06:04:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1211679607 2024-05-08T06:04:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-05-08T06:04:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6801", 2024-05-08T06:04:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1211679607 2024-05-08T06:04:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-08T06:04:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: "active_addr": "172.21.15.112:6801/1211679607", 2024-05-08T06:04:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: "active_change": "2024-05-08T05:43:03.313846+0000", 2024-05-08T06:04:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: "active_mgr_features": 4540701547738038271, 2024-05-08T06:04:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: "available": true, 2024-05-08T06:04:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: "standbys": [ 2024-05-08T06:04:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: "gid": 24109, 2024-05-08T06:04:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "z", 2024-05-08T06:04:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr_features": 4540701547738038271, 2024-05-08T06:04:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-05-08T06:04:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-05-08T06:04:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-08T06:04:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-08T06:04:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-08T06:04:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T06:04:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-05-08T06:04:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-05-08T06:04:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-05-08T06:04:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-05-08T06:04:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-05-08T06:04:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-05-08T06:04:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-05-08T06:04:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-05-08T06:04:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-05-08T06:04:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-05-08T06:04:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T06:04:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-05-08T06:04:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-05-08T06:04:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-05-08T06:04:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-05-08T06:04:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-05-08T06:04:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T06:04:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-05-08T06:04:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-05-08T06:04:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-05-08T06:04:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-05-08T06:04:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T06:04:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:04:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-05-08T06:04:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-05-08T06:04:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-08T06:04:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-05-08T06:04:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-08T06:04:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-05-08T06:04:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T06:04:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-05-08T06:04:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-05-08T06:04:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T06:04:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T06:04:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T06:04:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-05-08T06:04:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-05-08T06:04:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-05-08T06:04:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-05-08T06:04:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-05-08T06:04:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-05-08T06:04:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T06:04:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-05-08T06:04:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-05-08T06:04:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-05-08T06:04:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T06:04:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:04:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-05-08T06:04:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-05-08T06:04:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-08T06:04:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-05-08T06:04:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-05-08T06:04:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-05-08T06:04:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T06:04:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-05-08T06:04:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-05-08T06:04:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-05-08T06:04:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-05-08T06:04:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-05-08T06:04:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-05-08T06:04:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-05-08T06:04:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-05-08T06:04:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-05-08T06:04:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-05-08T06:04:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-05-08T06:04:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.632 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T06:04:10.632 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.632 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.632 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.632 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.632 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.632 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.632 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.632 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.632 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.632 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.632 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.632 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.632 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-05-08T06:04:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-05-08T06:04:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-08T06:04:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T06:04:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T06:04:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-05-08T06:04:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-05-08T06:04:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-05-08T06:04:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T06:04:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-05-08T06:04:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-05-08T06:04:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T06:04:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T06:04:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-05-08T06:04:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-05-08T06:04:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-08T06:04:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T06:04:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-05-08T06:04:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-05-08T06:04:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-05-08T06:04:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-05-08T06:04:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-05-08T06:04:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-08T06:04:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-05-08T06:04:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-05-08T06:04:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T06:04:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-05-08T06:04:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-05-08T06:04:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T06:04:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-05-08T06:04:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-05-08T06:04:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T06:04:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-08T06:04:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-05-08T06:04:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-05-08T06:04:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T06:04:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-05-08T06:04:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.647 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-05-08T06:04:10.647 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-05-08T06:04:10.647 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.649 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.649 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.649 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T06:04:10.649 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.649 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.649 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.649 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-05-08T06:04:10.649 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.649 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-05-08T06:04:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-05-08T06:04:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-08T06:04:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-08T06:04:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-05-08T06:04:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-05-08T06:04:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T06:04:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-05-08T06:04:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-05-08T06:04:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-05-08T06:04:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T06:04:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-05-08T06:04:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-05-08T06:04:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-05-08T06:04:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T06:04:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-05-08T06:04:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-05-08T06:04:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-05-08T06:04:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T06:04:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-05-08T06:04:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-05-08T06:04:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-05-08T06:04:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T06:04:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-05-08T06:04:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-05-08T06:04:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-05-08T06:04:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T06:04:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-05-08T06:04:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-05-08T06:04:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-05-08T06:04:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T06:04:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-08T06:04:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-05-08T06:04:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-05-08T06:04:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-08T06:04:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-05-08T06:04:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-05-08T06:04:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-08T06:04:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-05-08T06:04:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-05-08T06:04:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-05-08T06:04:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T06:04:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-05-08T06:04:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-05-08T06:04:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-05-08T06:04:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T06:04:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-05-08T06:04:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-05-08T06:04:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-05-08T06:04:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T06:04:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-05-08T06:04:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-05-08T06:04:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-05-08T06:04:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-05-08T06:04:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-05-08T06:04:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-05-08T06:04:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T06:04:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-05-08T06:04:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-05-08T06:04:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-08T06:04:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-05-08T06:04:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-05-08T06:04:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-05-08T06:04:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-08T06:04:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-08T06:04:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-05-08T06:04:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-05-08T06:04:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T06:04:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-05-08T06:04:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-05-08T06:04:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T06:04:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T06:04:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T06:04:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-05-08T06:04:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T06:04:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-05-08T06:04:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-05-08T06:04:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T06:04:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-05-08T06:04:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-05-08T06:04:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-05-08T06:04:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-05-08T06:04:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-05-08T06:04:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-05-08T06:04:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-05-08T06:04:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-05-08T06:04:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T06:04:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-05-08T06:04:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-05-08T06:04:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-05-08T06:04:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T06:04:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-05-08T06:04:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-05-08T06:04:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T06:04:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T06:04:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-05-08T06:04:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-05-08T06:04:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T06:04:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-05-08T06:04:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-05-08T06:04:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T06:04:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-05-08T06:04:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-05-08T06:04:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T06:04:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-05-08T06:04:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-05-08T06:04:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T06:04:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-05-08T06:04:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-05-08T06:04:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T06:04:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-05-08T06:04:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-05-08T06:04:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-05-08T06:04:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-05-08T06:04:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-05-08T06:04:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-05-08T06:04:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T06:04:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-05-08T06:04:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-05-08T06:04:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T06:04:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-05-08T06:04:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-05-08T06:04:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T06:04:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-05-08T06:04:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-05-08T06:04:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T06:04:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-05-08T06:04:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-05-08T06:04:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T06:04:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-05-08T06:04:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.705 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.705 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.705 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.705 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.705 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.705 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.705 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.705 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.705 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.705 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.705 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.705 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.705 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.706 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-05-08T06:04:10.706 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-05-08T06:04:10.706 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.706 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.706 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.706 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-05-08T06:04:10.706 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.706 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.706 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.706 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T06:04:10.706 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.706 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:10.707 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.708 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.708 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.708 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.708 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.708 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.708 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.708 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T06:04:10.708 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T06:04:10.708 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.708 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.708 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.708 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.708 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.708 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.709 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.709 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.709 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.709 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.709 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.709 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T06:04:10.709 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T06:04:10.709 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.709 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.709 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.709 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.709 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.709 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.710 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.710 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.710 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.710 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.710 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.710 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-05-08T06:04:10.710 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T06:04:10.710 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.710 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.710 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.710 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.710 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.711 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.711 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.711 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.711 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.711 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.711 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.711 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T06:04:10.711 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T06:04:10.711 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.711 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.711 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.711 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.711 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.711 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.712 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.713 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.713 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.713 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.713 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T06:04:10.713 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T06:04:10.713 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.713 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.713 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.713 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.713 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.713 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.713 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.713 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.713 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.714 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.714 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.714 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T06:04:10.714 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T06:04:10.714 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.714 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.714 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.714 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.714 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.714 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.714 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.714 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.714 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.715 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.715 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.715 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T06:04:10.715 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T06:04:10.715 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.715 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.715 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.715 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.715 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.715 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.715 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.715 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.715 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.715 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.716 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.716 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T06:04:10.716 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T06:04:10.716 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.716 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.716 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.716 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.716 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.716 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.716 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.716 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.716 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.716 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.717 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.717 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T06:04:10.717 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T06:04:10.717 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.717 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.717 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.717 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.717 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.717 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.717 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.717 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.717 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.717 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.717 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.718 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T06:04:10.718 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T06:04:10.718 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.718 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.718 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.718 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.718 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.718 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.718 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.718 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.718 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.718 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.718 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.719 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T06:04:10.719 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T06:04:10.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.719 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.719 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.719 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.719 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.719 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.719 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.719 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.719 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.719 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.720 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.720 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T06:04:10.720 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T06:04:10.720 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.720 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.720 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.720 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.720 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.720 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.720 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.720 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.720 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.720 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.720 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.721 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T06:04:10.721 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T06:04:10.721 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.721 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.721 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.721 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-08T06:04:10.721 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.721 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.721 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.721 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.721 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.721 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.721 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.722 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T06:04:10.722 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T06:04:10.722 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.722 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.722 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.722 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.722 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.722 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.722 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.722 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.722 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.722 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.723 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.723 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-05-08T06:04:10.723 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-05-08T06:04:10.723 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.723 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.723 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.723 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.723 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.723 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.723 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.723 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.723 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.724 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.724 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.724 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T06:04:10.724 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T06:04:10.724 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.724 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.724 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.724 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-08T06:04:10.724 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.724 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.724 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.724 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.724 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.725 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.725 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.725 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T06:04:10.725 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T06:04:10.725 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.725 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.725 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.725 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.725 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.725 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.725 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.725 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.725 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.726 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.726 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.726 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T06:04:10.726 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T06:04:10.726 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.726 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.726 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.726 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.726 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.726 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.726 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.726 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.727 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.727 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.727 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.727 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T06:04:10.727 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T06:04:10.727 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.727 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.727 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.727 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.727 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.727 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.727 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.727 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.728 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.728 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.728 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.728 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T06:04:10.728 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T06:04:10.728 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.728 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.728 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.728 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.728 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.728 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.728 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.728 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.728 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.729 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.729 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.729 INFO:teuthology.orchestra.run.smithi112.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-08T06:04:10.729 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-08T06:04:10.729 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.729 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.729 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.729 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "[]", 2024-05-08T06:04:10.729 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.729 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.729 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.729 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.729 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-05-08T06:04:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-08T06:04:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-05-08T06:04:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.731 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.731 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.731 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T06:04:10.731 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T06:04:10.731 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.731 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.731 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.731 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.731 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.731 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.731 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.731 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.731 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.732 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.732 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.732 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T06:04:10.732 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T06:04:10.732 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.732 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.732 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.732 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.732 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.732 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.732 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.732 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.732 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.732 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.733 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.733 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T06:04:10.733 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T06:04:10.733 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.733 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.733 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.733 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.733 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.733 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.733 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.733 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.733 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.733 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.734 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.734 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T06:04:10.734 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T06:04:10.734 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.734 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.734 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.734 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.734 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.734 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.734 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.734 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.734 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.734 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T06:04:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T06:04:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.735 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T06:04:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T06:04:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.736 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.737 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.737 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.737 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T06:04:10.737 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T06:04:10.737 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.737 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.737 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.737 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.737 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.737 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.737 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.737 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.737 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.738 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.738 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.738 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T06:04:10.738 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T06:04:10.738 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.738 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.738 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.738 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.738 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.738 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.738 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.738 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.738 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.739 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.739 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.739 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T06:04:10.739 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T06:04:10.739 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.739 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.739 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.739 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.739 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.739 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.739 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.739 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.745 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.745 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.745 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.745 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T06:04:10.745 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T06:04:10.745 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.745 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.745 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.745 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.745 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.745 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.745 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.745 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.746 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.746 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.746 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.746 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T06:04:10.746 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T06:04:10.746 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.746 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.746 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.746 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-08T06:04:10.746 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.746 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.746 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.746 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.747 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.747 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.747 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.747 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T06:04:10.747 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T06:04:10.747 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.747 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.747 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.747 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.747 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.747 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.747 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.747 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.748 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.748 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.748 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.748 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T06:04:10.748 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T06:04:10.748 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.748 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.748 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.748 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.748 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.748 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.748 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.749 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.749 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.749 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.749 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.749 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T06:04:10.749 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T06:04:10.749 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.749 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.749 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.749 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.749 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.749 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.749 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.750 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.750 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.750 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.750 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.750 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.750 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T06:04:10.750 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T06:04:10.750 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.750 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.750 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.750 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.750 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.750 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.751 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.751 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.751 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.751 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.751 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.751 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T06:04:10.751 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T06:04:10.751 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.751 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.751 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.751 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:10.751 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.751 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.752 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.752 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.752 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.752 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.752 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.752 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.752 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T06:04:10.752 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T06:04:10.752 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.752 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.752 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.752 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-08T06:04:10.753 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.753 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.753 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.753 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.753 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.753 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.753 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.753 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-05-08T06:04:10.753 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-05-08T06:04:10.753 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.753 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.753 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.754 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.754 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.754 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.754 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.754 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.754 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.754 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.754 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.754 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.754 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-05-08T06:04:10.754 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-05-08T06:04:10.754 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.754 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.755 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.755 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.755 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.755 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.755 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.755 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.755 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.755 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.755 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.755 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-05-08T06:04:10.755 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-05-08T06:04:10.755 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.755 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.755 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-05-08T06:04:10.756 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.757 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.757 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.757 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.757 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.757 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.757 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.757 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-05-08T06:04:10.757 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-05-08T06:04:10.757 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.757 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.757 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.757 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.758 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.758 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.758 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.758 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.758 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.758 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.758 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.758 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.758 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.758 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.758 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.758 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.758 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.759 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.760 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.760 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.760 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.760 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.760 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.760 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.760 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.760 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.760 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.760 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.760 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.760 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.760 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.760 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.761 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.761 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.761 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.761 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.761 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.761 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.761 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.761 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.761 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.761 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.761 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.761 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.761 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.761 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.761 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.762 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.762 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.762 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.762 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.762 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.762 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.762 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.762 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.762 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.762 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.762 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.763 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.763 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.763 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.763 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.763 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.763 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-05-08T06:04:10.763 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-05-08T06:04:10.763 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.763 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.763 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.763 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.763 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.764 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.764 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.764 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-05-08T06:04:10.764 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.764 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.764 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.764 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-05-08T06:04:10.764 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T06:04:10.764 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.764 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.764 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.764 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.764 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.764 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.765 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.765 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.765 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.765 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.765 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.765 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-08T06:04:10.765 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-08T06:04:10.765 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.765 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.765 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.765 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-05-08T06:04:10.765 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.766 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.766 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.766 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.766 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.766 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.766 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.766 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-08T06:04:10.766 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-08T06:04:10.766 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.766 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.766 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.766 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-05-08T06:04:10.766 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.767 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.767 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.767 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.767 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.767 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.767 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.767 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.767 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.767 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.767 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.767 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.767 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.768 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.768 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.768 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.768 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.768 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.768 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.768 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.768 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-05-08T06:04:10.768 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-05-08T06:04:10.768 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.768 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.768 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.768 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.768 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.769 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.769 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.769 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.769 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.769 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.769 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.769 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-05-08T06:04:10.769 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-05-08T06:04:10.769 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.769 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.769 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.769 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-05-08T06:04:10.770 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.770 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.770 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.770 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.770 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.770 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.770 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.770 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-05-08T06:04:10.770 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-05-08T06:04:10.770 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.770 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.770 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.771 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-05-08T06:04:10.771 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.771 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.771 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.771 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.771 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-05-08T06:04:10.771 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.771 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.771 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.771 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.771 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.771 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-05-08T06:04:10.771 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-05-08T06:04:10.772 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.772 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.772 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.772 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-05-08T06:04:10.772 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-05-08T06:04:10.772 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-05-08T06:04:10.772 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.772 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.772 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.772 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.772 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.772 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-05-08T06:04:10.772 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-05-08T06:04:10.773 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.773 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.773 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.773 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.773 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.773 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.773 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.773 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.773 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.773 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.773 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.773 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.773 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.774 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-05-08T06:04:10.774 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.774 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.774 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.774 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-05-08T06:04:10.774 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-05-08T06:04:10.774 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.774 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.774 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.774 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.774 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.774 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.775 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.775 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-05-08T06:04:10.775 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.775 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.775 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.775 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.775 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.775 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.775 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.775 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.775 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.775 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.775 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.776 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.776 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.776 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.776 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.776 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.776 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.776 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.776 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.776 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.776 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.776 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.776 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.777 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.777 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.777 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.777 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.777 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.777 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.777 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.777 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.777 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.777 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.777 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.777 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.778 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.778 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.778 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.778 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.778 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.778 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.778 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.778 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.778 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.778 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.778 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.778 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.779 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.779 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.779 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.779 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.779 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.779 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.779 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.779 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.779 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.779 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.779 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.779 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.779 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.780 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.780 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.780 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.780 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.780 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.780 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.780 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.780 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.780 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-05-08T06:04:10.780 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-05-08T06:04:10.780 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.780 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.781 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.781 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-05-08T06:04:10.781 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.781 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.781 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.781 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T06:04:10.781 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.781 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.781 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.781 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-05-08T06:04:10.781 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-05-08T06:04:10.781 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.781 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.781 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.782 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-05-08T06:04:10.782 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.782 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.782 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.782 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T06:04:10.782 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.782 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.782 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.782 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-05-08T06:04:10.782 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-05-08T06:04:10.782 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.782 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.783 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.783 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-05-08T06:04:10.783 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.783 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.783 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.783 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-05-08T06:04:10.783 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.783 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.783 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.783 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-05-08T06:04:10.783 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-05-08T06:04:10.783 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.784 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.784 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.784 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-05-08T06:04:10.784 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.784 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.784 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.784 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T06:04:10.784 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.784 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.784 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.784 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-05-08T06:04:10.784 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-05-08T06:04:10.784 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.785 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.786 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-08T06:04:10.786 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.786 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.786 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.786 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T06:04:10.786 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.786 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.786 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.786 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.786 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.786 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.786 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.787 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-05-08T06:04:10.788 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.788 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.788 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.788 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T06:04:10.788 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.788 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.788 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.788 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.788 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.788 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.788 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-05-08T06:04:10.788 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.788 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.789 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.789 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.789 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.789 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.789 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.789 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.789 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.789 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.789 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.789 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.789 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.789 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.789 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.789 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.790 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.790 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.790 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.790 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.790 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.790 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.790 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.790 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.790 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.790 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.790 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.790 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.790 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.791 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.791 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.791 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.791 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.791 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.791 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.791 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.791 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.791 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.791 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.791 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.791 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.791 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.792 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.792 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.792 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.792 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.792 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.792 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.792 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.792 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.792 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.792 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.792 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.792 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.792 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.792 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-05-08T06:04:10.794 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-05-08T06:04:10.794 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.794 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.794 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.794 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-05-08T06:04:10.794 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.794 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.794 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.794 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.794 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.794 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.794 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.794 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-05-08T06:04:10.795 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-05-08T06:04:10.795 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.795 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.795 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.795 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-05-08T06:04:10.795 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.795 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.795 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.795 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.795 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.795 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.795 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.795 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-08T06:04:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-08T06:04:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-08T06:04:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-05-08T06:04:10.798 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-05-08T06:04:10.798 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-05-08T06:04:10.798 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.798 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-05-08T06:04:10.798 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-05-08T06:04:10.798 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.798 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.798 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.798 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-05-08T06:04:10.798 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.798 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.798 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.798 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T06:04:10.798 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-05-08T06:04:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-05-08T06:04:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-05-08T06:04:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T06:04:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.803 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.803 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.803 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.803 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.803 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.803 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.803 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.803 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.803 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.803 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.803 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.803 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.803 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.804 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.804 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.804 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.804 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.804 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.804 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.804 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.804 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.804 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.804 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.804 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.804 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.804 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.805 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.805 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.805 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.805 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.805 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.805 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.805 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.805 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.805 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.805 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.805 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.805 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.805 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.805 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.806 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.806 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.806 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.806 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.806 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.806 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.806 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.806 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.806 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.806 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.806 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.806 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.807 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-05-08T06:04:10.807 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-05-08T06:04:10.807 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.807 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.807 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.807 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.807 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.807 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.807 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.807 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-05-08T06:04:10.807 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.807 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.807 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.808 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-05-08T06:04:10.808 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-05-08T06:04:10.808 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.808 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.808 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.808 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-05-08T06:04:10.808 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.808 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.808 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.808 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-05-08T06:04:10.808 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.808 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.808 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.809 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-05-08T06:04:10.810 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-05-08T06:04:10.810 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.810 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.810 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.810 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-05-08T06:04:10.810 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.810 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.810 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.810 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:04:10.810 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.810 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.810 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.810 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-05-08T06:04:10.811 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-05-08T06:04:10.811 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.811 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.811 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.811 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-08T06:04:10.811 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-08T06:04:10.811 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-05-08T06:04:10.811 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.811 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T06:04:10.811 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.811 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.811 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.811 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-05-08T06:04:10.811 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.812 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.813 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.813 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-05-08T06:04:10.813 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.813 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.813 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.813 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:04:10.813 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.813 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.813 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.813 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.813 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.813 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.813 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-05-08T06:04:10.814 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.814 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.814 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.814 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.814 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.814 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.814 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.814 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.814 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.814 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.814 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.814 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.814 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.814 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.814 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.815 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.815 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.815 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.815 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.815 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.815 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.815 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.815 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.815 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.815 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.815 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.816 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.816 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.816 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.816 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.816 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.816 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.816 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.816 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.816 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.816 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.816 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.816 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.816 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.816 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.817 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.817 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.817 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.817 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.817 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.817 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.817 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.817 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.817 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.817 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.817 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.817 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.817 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.817 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.818 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.818 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.818 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.818 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.818 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.818 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.818 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.818 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.818 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.818 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.818 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.818 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.818 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.818 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.819 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.819 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.819 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.819 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.819 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.819 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.819 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.819 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.819 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.819 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.819 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.819 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.819 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.819 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.820 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.820 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.820 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-05-08T06:04:10.820 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.820 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.820 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.820 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.820 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.820 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.820 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.820 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.820 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.820 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.821 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.821 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.821 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.821 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.821 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.821 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.821 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.821 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.821 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.821 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.821 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.821 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.821 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.821 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.822 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.822 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.822 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.822 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.822 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.822 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.822 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.822 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.822 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.822 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.822 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.822 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.822 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.822 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.823 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.823 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.823 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.823 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.823 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.823 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.823 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.823 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.823 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.823 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.823 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.823 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.823 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.823 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.823 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.824 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.824 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.824 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.824 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.824 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.824 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.824 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.824 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.824 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.824 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.824 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.824 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.824 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.824 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.825 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.825 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.825 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.825 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.825 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.825 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.825 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.825 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.825 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.825 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.825 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.825 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.825 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.825 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.826 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.826 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.826 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.826 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.826 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-05-08T06:04:10.826 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.826 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.826 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.826 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-05-08T06:04:10.826 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-05-08T06:04:10.826 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.826 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.827 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.827 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-05-08T06:04:10.827 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.827 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.827 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.827 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T06:04:10.827 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.827 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.827 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.827 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-05-08T06:04:10.827 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-05-08T06:04:10.827 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.827 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.827 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.828 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:10.828 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-05-08T06:04:10.828 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.828 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.828 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T06:04:10.828 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.828 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.828 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.828 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.828 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.828 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.828 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.828 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.829 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.829 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.829 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.829 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.829 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.829 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.829 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.829 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.829 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.829 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.829 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.829 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.829 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.829 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.829 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.830 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.830 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.830 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.830 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.830 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.830 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.830 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.830 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.830 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.830 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.830 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.830 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.831 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.831 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.831 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.831 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.831 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.831 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.831 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.831 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.831 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.831 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.831 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.831 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.831 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.832 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.832 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.832 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.832 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.832 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.832 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.832 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.832 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.832 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.832 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.832 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.832 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.832 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.833 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.833 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.833 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.833 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.833 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.833 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.833 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.833 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.833 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.833 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.833 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.833 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.833 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.833 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.834 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.834 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.834 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.834 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.834 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.834 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.834 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.834 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.834 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.834 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-05-08T06:04:10.834 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.834 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.835 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.835 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-05-08T06:04:10.835 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-05-08T06:04:10.835 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.835 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.835 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.835 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-05-08T06:04:10.835 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.835 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.835 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.835 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-05-08T06:04:10.835 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T06:04:10.835 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.835 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.836 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.836 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.836 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.836 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.836 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.836 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.836 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.836 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.836 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.836 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.836 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.836 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.836 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.836 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.837 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.838 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.838 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.838 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.838 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.838 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.838 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.838 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.838 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.838 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.838 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.838 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.838 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.838 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.838 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.839 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.839 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.839 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.839 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.839 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.839 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.839 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.839 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.839 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.839 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.839 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.839 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.839 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.839 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.840 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-05-08T06:04:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T06:04:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-05-08T06:04:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-05-08T06:04:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-05-08T06:04:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.842 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.842 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.842 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-05-08T06:04:10.842 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.842 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.842 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.842 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-05-08T06:04:10.842 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-05-08T06:04:10.842 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.842 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.842 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.842 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-05-08T06:04:10.842 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.843 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.843 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.843 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T06:04:10.843 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.843 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.843 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.843 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-05-08T06:04:10.843 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-05-08T06:04:10.843 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.843 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.843 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.843 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.843 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.843 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.844 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.845 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.845 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.845 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.845 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-05-08T06:04:10.845 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-05-08T06:04:10.845 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.845 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.845 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.845 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-05-08T06:04:10.845 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.845 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.845 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.845 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T06:04:10.845 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T06:04:10.846 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.846 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.846 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.846 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.846 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.846 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-05-08T06:04:10.846 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.846 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.846 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.846 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.846 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.847 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.847 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.847 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.847 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.847 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.847 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.847 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.847 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.847 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.847 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.847 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.847 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.847 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.847 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.848 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.849 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.849 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.849 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.849 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.849 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.849 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.849 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.849 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.849 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.849 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.849 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.849 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.849 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.849 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.850 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.850 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.850 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.850 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.850 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.850 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.850 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.850 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.850 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.850 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.850 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.850 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.851 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.852 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.852 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.852 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.852 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.852 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.852 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.852 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.852 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.852 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.852 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.852 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-05-08T06:04:10.852 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.852 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.853 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.853 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.853 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.853 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.853 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.853 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.853 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.853 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.853 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.853 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.853 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.853 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.853 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.853 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.854 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.854 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.854 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.854 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.854 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.854 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.854 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.854 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.854 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.854 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.854 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.854 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.855 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.855 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.855 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.855 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.855 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.855 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.855 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.855 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.855 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.855 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.855 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.855 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.855 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.855 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.856 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.856 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.856 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.856 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.856 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.856 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.856 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.856 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.856 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.856 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.856 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.856 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.856 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.856 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.857 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.857 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.857 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.857 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.857 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.857 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.857 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.857 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.857 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.857 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.857 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.857 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.857 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.858 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.858 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.858 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.858 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.858 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.858 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.858 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.858 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.858 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.858 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.858 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.859 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.859 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.859 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.859 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.859 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-05-08T06:04:10.859 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.859 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.859 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.859 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.859 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.859 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.860 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.860 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.860 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.860 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.860 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.860 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.860 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.860 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.860 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.860 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.860 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.860 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.860 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.861 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.861 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.861 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.861 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.861 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.861 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.861 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.861 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.861 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.861 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.861 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.861 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.861 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.862 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.862 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.862 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.862 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.862 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.862 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.862 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.862 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.862 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.862 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.862 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.863 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.863 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.863 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.863 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.863 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.863 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.863 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.863 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.863 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.863 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.863 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.863 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.863 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.864 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.864 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.864 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.864 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.864 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.864 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.864 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.864 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.864 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.864 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.864 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.864 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.865 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.865 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.865 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.865 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.865 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.865 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.865 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.865 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.865 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.865 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.865 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.865 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.865 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.866 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.866 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.866 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.866 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.866 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-05-08T06:04:10.866 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.866 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.866 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.866 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-05-08T06:04:10.866 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-05-08T06:04:10.866 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.866 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.867 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.867 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.867 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.867 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.867 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.867 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T06:04:10.867 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.867 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.867 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.867 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.867 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.867 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.867 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.868 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.868 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.868 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.868 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.868 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.868 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.868 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.868 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.868 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.868 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.868 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.868 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.868 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.868 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.869 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.869 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.869 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.869 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.869 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.869 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.869 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.869 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.869 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.869 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.869 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.869 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.869 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.870 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.870 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.870 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.870 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.870 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.870 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.870 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.870 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.870 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.870 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.870 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.871 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.871 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.871 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.871 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.871 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.871 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.871 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.871 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.871 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.871 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.871 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.871 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.871 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.872 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.873 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.873 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.873 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.873 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.873 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.873 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-05-08T06:04:10.873 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-05-08T06:04:10.873 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-05-08T06:04:10.873 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.873 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-05-08T06:04:10.873 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.873 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.873 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.873 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.874 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.874 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.874 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.874 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.874 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.874 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.874 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.874 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.874 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.874 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.874 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.874 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.874 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.875 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.875 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-05-08T06:04:10.875 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.875 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.875 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.875 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.875 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.875 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.875 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.875 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.875 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.875 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.877 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.877 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.877 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.877 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.877 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.877 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.877 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.877 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.877 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.877 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.877 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.877 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.877 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.877 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.878 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.878 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.878 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.878 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.878 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.878 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.878 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.878 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.878 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.878 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.878 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.878 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.878 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.878 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.879 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.879 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.879 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.879 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.879 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.879 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.879 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.879 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.879 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.879 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.879 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.879 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.879 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.879 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.880 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.880 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.880 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.880 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.880 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.880 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.880 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.880 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.880 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.880 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.880 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.880 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.880 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.880 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.882 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.882 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.882 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.882 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.882 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.882 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.882 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.882 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.882 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.882 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.882 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.882 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.882 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.883 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.883 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.883 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.883 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.883 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.883 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.883 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.883 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.883 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.883 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.883 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.883 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.883 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.883 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.884 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.884 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.884 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.884 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.884 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.884 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.884 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.884 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.884 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.884 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.884 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.884 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.884 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.884 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.885 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.885 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.885 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.885 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.885 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.885 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.885 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.885 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.885 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.885 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.885 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.885 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.885 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.885 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.886 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.886 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.886 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.886 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.886 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.886 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.886 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.886 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.886 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.886 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.886 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.886 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.886 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.886 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.887 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.887 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.887 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.887 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.887 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.887 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.887 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.887 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-05-08T06:04:10.887 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.887 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.887 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.887 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.887 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.887 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.888 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.889 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.889 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.889 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.889 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.889 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.889 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.889 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.889 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.889 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.889 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.889 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.889 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.889 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.890 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.890 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.890 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.890 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.890 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.890 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.890 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.890 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.890 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.890 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.890 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.890 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.890 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.890 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.891 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.891 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.891 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.891 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.891 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.891 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.891 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.891 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.891 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.891 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.891 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.891 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.891 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.891 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.892 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.893 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.894 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.894 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-05-08T06:04:10.894 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-05-08T06:04:10.894 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:10.894 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.894 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.894 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-05-08T06:04:10.894 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-05-08T06:04:10.894 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.894 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.894 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-05-08T06:04:10.894 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T06:04:10.894 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.895 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.895 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.895 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.895 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.895 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-05-08T06:04:10.895 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.895 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.895 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.895 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-05-08T06:04:10.895 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-05-08T06:04:10.895 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.895 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.895 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.896 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.896 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.896 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.896 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.896 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T06:04:10.896 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.896 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.896 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.896 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-05-08T06:04:10.896 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-05-08T06:04:10.896 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.896 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.896 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.896 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.897 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.898 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.898 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.898 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.898 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.898 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.898 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.898 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.898 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.898 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.898 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.898 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.898 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.899 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.899 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.899 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.899 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.899 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.899 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.899 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.899 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.899 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.899 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.899 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.899 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.899 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.899 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.900 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.901 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.902 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.903 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.904 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.905 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-05-08T06:04:10.905 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.905 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.905 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.905 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-05-08T06:04:10.905 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-05-08T06:04:10.905 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.905 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.905 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.905 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.905 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.905 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.905 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.905 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.905 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-05-08T06:04:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-05-08T06:04:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T06:04:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T06:04:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.907 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.907 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.907 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.907 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.907 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.907 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.907 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.907 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.907 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.907 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.907 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.907 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.907 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.907 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.908 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.908 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.908 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.908 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.908 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.908 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.908 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.908 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.908 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.908 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.909 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.909 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.909 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.909 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.909 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.909 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.909 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.909 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.909 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.909 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.909 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.909 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.909 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.909 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.910 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.910 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.910 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.910 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.910 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.910 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.910 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.910 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.910 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.910 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.910 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.910 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.910 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.910 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.911 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.912 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-05-08T06:04:10.912 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-05-08T06:04:10.912 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.912 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.912 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.912 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.912 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.912 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.912 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.912 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.912 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.912 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.912 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-05-08T06:04:10.914 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-05-08T06:04:10.914 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:10.914 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.914 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.914 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-05-08T06:04:10.914 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.914 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.914 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.914 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.914 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.914 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.914 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.914 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-08T06:04:10.915 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-08T06:04:10.915 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.915 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.915 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.915 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-05-08T06:04:10.915 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.915 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.915 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.915 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T06:04:10.915 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.915 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.915 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.915 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-08T06:04:10.915 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.916 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.917 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-05-08T06:04:10.918 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.918 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.918 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.918 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.918 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.918 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.918 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.918 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-05-08T06:04:10.918 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-05-08T06:04:10.918 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.918 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.918 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.918 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.919 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.920 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-05-08T06:04:10.920 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-05-08T06:04:10.920 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-05-08T06:04:10.920 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.920 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.920 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.920 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.920 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.920 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.920 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.920 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.920 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-05-08T06:04:10.920 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.922 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.922 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.922 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.922 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.922 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.922 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.922 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.922 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.922 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.922 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.922 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.922 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.922 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.922 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.923 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.923 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.923 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.923 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.923 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.923 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.923 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.923 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.923 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.923 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.923 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.923 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.923 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.923 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.925 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.925 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.925 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.925 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.925 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.925 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.925 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.925 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.925 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.925 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.925 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.925 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-05-08T06:04:10.925 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-05-08T06:04:10.925 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.926 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.926 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.926 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:10.926 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.926 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.926 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.926 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.926 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.926 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.926 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.926 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-05-08T06:04:10.926 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T06:04:10.926 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.927 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.927 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.927 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.927 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.927 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.927 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.927 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.927 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.927 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.927 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.927 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.927 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.927 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.927 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.928 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.928 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.928 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.928 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.928 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.928 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.928 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.928 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.928 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.928 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-05-08T06:04:10.928 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-05-08T06:04:10.928 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.928 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.929 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.929 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.929 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.929 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.929 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.929 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.929 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.929 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.929 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.929 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.929 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.929 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-05-08T06:04:10.929 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.930 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.930 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.930 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-05-08T06:04:10.930 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-05-08T06:04:10.930 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.930 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.930 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.930 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.930 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.930 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.930 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.930 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.931 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.931 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.931 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.931 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-05-08T06:04:10.931 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-05-08T06:04:10.931 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.931 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.931 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.931 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.931 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.931 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.931 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.931 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.931 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.932 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.932 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.932 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.932 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.932 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.932 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.932 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.932 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.932 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.932 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.932 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.932 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.932 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.934 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.934 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.934 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.934 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.934 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.934 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.934 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.934 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.934 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.934 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.934 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.934 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.935 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.935 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.935 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.935 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.935 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.935 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.935 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.935 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.935 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.935 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.935 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.935 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.935 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.935 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.935 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.936 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.936 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.936 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.936 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.936 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.936 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.936 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.936 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.936 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.936 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.936 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.936 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.936 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.937 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.937 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-08T06:04:10.937 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-08T06:04:10.937 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.937 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.937 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.937 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.937 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.937 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.937 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.937 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.937 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.937 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.938 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.938 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-08T06:04:10.938 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-08T06:04:10.938 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.938 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.938 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.938 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.938 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.938 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.938 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.938 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.938 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.939 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.939 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.939 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.939 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.939 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.939 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.939 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.939 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.939 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.939 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.939 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.939 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.939 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.939 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.940 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.940 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.940 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.940 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.940 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-05-08T06:04:10.940 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.940 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.940 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.940 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.940 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.940 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.940 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.940 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.940 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.941 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.941 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.941 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.941 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.941 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.941 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.941 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.941 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.941 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.941 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.941 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.941 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.941 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.941 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.942 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.942 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.942 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.942 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.942 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.942 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.942 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.942 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.942 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.942 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.942 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.942 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.942 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.943 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.943 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.943 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.943 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.943 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.943 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.943 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.943 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.943 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.943 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.943 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.943 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.943 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.943 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.943 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.944 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.944 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.944 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.944 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.944 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.944 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.944 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.944 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.944 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.944 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.944 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.944 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.944 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.944 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.945 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.945 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.945 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.945 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.945 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.945 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.945 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.945 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.945 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.945 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.945 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.945 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.945 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.945 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.946 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.946 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.946 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.946 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.946 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.946 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.946 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.946 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-05-08T06:04:10.946 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.946 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.946 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.946 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.946 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.947 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.947 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.947 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.947 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.947 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.947 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.947 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.947 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.947 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.947 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.947 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.947 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.947 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.947 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.948 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.948 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.948 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.948 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.948 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.948 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.948 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.948 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.948 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.948 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.948 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.949 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.949 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.949 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.949 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.949 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.949 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.949 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.949 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.949 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.949 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.949 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.949 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.949 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.950 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.950 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.950 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.950 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.950 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.950 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.950 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.950 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.950 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.950 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.950 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.950 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.950 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.951 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.952 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.952 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.952 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.952 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.952 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.952 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.952 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.952 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.952 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.952 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.952 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.952 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.952 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-05-08T06:04:10.952 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-05-08T06:04:10.953 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.953 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.953 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.953 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-05-08T06:04:10.953 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.953 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.953 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.953 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T06:04:10.953 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.953 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.953 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.953 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.953 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.953 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.954 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-05-08T06:04:10.954 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.954 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.954 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.954 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.954 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.954 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.954 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.954 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.954 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.954 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.954 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.954 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.955 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.956 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.956 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.956 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.956 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.956 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.956 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.956 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.956 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.956 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.956 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.956 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.956 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.956 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.956 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.958 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.958 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.958 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.958 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.958 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.958 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.958 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.958 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.958 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.958 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.958 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.958 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.958 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.959 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-05-08T06:04:10.959 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-05-08T06:04:10.959 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.959 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.959 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.959 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.959 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.959 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.959 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.959 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.959 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.959 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.959 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.960 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-05-08T06:04:10.960 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-05-08T06:04:10.960 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.960 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.960 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.960 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-05-08T06:04:10.960 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.960 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.960 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.960 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.960 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.960 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.960 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.961 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-05-08T06:04:10.961 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-05-08T06:04:10.961 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.961 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.961 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.961 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.961 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.961 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.961 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.961 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.961 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.961 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.961 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.961 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-05-08T06:04:10.962 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-05-08T06:04:10.962 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.962 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.962 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.962 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.962 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.962 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.962 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.962 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.962 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.962 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.962 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.962 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-05-08T06:04:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-05-08T06:04:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-05-08T06:04:10.963 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-05-08T06:04:10.964 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.964 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.964 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.964 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.964 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.964 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.964 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.964 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.964 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.964 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.964 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.964 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-05-08T06:04:10.964 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-05-08T06:04:10.964 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.965 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.966 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.966 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.966 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.966 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.966 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.966 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.966 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.966 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.966 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-05-08T06:04:10.966 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-05-08T06:04:10.966 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.966 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.966 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.967 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.967 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-08T06:04:10.967 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-05-08T06:04:10.967 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.967 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.967 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.967 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.967 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.967 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.967 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.967 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.967 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.967 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.967 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.968 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.969 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.969 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.969 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.969 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.969 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.969 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.969 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-05-08T06:04:10.969 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-05-08T06:04:10.969 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.969 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.969 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.969 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.969 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.970 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.970 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.970 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.970 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.970 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.970 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.970 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-05-08T06:04:10.970 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-05-08T06:04:10.970 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.970 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.970 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.970 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.970 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.970 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smb", 2024-05-08T06:04:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.972 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.972 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.972 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.972 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.972 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.972 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.972 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.972 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.972 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.972 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.972 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.972 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.972 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.972 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.973 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.973 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.973 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.973 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.973 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.973 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.973 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.973 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.973 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.973 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.973 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.973 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.973 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.974 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.974 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.974 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.974 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.974 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.974 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.974 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.974 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.974 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.974 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.974 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.974 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.975 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.975 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.975 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.975 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.975 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.975 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.975 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.975 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.975 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.975 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.975 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.975 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.976 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.976 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.976 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.976 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.976 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.976 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.976 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.976 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.976 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.976 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.976 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.976 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.976 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.977 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.977 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.977 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.977 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.977 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.977 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.977 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.977 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.977 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.977 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.977 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.977 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.977 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.978 INFO:teuthology.orchestra.run.smithi112.stdout: "update_orchestration": { 2024-05-08T06:04:10.978 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "update_orchestration", 2024-05-08T06:04:10.978 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.978 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.978 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:10.978 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:10.978 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.978 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.978 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.978 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-08T06:04:10.978 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.978 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.978 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.978 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.979 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.979 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.979 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-05-08T06:04:10.979 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.979 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.979 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.979 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-05-08T06:04:10.979 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-05-08T06:04:10.979 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.979 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.979 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.979 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.979 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.979 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.980 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.980 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T06:04:10.980 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.980 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.980 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.980 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-05-08T06:04:10.980 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-05-08T06:04:10.980 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.980 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.980 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.980 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.980 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.980 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.981 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.981 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-05-08T06:04:10.981 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.981 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.981 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.981 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.981 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.981 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.981 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.981 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.981 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.981 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.981 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.981 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.982 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.982 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.982 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.982 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.982 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.982 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.982 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.982 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.982 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.982 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.982 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.982 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.982 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.983 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.983 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.983 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.983 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.983 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.983 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.983 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.983 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.983 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.983 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.983 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.983 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.983 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.983 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.984 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.984 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.984 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.984 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.984 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.984 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.984 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.984 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.984 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.984 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.984 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.984 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.984 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.984 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.984 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.985 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.986 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.986 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.986 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.986 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.986 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.986 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.986 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.986 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.986 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.986 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.986 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.986 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.986 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.986 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.987 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.988 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.988 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.988 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.988 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.988 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.988 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.988 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.988 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.988 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.988 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.988 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.988 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.988 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.988 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.989 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.989 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.989 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.989 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.989 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.989 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.989 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.989 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.989 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.989 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.989 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.989 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.989 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.989 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.990 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.990 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.990 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.990 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.990 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.990 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.990 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.990 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.990 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.990 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.990 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.990 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.990 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.991 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.991 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.991 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.991 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.991 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.991 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.991 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.991 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.991 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.991 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.991 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.991 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.991 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.992 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.992 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.992 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.992 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.992 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.992 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.992 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.992 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.992 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.992 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.992 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.992 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.992 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.992 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.993 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.993 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.993 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.993 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.993 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-05-08T06:04:10.993 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.993 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:10.993 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:10.993 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:10.993 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:10.993 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.993 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.994 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.994 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:10.994 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.994 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.994 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.994 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.994 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.994 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.994 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.994 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.994 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.994 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.994 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.994 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.995 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.995 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.995 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:10.995 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:10.995 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.995 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.995 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.995 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.995 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.995 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.995 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.995 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.995 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.996 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.996 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.996 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:10.996 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:10.996 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:10.996 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.996 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.996 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:10.996 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.996 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.996 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:10.996 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:10.996 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:10.997 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:10.997 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:10.997 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:10.997 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:10.997 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:10.997 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.997 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.997 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.997 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.997 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:10.997 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:10.997 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:10.997 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:10.997 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.998 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:10.998 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.998 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.998 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.998 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.998 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.998 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.998 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.998 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:10.998 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:10.998 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:10.998 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:10.998 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:10.999 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:10.999 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:10.999 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:10.999 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:10.999 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:10.999 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:10.999 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:10.999 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:10.999 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.999 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:10.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:10.999 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:10.999 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-05-08T06:04:10.999 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:10.999 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.000 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.000 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-05-08T06:04:11.000 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-05-08T06:04:11.000 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.000 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.000 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.000 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T06:04:11.000 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.000 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.000 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.000 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.000 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.000 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.000 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.001 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-08T06:04:11.001 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-08T06:04:11.001 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.001 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.001 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.001 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-05-08T06:04:11.001 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.001 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.001 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.001 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.001 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.001 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.001 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.002 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.002 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.002 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.002 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.002 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.002 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.002 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.002 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.002 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.002 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.002 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.002 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.003 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.003 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.003 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.003 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.003 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.003 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.003 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.003 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.003 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.003 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.003 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.003 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.003 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.003 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.004 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.005 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.005 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.005 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.005 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.005 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.005 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.005 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.005 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.005 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.005 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.005 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.005 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.005 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.006 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.006 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.006 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.006 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.006 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.006 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.006 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.006 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.006 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.006 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.006 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.006 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.007 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.007 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.007 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.007 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.007 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.007 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.007 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.007 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.007 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.007 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.007 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.007 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.007 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.008 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.008 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.008 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-05-08T06:04:11.008 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.008 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.008 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.008 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-05-08T06:04:11.008 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-05-08T06:04:11.008 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.008 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.008 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.008 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.008 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.009 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.010 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T06:04:11.010 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.010 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.010 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.010 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-05-08T06:04:11.010 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-05-08T06:04:11.010 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.010 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.010 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.010 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.010 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.010 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.010 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T06:04:11.011 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.012 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.012 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.012 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-05-08T06:04:11.012 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-05-08T06:04:11.012 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.012 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.012 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.012 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.012 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.012 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.012 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.012 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T06:04:11.012 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.012 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.014 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-05-08T06:04:11.014 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-05-08T06:04:11.014 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.014 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.014 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.014 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.014 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.014 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.014 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.014 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.014 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.014 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.014 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.015 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-05-08T06:04:11.015 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-05-08T06:04:11.015 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.015 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.015 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.015 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T06:04:11.015 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.015 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.015 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.015 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.015 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.015 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.015 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.015 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-08T06:04:11.016 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.017 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.018 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-05-08T06:04:11.018 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.018 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.018 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.018 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.018 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.018 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.018 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.018 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-05-08T06:04:11.018 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-05-08T06:04:11.018 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.018 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.018 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.019 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.019 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.019 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.019 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.019 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.019 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.019 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.019 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.019 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-05-08T06:04:11.019 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-05-08T06:04:11.019 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.019 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.019 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.019 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.020 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.020 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.020 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.020 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.020 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.020 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.020 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.020 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.020 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.020 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.020 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.020 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.020 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.020 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.021 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.021 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.021 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.021 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.021 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.021 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.021 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.021 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.021 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.021 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.021 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.021 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.021 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.022 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.022 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.022 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.022 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.022 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.022 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.022 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.022 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.022 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.022 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.022 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.022 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.022 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.023 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.024 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.025 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.025 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.025 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-05-08T06:04:11.025 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-05-08T06:04:11.025 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.025 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.025 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.025 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.025 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.025 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.025 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.025 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.025 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.025 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.026 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.026 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-05-08T06:04:11.026 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-05-08T06:04:11.026 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.026 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.026 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.026 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.026 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.026 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.026 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.026 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.026 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.026 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.026 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.027 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.027 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.027 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.027 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.027 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.027 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.027 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.027 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.027 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.027 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.027 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.027 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.027 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.029 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.030 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.030 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.030 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.030 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.030 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.030 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.030 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.030 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.030 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.030 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.030 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.030 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.031 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.031 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.031 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.031 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.031 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.031 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.031 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.031 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.031 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.031 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.031 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.031 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.031 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.031 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.032 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.032 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.032 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.032 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.032 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.032 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.032 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.032 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.032 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.032 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.032 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.032 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.032 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.032 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.032 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.033 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.033 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.033 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.033 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.033 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.033 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.033 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.033 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.033 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.033 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.033 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.033 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.033 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.034 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.034 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.034 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.034 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.034 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.034 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.034 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.034 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.034 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.034 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.034 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.034 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.035 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.035 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.035 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.035 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.035 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.035 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-05-08T06:04:11.035 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.035 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.035 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.035 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.035 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.035 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.036 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.036 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.036 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.036 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.036 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.036 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.036 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.036 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.036 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.036 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.036 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.036 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.036 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.036 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.037 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.037 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.037 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.037 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.037 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.037 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.037 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.037 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.037 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.037 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.037 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.037 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.037 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.038 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.038 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.038 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.038 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.038 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.038 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.038 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.038 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.038 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.038 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.038 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.038 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.038 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.039 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.039 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.039 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.039 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.039 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.039 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.039 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.039 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.039 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.039 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.039 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.039 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.039 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.039 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.040 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.040 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.040 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.040 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.040 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.040 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.040 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.040 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.040 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.040 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-05-08T06:04:11.040 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-05-08T06:04:11.040 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.040 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.041 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.041 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-05-08T06:04:11.041 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.041 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.041 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.041 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T06:04:11.041 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.041 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.041 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.041 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-05-08T06:04:11.041 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-05-08T06:04:11.041 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.041 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.041 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.042 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.042 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.042 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.042 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.042 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-05-08T06:04:11.042 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.042 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.042 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.042 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-05-08T06:04:11.042 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-05-08T06:04:11.042 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.042 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.042 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.043 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.043 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.043 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.043 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.043 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T06:04:11.043 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.043 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.043 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.043 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-05-08T06:04:11.043 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T06:04:11.043 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.043 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.043 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.043 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.044 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.044 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.044 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.044 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T06:04:11.044 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.044 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.044 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.044 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.044 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.044 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.044 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.044 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.044 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.044 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-05-08T06:04:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-05-08T06:04:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-05-08T06:04:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-08T06:04:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-05-08T06:04:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-05-08T06:04:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-05-08T06:04:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-08T06:04:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-08T06:04:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-08T06:04:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-05-08T06:04:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-05-08T06:04:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-05-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-05-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-05-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.056 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.056 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.056 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.056 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.056 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-05-08T06:04:11.056 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-05-08T06:04:11.056 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.056 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.056 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.056 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T06:04:11.056 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.056 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.056 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: "gid": 24110, 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "y", 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr_features": 4540701547738038271, 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-05-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-05-08T06:04:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-05-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-05-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-05-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-05-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-05-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-05-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-05-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-05-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-05-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-05-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-05-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-05-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-05-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T06:04:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-05-08T06:04:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-05-08T06:04:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-05-08T06:04:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-05-08T06:04:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-05-08T06:04:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-05-08T06:04:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T06:04:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-05-08T06:04:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-05-08T06:04:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-05-08T06:04:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T06:04:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:04:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-05-08T06:04:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-05-08T06:04:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-08T06:04:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-05-08T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-05-08T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-05-08T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-05-08T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-08T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-05-08T06:04:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-08T06:04:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-05-08T06:04:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T06:04:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-05-08T06:04:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-05-08T06:04:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T06:04:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-05-08T06:04:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.078 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-05-08T06:04:11.078 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T06:04:11.078 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.078 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.078 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-05-08T06:04:11.078 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-05-08T06:04:11.078 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.078 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.078 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.078 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-05-08T06:04:11.078 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.078 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.079 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.079 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T06:04:11.079 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:04:11.079 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.079 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.079 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-05-08T06:04:11.079 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-05-08T06:04:11.079 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-08T06:04:11.079 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.079 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.079 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.079 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-05-08T06:04:11.079 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-05-08T06:04:11.079 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.080 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T06:04:11.080 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:04:11.080 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.080 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.080 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.080 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.080 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.080 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.080 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.080 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.080 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.080 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.080 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.080 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.081 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.081 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.081 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.081 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.081 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.081 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.081 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.081 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.081 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.081 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.081 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.081 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.081 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.081 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.082 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.082 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.082 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.082 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.082 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.082 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.082 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.082 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.082 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.082 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.082 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.082 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.082 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.082 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.083 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.083 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.083 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.083 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.083 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.083 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.083 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.083 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.083 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.083 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.083 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.083 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.083 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.083 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.083 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.085 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-05-08T06:04:11.086 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-05-08T06:04:11.086 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.086 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.086 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.086 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-05-08T06:04:11.086 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.086 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.086 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.086 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-05-08T06:04:11.086 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-05-08T06:04:11.086 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-05-08T06:04:11.086 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-05-08T06:04:11.086 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-05-08T06:04:11.086 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T06:04:11.087 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-08T06:04:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-08T06:04:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-08T06:04:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T06:04:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.090 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-05-08T06:04:11.090 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-05-08T06:04:11.090 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.090 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.090 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.090 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-08T06:04:11.090 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-08T06:04:11.090 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.090 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.090 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T06:04:11.090 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T06:04:11.090 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.090 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.091 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-05-08T06:04:11.091 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-05-08T06:04:11.091 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-08T06:04:11.091 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.091 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.091 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.091 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.091 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.091 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.091 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T06:04:11.091 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.091 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.091 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.092 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.092 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.092 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-05-08T06:04:11.092 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.092 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.092 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.092 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-05-08T06:04:11.092 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-05-08T06:04:11.092 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:11.092 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.092 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.092 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-05-08T06:04:11.093 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.093 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.093 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.093 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T06:04:11.093 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.093 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.093 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.093 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-05-08T06:04:11.093 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-05-08T06:04:11.093 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.093 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.093 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.093 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-05-08T06:04:11.093 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.094 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.094 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.094 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T06:04:11.094 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.094 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.094 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.094 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-05-08T06:04:11.094 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-05-08T06:04:11.094 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.094 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.094 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.094 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-05-08T06:04:11.094 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.094 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.095 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.095 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T06:04:11.095 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.095 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.095 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.095 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-05-08T06:04:11.095 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-05-08T06:04:11.095 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.095 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.095 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.095 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.095 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.095 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.095 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.096 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T06:04:11.096 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T06:04:11.096 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.096 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.096 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-05-08T06:04:11.096 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-05-08T06:04:11.096 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.096 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.096 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.096 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-08T06:04:11.096 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.096 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.096 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.097 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T06:04:11.097 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.097 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.097 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.097 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-05-08T06:04:11.097 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T06:04:11.097 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:11.097 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.097 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.097 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-05-08T06:04:11.097 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.097 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.097 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.097 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T06:04:11.098 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.098 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.098 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.098 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-05-08T06:04:11.098 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-05-08T06:04:11.098 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.098 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.098 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.098 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.098 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.098 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.098 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.098 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-05-08T06:04:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-05-08T06:04:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-05-08T06:04:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-08T06:04:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-05-08T06:04:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-05-08T06:04:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T06:04:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-05-08T06:04:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-05-08T06:04:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.101 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.102 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T06:04:11.102 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.102 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.102 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.102 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-05-08T06:04:11.102 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-05-08T06:04:11.102 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.102 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.102 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.102 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T06:04:11.102 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.102 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.103 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-05-08T06:04:11.104 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.104 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.104 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.104 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-05-08T06:04:11.104 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-05-08T06:04:11.104 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.104 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.104 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.104 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T06:04:11.104 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.104 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.104 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.104 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-05-08T06:04:11.104 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.105 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.105 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.105 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-05-08T06:04:11.105 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-05-08T06:04:11.105 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.105 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.105 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.105 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-08T06:04:11.105 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.105 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.105 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.105 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-08T06:04:11.105 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-05-08T06:04:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-05-08T06:04:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T06:04:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-05-08T06:04:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.106 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.107 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.107 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-05-08T06:04:11.107 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-05-08T06:04:11.107 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.107 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.107 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.107 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T06:04:11.107 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.107 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.107 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.107 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-05-08T06:04:11.107 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.107 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.108 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.108 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-05-08T06:04:11.108 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-05-08T06:04:11.108 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.108 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.108 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.108 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T06:04:11.108 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.108 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.108 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.108 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-05-08T06:04:11.108 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.108 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.108 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.109 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-05-08T06:04:11.110 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-05-08T06:04:11.110 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.110 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.110 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.110 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T06:04:11.110 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.110 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.110 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.110 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-05-08T06:04:11.110 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.110 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.110 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-05-08T06:04:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.113 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.113 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-08T06:04:11.113 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.113 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.113 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.113 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-05-08T06:04:11.113 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.113 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.113 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.113 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-05-08T06:04:11.113 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-05-08T06:04:11.113 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.113 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.114 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.114 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T06:04:11.114 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.114 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.114 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.114 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-08T06:04:11.114 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.114 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.114 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.114 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-05-08T06:04:11.114 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-05-08T06:04:11.114 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.114 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.115 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.115 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T06:04:11.115 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.115 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.115 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.115 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-05-08T06:04:11.115 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.115 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.115 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.115 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-05-08T06:04:11.115 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-05-08T06:04:11.115 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.115 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.116 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.116 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-08T06:04:11.116 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.116 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.116 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.116 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-05-08T06:04:11.116 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.116 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.116 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.116 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-05-08T06:04:11.116 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-05-08T06:04:11.116 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.116 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.116 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.117 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.118 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.118 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.118 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T06:04:11.118 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.118 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.118 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.118 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-05-08T06:04:11.118 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-05-08T06:04:11.118 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.118 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.118 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.118 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-08T06:04:11.119 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.119 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.119 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.119 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T06:04:11.119 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.119 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.119 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.119 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-05-08T06:04:11.119 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T06:04:11.119 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.119 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.119 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.119 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-05-08T06:04:11.119 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.120 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.120 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.120 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T06:04:11.120 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.120 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.120 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.120 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-05-08T06:04:11.120 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-05-08T06:04:11.120 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.120 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.120 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.120 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-05-08T06:04:11.120 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.120 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.121 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.121 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T06:04:11.121 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.121 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.121 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.121 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-05-08T06:04:11.121 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-05-08T06:04:11.121 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.121 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.121 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.121 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-05-08T06:04:11.121 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.121 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.122 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.122 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-05-08T06:04:11.122 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.122 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.122 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.122 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-05-08T06:04:11.122 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-05-08T06:04:11.122 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.122 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.122 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.122 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.122 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.122 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.123 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-05-08T06:04:11.124 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.124 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.124 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.124 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-05-08T06:04:11.124 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-05-08T06:04:11.124 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.124 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.124 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.124 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-08T06:04:11.124 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.124 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.124 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.124 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-08T06:04:11.125 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.125 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.125 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.125 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-05-08T06:04:11.125 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-05-08T06:04:11.125 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.125 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.125 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.125 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-08T06:04:11.125 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.125 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.125 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.126 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-05-08T06:04:11.126 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.126 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.126 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.126 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-05-08T06:04:11.126 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-05-08T06:04:11.126 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.126 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.126 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.126 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.126 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.126 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.126 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.127 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T06:04:11.127 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.127 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.127 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.127 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-05-08T06:04:11.127 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-05-08T06:04:11.127 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.127 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.127 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.127 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.127 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.127 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.128 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.128 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T06:04:11.128 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.128 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.128 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.128 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.128 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.128 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.128 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.128 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.128 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.128 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.128 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.129 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.129 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.129 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.129 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.129 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.129 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.129 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.129 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.129 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.129 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.129 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.129 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.129 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-05-08T06:04:11.130 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-05-08T06:04:11.130 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.130 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.130 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.130 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.130 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.130 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.130 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.130 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T06:04:11.130 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.130 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.130 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.130 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.131 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.131 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.131 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.131 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.131 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.131 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.131 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.131 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.131 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T06:04:11.131 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.131 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.131 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.131 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.131 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.132 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.132 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.132 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.132 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.132 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.132 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.132 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.132 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.132 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.132 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.132 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.132 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.132 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.132 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.132 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.133 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.133 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.133 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.133 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.133 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.133 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.133 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.133 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.133 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.133 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.133 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.133 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.133 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.134 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.134 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.134 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.134 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T06:04:11.134 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T06:04:11.134 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.134 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.134 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.134 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.134 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.134 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.134 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.134 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T06:04:11.134 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.135 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.135 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.135 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T06:04:11.135 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T06:04:11.135 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.135 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.135 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.135 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-05-08T06:04:11.135 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.135 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.135 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.135 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T06:04:11.135 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.136 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.136 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.136 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-05-08T06:04:11.136 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-05-08T06:04:11.136 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.136 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.136 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.136 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.136 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.136 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.136 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.136 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-05-08T06:04:11.136 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.136 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.137 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.137 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-05-08T06:04:11.137 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-05-08T06:04:11.137 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.137 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.137 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.137 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.137 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.137 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.137 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.137 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T06:04:11.137 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.137 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.137 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.138 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-05-08T06:04:11.138 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-05-08T06:04:11.138 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.138 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.138 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.138 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.138 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.138 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.138 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.138 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-05-08T06:04:11.138 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.138 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.138 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.139 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.140 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.140 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-05-08T06:04:11.140 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-05-08T06:04:11.140 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.140 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.140 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.140 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-05-08T06:04:11.140 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.140 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.140 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.140 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T06:04:11.140 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.140 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.140 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.141 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-05-08T06:04:11.141 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-05-08T06:04:11.141 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.141 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.141 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.141 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T06:04:11.141 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.141 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.141 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.141 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T06:04:11.141 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.141 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.141 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.142 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-05-08T06:04:11.142 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-05-08T06:04:11.142 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.142 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.142 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.142 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.142 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.142 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.142 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.142 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T06:04:11.142 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.142 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.142 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.143 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-05-08T06:04:11.143 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-05-08T06:04:11.143 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.143 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.143 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.143 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.143 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.143 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.143 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.143 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T06:04:11.143 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.143 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.143 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.143 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-05-08T06:04:11.144 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-05-08T06:04:11.144 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.144 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.144 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.144 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.144 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.144 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.144 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.144 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T06:04:11.144 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.144 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.144 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.144 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-05-08T06:04:11.145 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-05-08T06:04:11.145 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.145 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.145 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.145 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.145 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.145 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.145 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.145 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T06:04:11.145 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.145 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.145 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.145 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-05-08T06:04:11.146 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-05-08T06:04:11.146 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.146 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.146 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.146 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.146 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.146 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.146 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.146 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T06:04:11.146 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.146 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.146 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.146 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-05-08T06:04:11.147 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-05-08T06:04:11.147 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.147 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.147 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.147 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-05-08T06:04:11.147 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.147 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.147 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.147 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-05-08T06:04:11.147 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.147 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.147 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.147 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.148 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.148 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.148 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.148 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.148 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.148 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.148 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.148 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.148 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.148 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.148 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.148 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.148 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-05-08T06:04:11.149 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-05-08T06:04:11.149 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.149 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.149 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.149 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.149 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.149 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.149 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.149 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T06:04:11.149 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.149 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.149 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.149 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-05-08T06:04:11.150 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-05-08T06:04:11.150 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.150 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.150 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.150 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.150 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.150 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.150 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.150 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T06:04:11.150 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.150 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.150 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.151 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-05-08T06:04:11.151 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-05-08T06:04:11.151 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.151 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.151 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.151 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.151 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.151 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.151 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.151 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T06:04:11.151 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.151 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.151 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.152 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-05-08T06:04:11.152 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-05-08T06:04:11.152 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.152 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.152 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.152 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.152 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.152 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.152 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.152 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T06:04:11.152 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.152 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.152 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.153 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-05-08T06:04:11.153 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-05-08T06:04:11.153 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.153 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.153 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.153 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.153 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.153 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.153 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.153 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T06:04:11.153 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.153 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.153 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.153 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-05-08T06:04:11.154 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-05-08T06:04:11.154 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.154 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.154 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.154 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.154 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.154 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.154 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.154 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T06:04:11.154 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.154 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.154 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.154 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.154 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.155 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-05-08T06:04:11.155 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.155 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.155 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.155 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.155 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.155 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.155 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.155 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.155 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.155 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.155 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.156 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.157 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.157 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.157 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.157 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.157 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.157 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.157 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.157 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.157 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.157 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.157 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.157 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.157 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.157 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.158 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.158 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.158 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.158 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.158 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.158 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.158 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.158 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.158 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.158 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.158 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.158 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.158 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.158 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.159 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.159 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.159 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.159 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.159 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.159 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.159 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.159 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.159 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.159 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.159 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.159 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.159 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.159 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.160 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.160 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-05-08T06:04:11.160 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-05-08T06:04:11.160 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.160 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.160 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.160 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-05-08T06:04:11.160 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.160 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.160 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.160 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T06:04:11.160 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.160 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.161 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.161 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.161 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.161 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.161 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.161 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.161 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.161 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.161 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.161 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.161 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.161 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.161 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.161 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.162 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-05-08T06:04:11.162 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-05-08T06:04:11.162 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.162 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.162 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.162 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-05-08T06:04:11.162 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.162 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.162 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.162 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T06:04:11.162 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.162 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.162 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.163 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.163 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.163 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.163 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-05-08T06:04:11.163 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.163 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.163 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.163 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T06:04:11.163 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T06:04:11.163 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.163 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.163 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.163 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.163 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.164 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.165 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.165 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.165 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.165 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.165 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T06:04:11.165 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T06:04:11.165 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.165 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.165 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.165 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.165 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.165 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.165 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.165 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.166 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.166 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.166 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.166 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-05-08T06:04:11.166 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T06:04:11.166 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.166 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.166 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.166 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.166 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.166 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.166 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.167 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.167 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.167 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.167 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.167 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T06:04:11.167 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T06:04:11.167 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.167 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.167 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.167 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.167 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.167 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.167 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.167 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.168 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.168 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.168 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.168 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T06:04:11.168 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T06:04:11.168 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.168 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.168 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.168 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.168 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.168 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.168 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.168 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.168 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.169 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.169 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.169 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T06:04:11.169 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T06:04:11.169 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.169 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.169 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.169 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.169 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.169 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.169 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.169 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.169 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.169 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.170 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.170 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T06:04:11.170 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T06:04:11.170 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.170 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.170 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.170 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.170 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.170 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.170 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.170 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.170 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.171 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.171 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.171 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T06:04:11.171 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T06:04:11.171 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.171 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.171 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.171 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.171 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.171 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.171 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.171 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.171 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T06:04:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T06:04:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.173 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.173 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.173 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T06:04:11.173 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T06:04:11.173 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.173 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.173 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.173 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.173 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.173 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.173 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.173 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.173 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.174 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.174 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.174 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T06:04:11.174 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T06:04:11.174 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.174 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.174 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.174 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.174 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.174 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.174 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.174 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.174 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.175 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.175 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.175 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T06:04:11.175 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T06:04:11.175 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.175 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.175 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.175 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.175 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.175 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.175 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.175 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.176 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.176 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.176 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.176 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T06:04:11.176 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T06:04:11.176 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.176 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.176 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.176 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.176 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.176 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.176 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.177 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.177 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.177 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.177 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.177 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T06:04:11.177 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T06:04:11.177 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.177 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.177 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.177 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-08T06:04:11.177 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.177 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.178 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.178 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.178 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.178 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.178 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.178 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T06:04:11.178 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T06:04:11.178 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.178 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.178 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.178 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.178 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.179 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.179 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.179 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.179 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.179 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.179 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.179 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-05-08T06:04:11.179 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-05-08T06:04:11.179 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.179 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.179 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.179 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.179 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.180 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.180 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.180 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.180 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.180 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.180 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.180 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T06:04:11.180 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T06:04:11.180 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.180 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.180 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.180 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-08T06:04:11.181 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.181 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.181 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.181 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.181 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.181 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.181 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.181 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T06:04:11.181 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T06:04:11.181 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.181 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.181 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.182 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.182 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.182 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.182 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.182 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.182 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.182 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.182 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.182 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T06:04:11.182 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T06:04:11.182 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.182 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.183 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.183 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.183 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.183 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.183 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.183 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.183 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.183 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.183 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.183 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T06:04:11.183 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T06:04:11.183 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.183 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.184 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.184 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.184 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.184 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.184 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.184 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.184 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.184 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.184 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.184 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T06:04:11.184 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T06:04:11.184 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.184 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.185 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.185 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.185 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.185 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.185 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.185 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.185 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.185 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.185 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.185 INFO:teuthology.orchestra.run.smithi112.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-08T06:04:11.185 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-08T06:04:11.185 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.185 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.185 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.186 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "[]", 2024-05-08T06:04:11.186 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.186 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.186 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.186 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.186 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.186 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.186 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.186 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-05-08T06:04:11.186 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-08T06:04:11.186 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.186 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.186 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.187 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.188 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.189 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.189 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.189 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.189 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.189 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T06:04:11.189 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T06:04:11.189 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.189 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.189 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.189 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.189 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.189 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.189 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.190 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.190 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.190 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.190 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.190 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T06:04:11.190 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T06:04:11.190 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.190 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.190 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.190 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.190 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.190 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.190 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.190 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T06:04:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T06:04:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T06:04:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T06:04:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T06:04:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T06:04:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.196 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T06:04:11.196 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T06:04:11.196 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.196 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.196 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.196 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.196 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.196 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.196 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.196 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.196 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.196 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.196 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T06:04:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T06:04:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T06:04:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.197 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T06:04:11.198 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T06:04:11.198 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.198 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.198 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.198 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.198 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.198 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.198 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.198 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.198 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.198 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.198 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T06:04:11.199 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T06:04:11.200 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.200 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.200 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.200 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-05-08T06:04:11.200 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.200 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.200 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.200 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.200 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.200 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.200 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.200 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T06:04:11.200 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T06:04:11.200 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.201 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.201 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.201 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.201 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.201 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.201 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.201 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.201 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.201 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.201 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.201 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T06:04:11.201 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T06:04:11.201 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.202 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.202 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.202 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-08T06:04:11.202 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.202 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.202 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.202 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.202 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.202 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.202 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.202 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T06:04:11.202 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T06:04:11.202 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.203 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.203 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.203 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.203 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.203 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.203 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.203 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.203 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.203 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.203 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.203 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T06:04:11.203 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T06:04:11.204 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.204 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.204 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.204 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.204 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.204 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.204 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.204 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.204 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.204 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.204 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.204 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T06:04:11.204 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T06:04:11.204 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.205 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.205 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.205 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.205 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.205 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.205 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.205 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.205 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.205 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.205 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.205 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T06:04:11.205 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T06:04:11.205 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.205 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.206 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.206 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.206 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.206 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.206 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.206 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.206 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.206 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.206 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.206 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T06:04:11.206 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T06:04:11.206 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.207 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.207 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.207 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.207 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.207 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.207 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.207 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.207 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.207 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.207 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.207 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T06:04:11.207 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T06:04:11.207 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.207 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.208 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.208 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-08T06:04:11.208 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.208 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.208 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.208 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.208 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.208 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.208 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.208 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-05-08T06:04:11.208 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-05-08T06:04:11.208 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.209 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.209 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.209 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.209 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.209 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.209 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.209 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.209 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.209 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.209 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.209 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-05-08T06:04:11.209 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-05-08T06:04:11.209 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.209 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.210 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.210 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.210 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.210 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.210 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.210 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.210 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.210 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.210 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.210 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-05-08T06:04:11.210 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-05-08T06:04:11.210 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-05-08T06:04:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-05-08T06:04:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-05-08T06:04:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.212 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.212 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-05-08T06:04:11.212 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.212 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.212 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.212 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.212 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.212 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.212 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.212 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-05-08T06:04:11.212 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-05-08T06:04:11.212 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.212 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.212 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.213 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.213 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.213 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.213 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.213 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.213 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.213 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.213 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.213 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.213 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.213 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.213 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.214 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.214 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.214 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.214 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.214 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.214 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.214 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.214 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.214 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.214 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.214 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.214 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.214 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.214 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.217 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.217 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.217 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.217 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.217 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.217 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.217 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.217 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.217 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.217 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.217 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.217 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.217 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.217 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.218 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.218 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.218 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.218 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.218 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.218 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.218 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.218 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-05-08T06:04:11.218 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-05-08T06:04:11.218 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.218 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.218 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.218 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.219 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.219 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.219 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.219 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-05-08T06:04:11.219 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.219 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.219 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.219 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-05-08T06:04:11.219 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T06:04:11.219 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.219 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.219 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.219 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.219 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.220 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.220 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.220 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.220 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.220 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.220 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.220 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-08T06:04:11.220 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-08T06:04:11.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.220 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.220 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.220 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-05-08T06:04:11.220 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.220 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.221 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.222 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.222 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.222 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.222 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.222 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.222 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.222 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.222 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.222 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.222 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.222 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.222 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.222 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.223 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.223 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.223 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.223 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-05-08T06:04:11.223 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-05-08T06:04:11.223 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.223 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.223 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.223 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.223 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.223 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.223 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.223 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.224 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.224 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.224 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.224 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-05-08T06:04:11.224 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-05-08T06:04:11.224 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.224 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.224 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.224 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-05-08T06:04:11.224 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.224 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.224 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.225 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.225 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.225 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.225 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.225 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-05-08T06:04:11.225 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-05-08T06:04:11.225 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.225 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.225 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.225 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-05-08T06:04:11.225 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.225 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.225 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.226 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.226 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-05-08T06:04:11.226 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.226 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.226 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.226 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.226 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.226 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-05-08T06:04:11.226 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-05-08T06:04:11.226 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.226 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.226 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-05-08T06:04:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-05-08T06:04:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-05-08T06:04:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-05-08T06:04:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-05-08T06:04:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.228 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.228 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.228 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.228 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.228 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.228 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.228 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.228 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.228 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.228 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.228 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.229 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-05-08T06:04:11.229 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.229 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.229 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.229 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-05-08T06:04:11.229 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-05-08T06:04:11.229 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.229 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.229 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.229 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.229 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.229 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.229 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.229 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-05-08T06:04:11.230 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.230 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.230 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.230 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.230 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.230 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.230 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.230 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.230 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.230 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.230 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.230 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.232 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.232 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.232 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.232 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.232 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.232 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.232 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.232 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.232 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.232 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.232 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.232 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.232 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.233 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.233 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.233 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.233 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.233 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.233 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.233 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.233 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.233 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.233 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.233 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.233 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.233 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.233 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.234 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.234 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.234 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.234 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.234 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.234 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.234 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.234 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.234 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.234 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.234 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.234 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.234 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.235 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.235 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.235 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.235 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.235 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-05-08T06:04:11.235 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-05-08T06:04:11.235 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.235 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.235 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.235 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-05-08T06:04:11.235 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.235 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.235 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.235 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T06:04:11.236 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.236 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.236 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.236 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-05-08T06:04:11.236 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-05-08T06:04:11.236 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.236 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.236 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.236 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-05-08T06:04:11.236 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.236 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.236 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.236 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T06:04:11.237 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.237 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.237 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.237 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-05-08T06:04:11.237 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-05-08T06:04:11.237 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.237 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.237 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.237 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-05-08T06:04:11.237 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.237 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.237 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.237 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-05-08T06:04:11.238 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.238 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.238 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.238 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-05-08T06:04:11.238 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-05-08T06:04:11.238 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.238 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.238 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.238 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-05-08T06:04:11.238 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.238 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.238 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.238 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T06:04:11.239 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.239 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.239 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.239 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-05-08T06:04:11.239 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-05-08T06:04:11.239 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.239 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.239 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.239 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.239 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.239 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.239 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.239 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T06:04:11.239 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.240 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.240 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.240 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-08T06:04:11.240 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-08T06:04:11.240 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.240 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.240 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.240 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-08T06:04:11.240 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.240 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.240 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.240 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T06:04:11.240 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.241 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.241 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.241 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.241 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.241 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.241 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.241 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.241 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.241 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.241 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.242 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.242 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.242 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.242 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.242 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.242 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-05-08T06:04:11.242 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-05-08T06:04:11.242 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.242 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.242 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.242 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-05-08T06:04:11.242 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.243 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.244 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.245 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.245 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.245 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.245 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.245 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.245 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.245 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.245 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.245 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.245 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.245 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.245 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.245 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.245 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.246 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.246 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.246 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.246 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.246 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.246 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.246 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.246 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.246 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.246 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.246 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.246 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.247 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.247 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.247 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.247 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.247 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.247 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.247 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.247 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.247 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.247 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.247 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.247 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.247 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.247 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.248 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.248 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.248 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.248 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.248 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.248 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.248 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.248 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.248 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-05-08T06:04:11.248 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-05-08T06:04:11.248 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.248 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.248 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.249 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-05-08T06:04:11.249 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.249 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.249 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.249 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.249 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.249 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.249 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.249 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-05-08T06:04:11.249 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-05-08T06:04:11.249 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.249 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.249 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.250 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-05-08T06:04:11.250 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.250 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.250 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.250 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.250 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.250 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.250 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.250 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-08T06:04:11.250 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-08T06:04:11.250 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.250 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.250 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-08T06:04:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.252 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.252 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.252 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.252 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.252 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.252 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.252 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.252 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.252 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.252 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.252 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-05-08T06:04:11.252 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-05-08T06:04:11.252 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-05-08T06:04:11.252 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.253 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-05-08T06:04:11.253 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-05-08T06:04:11.253 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.253 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.253 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.253 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-05-08T06:04:11.253 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.253 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.253 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.253 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T06:04:11.253 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.253 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.253 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.253 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-05-08T06:04:11.254 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-05-08T06:04:11.254 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.254 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.254 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.254 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-05-08T06:04:11.254 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.254 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.254 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.254 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T06:04:11.254 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.254 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.254 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.255 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-05-08T06:04:11.255 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-05-08T06:04:11.255 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.255 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.255 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.255 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.255 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.255 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.255 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.255 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-05-08T06:04:11.255 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.255 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.256 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.256 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-08T06:04:11.256 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-08T06:04:11.256 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.256 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.256 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.256 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-05-08T06:04:11.256 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-05-08T06:04:11.256 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.256 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.256 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T06:04:11.256 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.256 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.256 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.257 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.258 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.258 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.258 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.258 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.258 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.258 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.258 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.258 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.258 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.258 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.258 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.258 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.258 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.259 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.259 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.259 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.259 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.259 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.259 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.259 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.259 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.259 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.259 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.259 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.259 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.259 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.259 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.259 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.260 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.260 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.260 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.260 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.260 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.260 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.260 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.260 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.260 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.260 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.260 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.260 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.260 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.260 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.261 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.261 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.261 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.261 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.261 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.261 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.261 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.261 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.261 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.261 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.261 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-05-08T06:04:11.261 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-05-08T06:04:11.261 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.262 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.262 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.262 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.262 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.262 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.262 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.262 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-05-08T06:04:11.262 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.262 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.262 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.262 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-05-08T06:04:11.262 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-05-08T06:04:11.262 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.263 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.263 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.263 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-05-08T06:04:11.263 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.263 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.263 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.263 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-05-08T06:04:11.263 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.263 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.263 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.263 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.263 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.263 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-05-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-05-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.264 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.265 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.265 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-05-08T06:04:11.265 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.265 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.265 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.265 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:04:11.265 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.265 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.265 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.265 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-05-08T06:04:11.265 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-05-08T06:04:11.265 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.265 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.265 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-08T06:04:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-08T06:04:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-05-08T06:04:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T06:04:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-05-08T06:04:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-05-08T06:04:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.267 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.267 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.267 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.267 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.267 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-05-08T06:04:11.267 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.267 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.267 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.267 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-05-08T06:04:11.267 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-05-08T06:04:11.267 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.267 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.268 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.268 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-05-08T06:04:11.268 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.268 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.268 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.268 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:04:11.268 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.268 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.268 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.268 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.268 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.268 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-05-08T06:04:11.269 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.269 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.269 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.269 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.269 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.269 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.269 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.269 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.269 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.269 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.269 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.269 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.269 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.269 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.269 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.270 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.270 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.270 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.270 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.270 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.270 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.270 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.270 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.270 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.270 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.270 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.270 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.270 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.271 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.272 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.272 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.272 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.272 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.272 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.272 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.272 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.272 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.272 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.272 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.272 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.272 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.272 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.272 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.273 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.273 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.273 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.273 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.273 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.273 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.273 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.273 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.273 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.273 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.273 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.273 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.273 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.275 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.276 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.276 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.276 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.276 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.276 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.276 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.276 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.276 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.276 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.276 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.276 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.276 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.276 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.277 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.278 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.278 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.278 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.278 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.278 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.278 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.278 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.278 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.278 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.278 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.278 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.278 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.278 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.278 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.278 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.279 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.279 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.279 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.279 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.279 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.279 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.279 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.279 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.279 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.279 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.279 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.279 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.279 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.279 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.280 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.281 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-05-08T06:04:11.281 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.281 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.281 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.281 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-05-08T06:04:11.281 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-05-08T06:04:11.281 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.281 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.281 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.281 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-05-08T06:04:11.281 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.281 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.281 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.281 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T06:04:11.282 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.282 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.282 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.282 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-05-08T06:04:11.282 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-05-08T06:04:11.282 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.282 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.282 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.282 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.282 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-05-08T06:04:11.282 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.282 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.282 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.283 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.284 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.284 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.284 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.284 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.284 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.284 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.284 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.284 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.284 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.284 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.284 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.284 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.284 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.285 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.285 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.285 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.285 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.285 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.285 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.285 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.285 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.286 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.286 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.286 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.286 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.286 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.286 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.287 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.287 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.287 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.287 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.287 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.287 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.287 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.287 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.288 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.288 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.288 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.288 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.288 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.288 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.288 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.289 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.289 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.289 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.289 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.289 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.289 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.289 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.289 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.290 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.290 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.290 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.290 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.290 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.290 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.290 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.291 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.292 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.292 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.292 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.292 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.292 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.292 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-05-08T06:04:11.292 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-05-08T06:04:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-05-08T06:04:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-05-08T06:04:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-05-08T06:04:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T06:04:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.294 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.294 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.294 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.294 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.294 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.294 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.294 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.294 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.294 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.294 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.294 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.295 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.295 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.295 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.295 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.295 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.295 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.295 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.295 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.295 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.295 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.295 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.295 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.295 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.295 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.296 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.296 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.296 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.296 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.296 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.296 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.296 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.296 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.296 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.296 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.296 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.297 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.297 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.297 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.297 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.297 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.297 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.297 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.297 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.297 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.297 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.297 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.297 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.297 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.298 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.298 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.298 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.298 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.298 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.298 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.298 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.298 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.298 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.298 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.298 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.299 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.299 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.299 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.299 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.299 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.299 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.299 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.299 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-05-08T06:04:11.299 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-05-08T06:04:11.299 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.299 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.299 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.299 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.300 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.300 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.300 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.300 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-05-08T06:04:11.300 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T06:04:11.300 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.300 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.300 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-05-08T06:04:11.300 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-05-08T06:04:11.300 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.300 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.300 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.300 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-05-08T06:04:11.301 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.301 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.301 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.301 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-05-08T06:04:11.301 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.301 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.301 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.301 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-05-08T06:04:11.301 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-05-08T06:04:11.301 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.301 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.301 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.302 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-05-08T06:04:11.302 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.302 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.302 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.302 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T06:04:11.302 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.302 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.302 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.302 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-05-08T06:04:11.302 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-05-08T06:04:11.302 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.302 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.302 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.303 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.304 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.304 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.304 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.304 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.304 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.304 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.304 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-05-08T06:04:11.304 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-05-08T06:04:11.304 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.304 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.304 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.304 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-05-08T06:04:11.304 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.304 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.305 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.306 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.306 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.306 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.306 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.306 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.306 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.306 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.306 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.306 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.306 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.306 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.306 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.306 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.307 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.308 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.308 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.308 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.308 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.308 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.308 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.308 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.308 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.308 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.308 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.308 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.308 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.308 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.309 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.309 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.309 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.309 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.309 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.309 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.309 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.309 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.309 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.309 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.309 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.309 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.309 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.309 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.310 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.310 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.310 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.310 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.310 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.310 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.310 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.310 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.310 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.310 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.310 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.310 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.311 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.311 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.311 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.311 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.311 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.311 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.311 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.311 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.311 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.311 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.311 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.311 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.311 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.312 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.313 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.313 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.313 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.313 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.313 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.313 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.313 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.313 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.313 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.313 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.313 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.313 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.313 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.313 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.314 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.314 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.314 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.314 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.314 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.314 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.314 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.314 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.314 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.314 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.314 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.314 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.314 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.315 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.315 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.315 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.315 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.315 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.315 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.315 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.315 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.315 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.315 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.315 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.315 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.315 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.315 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.316 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-05-08T06:04:11.317 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.318 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.318 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.318 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.318 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.318 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.318 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.318 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.318 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.318 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.318 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.318 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.318 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.318 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.318 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.319 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.319 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.319 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.319 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.319 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.319 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.319 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.319 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.319 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.319 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.319 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.319 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.319 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.319 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.320 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.321 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.322 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.322 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.322 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.322 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.322 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.322 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.322 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.322 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.322 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.322 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.322 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.322 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.322 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-05-08T06:04:11.323 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-05-08T06:04:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-05-08T06:04:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T06:04:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.324 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.325 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.325 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.325 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.325 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.325 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.325 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.325 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.325 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.325 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.325 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.325 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.325 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.325 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.325 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.326 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.326 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.326 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.326 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.326 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.326 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.326 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.326 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.326 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.326 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.326 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.326 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.326 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.326 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.327 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.327 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.327 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.327 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.327 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.327 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.327 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.327 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.327 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.327 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.327 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.327 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.327 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.328 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.328 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.328 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.328 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.328 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.328 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.328 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.328 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.328 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.328 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.328 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.328 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.328 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.329 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.329 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.329 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.329 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.329 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.329 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.329 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.329 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.329 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.329 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.329 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.329 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.329 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-05-08T06:04:11.330 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-05-08T06:04:11.330 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.330 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.330 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.330 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.330 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.330 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.330 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.330 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-05-08T06:04:11.330 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-05-08T06:04:11.330 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-05-08T06:04:11.330 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.330 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-05-08T06:04:11.330 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.331 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.331 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.331 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.331 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.331 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.331 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.331 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.331 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.331 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.331 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.331 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.331 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.331 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-05-08T06:04:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-05-08T06:04:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-05-08T06:04:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-08T06:04:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-08T06:04:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-08T06:04:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-05-08T06:04:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-05-08T06:04:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-05-08T06:04:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-05-08T06:04:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-05-08T06:04:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T06:04:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-05-08T06:04:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-05-08T06:04:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-05-08T06:04:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T06:04:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-05-08T06:04:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-05-08T06:04:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-05-08T06:04:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-05-08T06:04:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-05-08T06:04:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-05-08T06:04:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-08T06:04:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-08T06:04:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-08T06:04:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-05-08T06:04:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-05-08T06:04:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T06:04:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T06:04:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-05-08T06:04:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-05-08T06:04:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T06:04:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T06:04:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-05-08T06:04:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-05-08T06:04:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-05-08T06:04:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-05-08T06:04:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-08T06:04:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-08T06:04:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-05-08T06:04:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T06:04:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-08T06:04:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-08T06:04:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-05-08T06:04:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T06:04:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-05-08T06:04:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-05-08T06:04:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-05-08T06:04:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-05-08T06:04:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-05-08T06:04:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-05-08T06:04:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-05-08T06:04:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-05-08T06:04:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-05-08T06:04:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-05-08T06:04:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-05-08T06:04:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-05-08T06:04:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-05-08T06:04:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-05-08T06:04:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-05-08T06:04:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-05-08T06:04:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-05-08T06:04:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T06:04:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-05-08T06:04:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-05-08T06:04:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-05-08T06:04:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-05-08T06:04:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-05-08T06:04:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-05-08T06:04:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-05-08T06:04:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-08T06:04:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-08T06:04:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-08T06:04:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-08T06:04:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-05-08T06:04:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-05-08T06:04:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-05-08T06:04:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T06:04:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-05-08T06:04:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-05-08T06:04:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-05-08T06:04:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-05-08T06:04:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-05-08T06:04:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-05-08T06:04:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-05-08T06:04:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-05-08T06:04:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-05-08T06:04:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-05-08T06:04:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-05-08T06:04:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-05-08T06:04:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-05-08T06:04:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-05-08T06:04:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smb", 2024-05-08T06:04:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: "update_orchestration": { 2024-05-08T06:04:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "update_orchestration", 2024-05-08T06:04:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T06:04:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-05-08T06:04:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-05-08T06:04:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-05-08T06:04:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-05-08T06:04:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-05-08T06:04:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-05-08T06:04:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-05-08T06:04:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T06:04:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-08T06:04:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-08T06:04:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-05-08T06:04:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-05-08T06:04:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-05-08T06:04:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-05-08T06:04:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T06:04:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-05-08T06:04:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-05-08T06:04:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T06:04:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-05-08T06:04:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-05-08T06:04:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T06:04:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-05-08T06:04:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-05-08T06:04:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T06:04:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-05-08T06:04:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-05-08T06:04:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T06:04:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-05-08T06:04:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-05-08T06:04:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-05-08T06:04:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-05-08T06:04:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-05-08T06:04:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-05-08T06:04:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T06:04:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-05-08T06:04:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-05-08T06:04:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-08T06:04:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-08T06:04:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-05-08T06:04:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-05-08T06:04:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-05-08T06:04:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-05-08T06:04:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-05-08T06:04:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-05-08T06:04:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-05-08T06:04:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-05-08T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-05-08T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-05-08T06:04:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-05-08T06:04:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-05-08T06:04:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-05-08T06:04:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-05-08T06:04:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-05-08T06:04:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-05-08T06:04:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-05-08T06:04:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-05-08T06:04:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T06:04:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-05-08T06:04:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-05-08T06:04:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-05-08T06:04:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-05-08T06:04:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-05-08T06:04:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-05-08T06:04:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-05-08T06:04:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-05-08T06:04:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-08T06:04:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-05-08T06:04:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-05-08T06:04:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-05-08T06:04:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-05-08T06:04:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-05-08T06:04:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-05-08T06:04:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-05-08T06:04:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-05-08T06:04:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T06:04:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-08T06:04:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: "modules": [ 2024-05-08T06:04:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-05-08T06:04:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: "dashboard", 2024-05-08T06:04:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: "iostat", 2024-05-08T06:04:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: "nfs", 2024-05-08T06:04:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: "restful" 2024-05-08T06:04:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-05-08T06:04:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-05-08T06:04:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-08T06:04:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-08T06:04:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-05-08T06:04:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-05-08T06:04:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-05-08T06:04:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-05-08T06:04:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-05-08T06:04:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-05-08T06:04:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-05-08T06:04:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-05-08T06:04:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-05-08T06:04:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-05-08T06:04:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-05-08T06:04:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-05-08T06:04:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-05-08T06:04:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-05-08T06:04:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T06:04:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-05-08T06:04:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-05-08T06:04:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-05-08T06:04:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-05-08T06:04:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-05-08T06:04:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-05-08T06:04:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T06:04:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-05-08T06:04:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-05-08T06:04:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-05-08T06:04:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T06:04:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-05-08T06:04:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-05-08T06:04:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-08T06:04:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-05-08T06:04:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-08T06:04:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-05-08T06:04:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T06:04:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-05-08T06:04:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-05-08T06:04:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-05-08T06:04:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-05-08T06:04:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-05-08T06:04:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-05-08T06:04:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T06:04:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-05-08T06:04:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T06:04:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-05-08T06:04:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-05-08T06:04:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-05-08T06:04:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-05-08T06:04:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-05-08T06:04:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-05-08T06:04:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-05-08T06:04:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-05-08T06:04:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-05-08T06:04:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-05-08T06:04:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-05-08T06:04:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T06:04:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-08T06:04:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-08T06:04:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-05-08T06:04:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-05-08T06:04:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-08T06:04:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-08T06:04:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T06:04:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T06:04:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-05-08T06:04:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-05-08T06:04:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-08T06:04:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T06:04:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-05-08T06:04:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-05-08T06:04:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-05-08T06:04:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-05-08T06:04:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T06:04:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-05-08T06:04:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-05-08T06:04:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-05-08T06:04:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T06:04:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-05-08T06:04:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-05-08T06:04:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-05-08T06:04:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T06:04:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-05-08T06:04:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-05-08T06:04:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T06:04:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T06:04:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-05-08T06:04:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-05-08T06:04:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-08T06:04:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T06:04:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-05-08T06:04:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T06:04:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-05-08T06:04:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T06:04:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-05-08T06:04:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-05-08T06:04:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-05-08T06:04:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-05-08T06:04:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-05-08T06:04:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-08T06:04:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-05-08T06:04:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-05-08T06:04:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T06:04:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-05-08T06:04:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-05-08T06:04:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T06:04:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-05-08T06:04:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-05-08T06:04:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T06:04:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-05-08T06:04:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-05-08T06:04:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T06:04:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-08T06:04:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-05-08T06:04:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-05-08T06:04:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T06:04:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-05-08T06:04:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-05-08T06:04:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-05-08T06:04:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T06:04:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-05-08T06:04:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-05-08T06:04:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-05-08T06:04:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-08T06:04:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-08T06:04:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-05-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-05-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-05-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-05-08T06:04:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-05-08T06:04:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T06:04:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-05-08T06:04:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-05-08T06:04:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-05-08T06:04:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T06:04:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-05-08T06:04:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-05-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-05-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-05-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.570 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-05-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-05-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-05-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.571 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.572 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-05-08T06:04:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-05-08T06:04:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T06:04:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-08T06:04:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.573 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-05-08T06:04:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-05-08T06:04:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-08T06:04:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-05-08T06:04:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.574 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-05-08T06:04:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-05-08T06:04:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T06:04:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-05-08T06:04:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-05-08T06:04:11.575 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-05-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-05-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-05-08T06:04:11.576 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-05-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-05-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-05-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-05-08T06:04:11.577 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.578 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.578 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.578 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T06:04:11.578 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.578 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.578 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.578 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-05-08T06:04:11.578 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.578 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.578 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.578 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-05-08T06:04:11.578 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-05-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-05-08T06:04:11.579 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-05-08T06:04:11.580 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.580 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.580 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.580 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-08T06:04:11.580 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.580 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.580 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.580 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T06:04:11.580 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.580 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.580 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.580 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-05-08T06:04:11.581 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T06:04:11.581 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.581 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.581 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.581 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-05-08T06:04:11.581 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.581 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.581 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.581 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T06:04:11.581 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.581 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.581 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.581 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-05-08T06:04:11.582 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-05-08T06:04:11.582 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.582 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.582 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.582 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-05-08T06:04:11.582 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.582 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.582 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.582 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T06:04:11.582 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.582 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.582 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.583 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-05-08T06:04:11.583 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-05-08T06:04:11.583 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.583 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.583 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.583 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-05-08T06:04:11.583 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.583 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.583 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.583 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-05-08T06:04:11.583 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.583 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.583 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.584 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-05-08T06:04:11.584 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-05-08T06:04:11.584 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.584 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.584 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.584 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.584 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.584 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.584 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.584 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T06:04:11.584 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.584 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.584 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.585 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-05-08T06:04:11.585 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-05-08T06:04:11.585 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.585 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.585 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.585 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-08T06:04:11.585 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.585 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.585 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.585 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-05-08T06:04:11.585 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.585 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.586 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.586 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-05-08T06:04:11.586 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-05-08T06:04:11.586 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.586 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.586 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.586 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-08T06:04:11.586 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.586 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.586 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.586 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-08T06:04:11.586 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.587 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.587 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.587 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-05-08T06:04:11.587 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-05-08T06:04:11.587 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.587 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.587 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.587 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-08T06:04:11.587 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.587 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.587 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.587 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-05-08T06:04:11.587 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-05-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-05-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.588 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.589 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.589 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-05-08T06:04:11.589 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-05-08T06:04:11.589 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.589 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.589 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.589 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.589 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.589 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.589 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.589 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T06:04:11.589 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.590 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.590 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.590 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.590 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.590 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.590 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.590 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.590 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.590 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.590 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.590 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.590 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.591 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.592 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.592 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.592 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.592 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.592 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T06:04:11.592 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.592 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.592 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.592 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.592 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.592 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.592 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.592 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.593 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.593 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.593 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.593 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.593 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T06:04:11.593 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.593 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.593 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.593 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.593 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.593 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.593 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.594 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.594 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.594 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.594 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.594 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.594 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.594 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.594 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.594 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.594 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.594 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.594 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.594 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.595 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.595 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.595 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.595 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.595 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.595 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.595 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.595 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.595 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.595 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.595 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.595 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.595 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.595 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.596 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.596 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.596 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T06:04:11.596 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T06:04:11.596 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.596 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.596 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.596 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.596 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.596 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.596 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.596 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T06:04:11.596 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.597 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.597 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.597 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T06:04:11.597 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T06:04:11.597 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.597 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.597 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.597 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-05-08T06:04:11.597 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.597 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.597 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.597 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T06:04:11.597 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.597 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.598 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.598 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-05-08T06:04:11.598 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-05-08T06:04:11.598 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.598 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.598 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.598 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.598 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.598 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.598 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.598 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-05-08T06:04:11.598 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.598 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.599 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.599 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-05-08T06:04:11.599 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-05-08T06:04:11.599 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.599 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.599 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.599 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.599 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.599 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T06:04:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-05-08T06:04:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-05-08T06:04:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-05-08T06:04:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-05-08T06:04:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T06:04:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-05-08T06:04:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-05-08T06:04:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-05-08T06:04:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T06:04:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-05-08T06:04:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-05-08T06:04:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T06:04:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.604 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.604 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T06:04:11.604 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.604 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.604 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.604 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-05-08T06:04:11.604 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-05-08T06:04:11.604 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.604 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.604 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.604 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.604 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.604 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T06:04:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-05-08T06:04:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-05-08T06:04:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.606 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T06:04:11.606 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.606 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.606 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.606 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-05-08T06:04:11.606 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-05-08T06:04:11.606 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.606 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.606 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.606 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.606 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.606 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.606 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.606 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.607 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.609 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.610 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.610 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.610 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.610 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.610 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.610 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.610 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.610 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.610 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.610 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.610 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.610 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-05-08T06:04:11.611 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-05-08T06:04:11.611 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.611 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.611 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.611 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.611 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.611 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.611 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.611 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T06:04:11.611 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.611 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.611 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.611 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-05-08T06:04:11.612 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.613 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.613 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.613 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.613 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.613 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.613 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.613 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T06:04:11.613 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.613 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.613 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.613 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-05-08T06:04:11.613 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-05-08T06:04:11.613 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.614 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.614 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.614 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.614 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.614 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.614 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.614 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T06:04:11.614 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.614 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.614 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.614 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-05-08T06:04:11.614 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-05-08T06:04:11.614 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T06:04:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-05-08T06:04:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-05-08T06:04:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T06:04:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-05-08T06:04:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-05-08T06:04:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-05-08T06:04:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-05-08T06:04:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T06:04:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-05-08T06:04:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-05-08T06:04:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T06:04:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T06:04:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T06:04:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T06:04:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T06:04:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T06:04:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-05-08T06:04:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T06:04:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T06:04:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T06:04:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T06:04:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T06:04:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T06:04:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T06:04:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T06:04:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T06:04:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T06:04:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T06:04:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T06:04:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T06:04:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T06:04:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T06:04:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T06:04:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T06:04:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T06:04:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T06:04:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T06:04:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T06:04:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T06:04:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T06:04:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T06:04:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T06:04:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T06:04:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T06:04:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-08T06:04:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-08T06:04:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "[]", 2024-05-08T06:04:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T06:04:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T06:04:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T06:04:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T06:04:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T06:04:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T06:04:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T06:04:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T06:04:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T06:04:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T06:04:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T06:04:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T06:04:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T06:04:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T06:04:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T06:04:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T06:04:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T06:04:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T06:04:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T06:04:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T06:04:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T06:04:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T06:04:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-05-08T06:04:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T06:04:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T06:04:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-05-08T06:04:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T06:04:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T06:04:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T06:04:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T06:04:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T06:04:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T06:04:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T06:04:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T06:04:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-08T06:04:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-05-08T06:04:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-05-08T06:04:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-05-08T06:04:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-05-08T06:04:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-05-08T06:04:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-05-08T06:04:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-05-08T06:04:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-05-08T06:04:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-05-08T06:04:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-05-08T06:04:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-05-08T06:04:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-05-08T06:04:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-05-08T06:04:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-05-08T06:04:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T06:04:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-08T06:04:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-08T06:04:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-05-08T06:04:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-08T06:04:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-08T06:04:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-05-08T06:04:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-05-08T06:04:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-05-08T06:04:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-05-08T06:04:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-05-08T06:04:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-05-08T06:04:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-05-08T06:04:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-05-08T06:04:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-05-08T06:04:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-05-08T06:04:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-05-08T06:04:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-05-08T06:04:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-05-08T06:04:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-05-08T06:04:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-05-08T06:04:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-05-08T06:04:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-05-08T06:04:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-05-08T06:04:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-05-08T06:04:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-05-08T06:04:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-05-08T06:04:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-05-08T06:04:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-05-08T06:04:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-05-08T06:04:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T06:04:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-05-08T06:04:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-05-08T06:04:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-05-08T06:04:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T06:04:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-05-08T06:04:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-05-08T06:04:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-05-08T06:04:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-05-08T06:04:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-05-08T06:04:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-05-08T06:04:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-05-08T06:04:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T06:04:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-05-08T06:04:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-05-08T06:04:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T06:04:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-08T06:04:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-08T06:04:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-08T06:04:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T06:04:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-05-08T06:04:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-05-08T06:04:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-05-08T06:04:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T06:04:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-05-08T06:04:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-08T06:04:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-08T06:04:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-05-08T06:04:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-05-08T06:04:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-05-08T06:04:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-05-08T06:04:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-05-08T06:04:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-05-08T06:04:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-05-08T06:04:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T06:04:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-05-08T06:04:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-05-08T06:04:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-05-08T06:04:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-08T06:04:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-08T06:04:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-05-08T06:04:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-05-08T06:04:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T06:04:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-05-08T06:04:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-05-08T06:04:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:04:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-05-08T06:04:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-05-08T06:04:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-05-08T06:04:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-05-08T06:04:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-05-08T06:04:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-05-08T06:04:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T06:04:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-05-08T06:04:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-05-08T06:04:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-05-08T06:04:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T06:04:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-05-08T06:04:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-05-08T06:04:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-05-08T06:04:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-05-08T06:04:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-05-08T06:04:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T06:04:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-05-08T06:04:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-05-08T06:04:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-05-08T06:04:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T06:04:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-05-08T06:04:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-05-08T06:04:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-05-08T06:04:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-05-08T06:04:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-05-08T06:04:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-05-08T06:04:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-05-08T06:04:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T06:04:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-05-08T06:04:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-05-08T06:04:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-05-08T06:04:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-05-08T06:04:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-05-08T06:04:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-05-08T06:04:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T06:04:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T06:04:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-05-08T06:04:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-05-08T06:04:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-05-08T06:04:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-05-08T06:04:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-05-08T06:04:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-05-08T06:04:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T06:04:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-05-08T06:04:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-05-08T06:04:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-05-08T06:04:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-05-08T06:04:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-05-08T06:04:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-08T06:04:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-08T06:04:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T06:04:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-05-08T06:04:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-05-08T06:04:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-05-08T06:04:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-05-08T06:04:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-05-08T06:04:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-05-08T06:04:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-05-08T06:04:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-05-08T06:04:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-05-08T06:04:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-05-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-05-08T06:04:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-05-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-05-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-05-08T06:04:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-05-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T06:04:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-05-08T06:04:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-05-08T06:04:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-05-08T06:04:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-05-08T06:04:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T06:04:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-05-08T06:04:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-05-08T06:04:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-05-08T06:04:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-05-08T06:04:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-05-08T06:04:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-05-08T06:04:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-05-08T06:04:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-05-08T06:04:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-05-08T06:04:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-05-08T06:04:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-05-08T06:04:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-05-08T06:04:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-05-08T06:04:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-05-08T06:04:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-05-08T06:04:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-05-08T06:04:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T06:04:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-05-08T06:04:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-05-08T06:04:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-05-08T06:04:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-05-08T06:04:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-05-08T06:04:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-05-08T06:04:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-05-08T06:04:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-05-08T06:04:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-05-08T06:04:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-05-08T06:04:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-05-08T06:04:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-05-08T06:04:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-05-08T06:04:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-05-08T06:04:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-05-08T06:04:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-05-08T06:04:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-05-08T06:04:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-05-08T06:04:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-05-08T06:04:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-05-08T06:04:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-05-08T06:04:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-05-08T06:04:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-05-08T06:04:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smb", 2024-05-08T06:04:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "update_orchestration": { 2024-05-08T06:04:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "update_orchestration", 2024-05-08T06:04:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-05-08T06:04:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-05-08T06:04:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T06:04:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-05-08T06:04:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-05-08T06:04:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-05-08T06:04:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.897 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.897 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.897 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.897 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.897 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.897 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.897 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.897 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.897 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.897 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.897 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.897 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.897 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.897 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.898 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.898 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.898 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.898 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.898 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.898 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.898 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.898 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.898 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.898 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.898 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.898 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.898 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.899 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.899 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.899 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.899 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.899 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.899 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.899 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.899 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.899 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.899 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.899 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.899 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.899 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.899 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.899 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.900 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.900 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.900 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.900 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.900 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.900 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.900 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.900 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.900 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.900 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.900 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.900 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.900 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.901 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.901 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.901 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.901 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.901 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.901 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.901 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.901 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.901 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.901 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.901 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.901 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.901 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.902 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.902 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.902 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-05-08T06:04:11.902 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.902 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.902 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.902 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.902 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.902 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.902 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.902 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.902 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.902 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.902 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.902 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.903 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.904 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.904 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.904 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.904 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.904 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.904 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.904 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.904 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.904 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.904 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.904 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.904 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.904 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.904 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.904 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.905 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.906 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.906 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.906 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.906 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.906 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.906 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.906 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.906 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.906 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.906 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.906 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.906 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.906 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.907 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.908 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.909 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.910 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.910 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.910 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.910 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.910 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.910 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.910 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.910 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.910 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.910 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.910 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.910 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.910 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.911 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.912 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.912 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.912 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.912 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.912 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.912 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.912 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.912 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.912 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.912 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.912 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.912 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.912 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.912 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.913 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.913 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.913 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.913 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.913 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.913 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.913 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.913 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.913 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.913 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.913 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.913 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.913 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-05-08T06:04:11.914 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.914 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.914 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.914 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-05-08T06:04:11.914 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-05-08T06:04:11.914 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.914 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.914 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.914 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T06:04:11.914 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.914 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.914 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.914 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.914 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.914 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.915 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.916 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.916 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.916 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.916 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.916 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.916 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.916 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.916 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.916 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.916 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.916 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.916 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.916 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.916 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.919 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.919 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.919 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.919 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.919 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.919 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.919 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.919 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.919 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.919 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.919 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.919 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.919 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.919 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.920 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-05-08T06:04:11.921 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-05-08T06:04:11.922 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.922 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.922 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.922 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.922 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.922 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.922 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.922 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T06:04:11.922 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.922 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.922 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.922 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-05-08T06:04:11.922 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-05-08T06:04:11.923 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.923 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.923 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.923 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.923 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.923 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.923 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.923 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T06:04:11.923 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.923 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.923 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.923 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-05-08T06:04:11.923 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-05-08T06:04:11.923 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.925 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.926 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.926 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T06:04:11.926 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.926 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.926 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.926 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-05-08T06:04:11.926 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-05-08T06:04:11.926 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.926 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.926 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.926 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.926 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.927 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.928 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.929 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-08T06:04:11.930 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-08T06:04:11.930 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.930 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.930 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.930 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-05-08T06:04:11.930 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-05-08T06:04:11.930 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.930 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.930 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.930 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.930 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.930 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.930 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-05-08T06:04:11.930 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-05-08T06:04:11.931 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.931 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.931 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.931 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-05-08T06:04:11.931 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.931 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.931 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.931 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.931 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.931 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.931 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.931 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-05-08T06:04:11.931 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-05-08T06:04:11.931 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.932 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.933 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.934 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.934 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.934 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.934 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.934 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.934 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.934 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.934 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.934 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.934 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.934 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.934 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.934 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.935 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.936 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.936 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.936 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.936 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.936 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.936 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.936 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.936 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.936 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.936 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.936 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.936 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.936 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.936 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.937 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.937 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.937 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.937 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.937 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.937 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.937 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.937 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.937 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.937 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.937 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.937 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.938 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.938 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.938 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.938 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.938 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.938 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-05-08T06:04:11.938 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-05-08T06:04:11.938 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.938 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.938 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.938 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.938 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.939 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.939 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.939 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.939 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.939 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.939 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.939 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-05-08T06:04:11.939 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-05-08T06:04:11.939 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.939 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.939 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.939 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.939 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.940 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.940 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.940 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.940 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.940 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.940 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.940 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.940 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.940 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.940 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.940 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.940 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.940 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.941 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.941 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.941 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.941 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.941 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.941 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.941 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-05-08T06:04:11.941 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-05-08T06:04:11.941 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.941 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.941 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.941 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T06:04:11.942 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.942 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.942 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.942 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.942 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.942 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.942 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.942 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.942 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.942 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.942 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-05-08T06:04:11.942 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.942 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.943 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.943 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.943 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.943 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.943 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.943 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.943 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.943 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.943 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.943 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.943 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.943 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.943 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.943 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.943 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.944 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.944 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.944 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.944 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.944 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.944 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.944 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.944 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.944 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.944 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.944 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.944 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.944 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.945 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.945 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.945 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.945 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.945 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.945 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.945 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.945 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.945 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.945 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.945 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.945 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.945 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.946 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.946 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.946 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.946 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.946 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.946 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.946 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.946 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.946 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.946 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.946 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.946 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.946 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.947 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.947 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.947 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.947 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.947 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.947 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.947 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.947 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.947 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.947 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.947 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.947 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.947 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.947 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.948 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.948 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.948 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.948 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.948 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.948 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.948 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.948 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.948 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.948 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.948 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.948 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.948 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.949 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.949 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.949 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.949 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.949 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-05-08T06:04:11.949 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.949 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.949 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.949 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.949 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.949 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.949 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.949 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.950 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.950 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.950 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.950 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.950 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.950 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.950 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.950 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.950 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.950 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.950 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.950 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.950 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.950 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.952 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.953 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.953 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.953 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.953 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.953 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.953 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.953 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.953 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.953 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.953 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.953 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.953 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.953 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.954 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.954 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.954 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.954 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.954 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.954 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-05-08T06:04:11.954 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-05-08T06:04:11.954 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.954 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.954 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.954 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-05-08T06:04:11.954 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.954 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.955 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.955 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T06:04:11.955 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.955 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.955 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.955 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-05-08T06:04:11.955 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-05-08T06:04:11.955 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.955 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.955 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.955 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.955 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.956 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.956 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.956 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-05-08T06:04:11.956 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.956 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.956 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.956 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-05-08T06:04:11.956 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-05-08T06:04:11.956 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.956 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.956 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.956 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.956 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.957 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.957 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.957 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T06:04:11.957 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.957 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.957 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.957 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-05-08T06:04:11.957 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T06:04:11.957 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.957 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.957 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.957 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-05-08T06:04:11.957 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.957 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.958 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.958 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T06:04:11.958 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.958 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.958 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.958 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.958 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.958 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.958 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.958 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.958 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.958 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.958 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.959 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.959 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.959 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.959 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.959 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.959 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.959 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.959 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-05-08T06:04:11.959 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-05-08T06:04:11.959 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-05-08T06:04:11.959 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-05-08T06:04:11.959 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-05-08T06:04:11.959 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-05-08T06:04:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-05-08T06:04:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-05-08T06:04:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-05-08T06:04:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-05-08T06:04:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.961 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.961 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.961 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.961 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.961 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.961 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.961 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.961 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.961 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.961 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.961 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-05-08T06:04:11.961 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-05-08T06:04:11.961 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-05-08T06:04:11.961 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.962 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.963 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.964 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.965 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.966 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-05-08T06:04:11.966 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-05-08T06:04:11.966 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-05-08T06:04:11.966 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.966 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.966 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-05-08T06:04:11.966 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.966 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.966 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.966 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.966 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.966 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.966 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.967 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.967 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-05-08T06:04:11.967 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:04:11.967 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.967 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-05-08T06:04:11.967 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-05-08T06:04:11.967 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-05-08T06:04:11.967 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.967 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.967 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.967 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.967 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.967 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.967 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.968 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-05-08T06:04:11.968 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-05-08T06:04:11.968 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.968 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.968 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.968 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-05-08T06:04:11.968 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.968 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.968 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.968 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.968 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.968 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.968 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-05-08T06:04:11.969 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-05-08T06:04:11.970 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-05-08T06:04:11.970 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-05-08T06:04:11.970 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-05-08T06:04:11.970 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T06:04:11.970 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-05-08T06:04:11.970 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-05-08T06:04:11.970 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-05-08T06:04:11.970 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-05-08T06:04:11.970 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-05-08T06:04:11.970 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-05-08T06:04:11.970 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-05-08T06:04:11.970 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.971 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.971 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.971 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.971 INFO:teuthology.orchestra.run.smithi112.stdout: "services": { 2024-05-08T06:04:11.971 INFO:teuthology.orchestra.run.smithi112.stdout: "dashboard": "https://172.21.15.112:8443/" 2024-05-08T06:04:11.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.971 INFO:teuthology.orchestra.run.smithi112.stdout: "always_on_modules": { 2024-05-08T06:04:11.971 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus": [ 2024-05-08T06:04:11.971 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-05-08T06:04:11.971 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-05-08T06:04:11.971 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-05-08T06:04:11.971 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-05-08T06:04:11.971 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-05-08T06:04:11.971 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-05-08T06:04:11.971 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-05-08T06:04:11.972 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-05-08T06:04:11.972 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-05-08T06:04:11.972 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-05-08T06:04:11.972 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.972 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific": [ 2024-05-08T06:04:11.972 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-05-08T06:04:11.972 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-05-08T06:04:11.972 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-05-08T06:04:11.972 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-05-08T06:04:11.972 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-05-08T06:04:11.972 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-05-08T06:04:11.972 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-05-08T06:04:11.972 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-05-08T06:04:11.973 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-05-08T06:04:11.973 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-05-08T06:04:11.973 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.973 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy": [ 2024-05-08T06:04:11.973 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-05-08T06:04:11.973 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-05-08T06:04:11.973 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-05-08T06:04:11.973 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-05-08T06:04:11.973 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-05-08T06:04:11.973 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-05-08T06:04:11.973 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-05-08T06:04:11.973 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-05-08T06:04:11.973 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-05-08T06:04:11.974 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-05-08T06:04:11.974 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.974 INFO:teuthology.orchestra.run.smithi112.stdout: "reef": [ 2024-05-08T06:04:11.974 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-05-08T06:04:11.974 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-05-08T06:04:11.974 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-05-08T06:04:11.974 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-05-08T06:04:11.974 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-05-08T06:04:11.974 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-05-08T06:04:11.974 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-05-08T06:04:11.974 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-05-08T06:04:11.974 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-05-08T06:04:11.974 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-05-08T06:04:11.975 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-05-08T06:04:11.975 INFO:teuthology.orchestra.run.smithi112.stdout: "squid": [ 2024-05-08T06:04:11.975 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-05-08T06:04:11.975 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-05-08T06:04:11.975 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-05-08T06:04:11.975 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-05-08T06:04:11.975 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-05-08T06:04:11.975 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-05-08T06:04:11.975 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-05-08T06:04:11.975 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-05-08T06:04:11.975 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-05-08T06:04:11.975 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-05-08T06:04:11.975 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-08T06:04:11.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.975 INFO:teuthology.orchestra.run.smithi112.stdout: "last_failure_osd_epoch": 3, 2024-05-08T06:04:11.976 INFO:teuthology.orchestra.run.smithi112.stdout: "active_clients": [ 2024-05-08T06:04:11.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.976 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-05-08T06:04:11.976 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-05-08T06:04:11.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.976 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-05-08T06:04:11.976 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-05-08T06:04:11.976 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 74091236 2024-05-08T06:04:11.976 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.976 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-08T06:04:11.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.976 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "libcephsqlite", 2024-05-08T06:04:11.976 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-05-08T06:04:11.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.977 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-05-08T06:04:11.977 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-05-08T06:04:11.977 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 861445710 2024-05-08T06:04:11.977 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.977 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-08T06:04:11.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.977 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-05-08T06:04:11.977 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-05-08T06:04:11.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.977 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-05-08T06:04:11.977 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-05-08T06:04:11.977 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 3302469559 2024-05-08T06:04:11.977 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.978 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-08T06:04:11.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:11.978 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.978 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-05-08T06:04:11.978 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-05-08T06:04:11.978 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-05-08T06:04:11.978 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-05-08T06:04:11.978 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-05-08T06:04:11.978 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 4195767286 2024-05-08T06:04:11.978 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.978 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-08T06:04:11.978 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:11.978 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-05-08T06:04:11.978 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:04:11.980 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:10 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/2266056960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T06:04:11.980 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:10 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3921926251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T06:04:11.980 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:11 smithi162 ceph-mon[34800]: pgmap v707: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T06:04:11.980 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:11 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1456795926' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T06:04:11.980 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:10 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/2266056960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T06:04:11.980 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:10 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3921926251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T06:04:11.980 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:11 smithi150 ceph-mon[34315]: pgmap v707: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T06:04:11.980 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:11 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1456795926' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T06:04:11.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.595+0000 7f7277e00640 1 -- 172.21.15.112:0/1456795926 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f725c0b2870 msgr2=0x7f725c0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:11.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.595+0000 7f7277e00640 1 --2- 172.21.15.112:0/1456795926 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f725c0b2870 0x7f725c0b4d30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7264031500 tx=0x7f7264005a20 comp rx=0 tx=0).stop 2024-05-08T06:04:11.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.595+0000 7f7277e00640 1 -- 172.21.15.112:0/1456795926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7278102820 msgr2=0x7f7278198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:11.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.595+0000 7f7277e00640 1 --2- 172.21.15.112:0/1456795926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7278102820 0x7f7278198ad0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f726800e9c0 tx=0x7f726800edc0 comp rx=0 tx=0).stop 2024-05-08T06:04:11.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.595+0000 7f7277e00640 1 -- 172.21.15.112:0/1456795926 shutdown_connections 2024-05-08T06:04:11.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.595+0000 7f7277e00640 1 --2- 172.21.15.112:0/1456795926 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f725c0b2870 0x7f725c0b4d30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:11.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.595+0000 7f7277e00640 1 --2- 172.21.15.112:0/1456795926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72781043d0 0x7f72781a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:11.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.595+0000 7f7277e00640 1 --2- 172.21.15.112:0/1456795926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7278103a10 0x7f7278199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:11.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.595+0000 7f7277e00640 1 --2- 172.21.15.112:0/1456795926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7278102820 0x7f7278198ad0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:11.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.595+0000 7f7277e00640 1 -- 172.21.15.112:0/1456795926 >> 172.21.15.112:0/1456795926 conn(0x7f72780fdd90 msgr2=0x7f7278106de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:11.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.595+0000 7f7277e00640 1 -- 172.21.15.112:0/1456795926 shutdown_connections 2024-05-08T06:04:11.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:10.595+0000 7f7277e00640 1 -- 172.21.15.112:0/1456795926 wait complete. 2024-05-08T06:04:11.987 DEBUG:tasks.check_counter:Getting stats from a 2024-05-08T06:04:11.987 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-08T06:04:11.988 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:11 smithi112 ceph-mon[30003]: pgmap v707: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T06:04:11.988 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:11 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1456795926' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T06:04:12.439 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:13.647 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:13 smithi112 ceph-mon[30003]: pgmap v708: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T06:04:13.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:13 smithi150 ceph-mon[34315]: pgmap v708: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T06:04:13.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:13 smithi162 ceph-mon[34800]: pgmap v708: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T06:04:14.763 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:04:14.763 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:14.763 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 27709, 2024-05-08T06:04:14.763 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 27487, 2024-05-08T06:04:14.764 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 7049219, 2024-05-08T06:04:14.764 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 51437200, 2024-05-08T06:04:14.764 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 85, 2024-05-08T06:04:14.764 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 8, 2024-05-08T06:04:14.764 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 5.288985009, 2024-05-08T06:04:14.764 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 1.854722290, 2024-05-08T06:04:14.764 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 1.388686788, 2024-05-08T06:04:14.764 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 1.646455648, 2024-05-08T06:04:14.764 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:14.764 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 27485, 2024-05-08T06:04:14.764 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 4.770566773, 2024-05-08T06:04:14.764 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000173569 2024-05-08T06:04:14.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.764 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:14.765 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3212, 2024-05-08T06:04:14.765 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.001225016, 2024-05-08T06:04:14.765 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000381 2024-05-08T06:04:14.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.765 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:04:14.765 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:04:14.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.765 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:14.765 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 49489, 2024-05-08T06:04:14.765 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 95988, 2024-05-08T06:04:14.765 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 15783488, 2024-05-08T06:04:14.765 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 51921233, 2024-05-08T06:04:14.765 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 13, 2024-05-08T06:04:14.766 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 5, 2024-05-08T06:04:14.766 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 17.521335131, 2024-05-08T06:04:14.766 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 10.412491833, 2024-05-08T06:04:14.766 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 6.203898159, 2024-05-08T06:04:14.766 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 1.364402645, 2024-05-08T06:04:14.766 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:14.766 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 95987, 2024-05-08T06:04:14.766 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 12.743096690, 2024-05-08T06:04:14.767 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000132758 2024-05-08T06:04:14.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.767 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:14.767 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 82861, 2024-05-08T06:04:14.767 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.051959950, 2024-05-08T06:04:14.767 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000627 2024-05-08T06:04:14.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.767 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 668592, 2024-05-08T06:04:14.767 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 183136 2024-05-08T06:04:14.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.768 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:14.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 25238, 2024-05-08T06:04:14.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 24034, 2024-05-08T06:04:14.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 7571090, 2024-05-08T06:04:14.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 29104120, 2024-05-08T06:04:14.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 8, 2024-05-08T06:04:14.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 6, 2024-05-08T06:04:14.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 5.096684963, 2024-05-08T06:04:14.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 1.894316086, 2024-05-08T06:04:14.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 1.464564347, 2024-05-08T06:04:14.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 1.557498322, 2024-05-08T06:04:14.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:14.768 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 24033, 2024-05-08T06:04:14.769 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 4.365921194, 2024-05-08T06:04:14.769 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000181663 2024-05-08T06:04:14.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.769 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:14.769 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 7864, 2024-05-08T06:04:14.769 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.002790874, 2024-05-08T06:04:14.769 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000354 2024-05-08T06:04:14.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.769 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-08T06:04:14.769 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 376224 2024-05-08T06:04:14.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.769 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-05-08T06:04:14.769 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-05-08T06:04:14.769 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-05-08T06:04:14.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.770 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-05-08T06:04:14.770 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-05-08T06:04:14.770 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-05-08T06:04:14.770 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 85380, 2024-05-08T06:04:14.770 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 50.548378034, 2024-05-08T06:04:14.770 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000592040 2024-05-08T06:04:14.770 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:14.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.770 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-05-08T06:04:14.770 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-05-08T06:04:14.770 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-05-08T06:04:14.770 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 965, 2024-05-08T06:04:14.771 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.206203040, 2024-05-08T06:04:14.771 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000213681 2024-05-08T06:04:14.771 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:14.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.771 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-05-08T06:04:14.771 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 15766, 2024-05-08T06:04:14.771 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 15201, 2024-05-08T06:04:14.771 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-05-08T06:04:14.771 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 15201, 2024-05-08T06:04:14.771 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 84.231338133, 2024-05-08T06:04:14.771 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.005541170 2024-05-08T06:04:14.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.771 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-05-08T06:04:14.771 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 565, 2024-05-08T06:04:14.772 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-05-08T06:04:14.772 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-05-08T06:04:14.772 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 130, 2024-05-08T06:04:14.772 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 14, 2024-05-08T06:04:14.772 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-05-08T06:04:14.772 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 32, 2024-05-08T06:04:14.772 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 4, 2024-05-08T06:04:14.772 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 7, 2024-05-08T06:04:14.772 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 21, 2024-05-08T06:04:14.772 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 26, 2024-05-08T06:04:14.772 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 5582, 2024-05-08T06:04:14.772 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 2, 2024-05-08T06:04:14.772 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 2, 2024-05-08T06:04:14.772 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 3, 2024-05-08T06:04:14.773 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 17204, 2024-05-08T06:04:14.773 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 15526, 2024-05-08T06:04:14.773 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 541, 2024-05-08T06:04:14.773 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 9, 2024-05-08T06:04:14.773 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-05-08T06:04:14.773 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-05-08T06:04:14.773 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 4, 2024-05-08T06:04:14.773 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 12, 2024-05-08T06:04:14.773 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-05-08T06:04:14.773 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 82, 2024-05-08T06:04:14.773 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 5462, 2024-05-08T06:04:14.773 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 25, 2024-05-08T06:04:14.773 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 955, 2024-05-08T06:04:14.773 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-05-08T06:04:14.773 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 3444, 2024-05-08T06:04:14.774 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-05-08T06:04:14.774 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 4, 2024-05-08T06:04:14.774 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 234, 2024-05-08T06:04:14.774 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 0, 2024-05-08T06:04:14.774 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 3444, 2024-05-08T06:04:14.774 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-05-08T06:04:14.774 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T06:04:14.774 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-05-08T06:04:14.774 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 160, 2024-05-08T06:04:14.774 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 10, 2024-05-08T06:04:14.774 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 150, 2024-05-08T06:04:14.774 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 3284, 2024-05-08T06:04:14.774 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 11, 2024-05-08T06:04:14.775 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 8285, 2024-05-08T06:04:14.775 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 13835, 2024-05-08T06:04:14.775 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 7861, 2024-05-08T06:04:14.775 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T06:04:14.775 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T06:04:14.775 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 12259, 2024-05-08T06:04:14.775 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 10878, 2024-05-08T06:04:14.775 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 426, 2024-05-08T06:04:14.775 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 9728 2024-05-08T06:04:14.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.775 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-05-08T06:04:14.775 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-05-08T06:04:14.775 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 0, 2024-05-08T06:04:14.775 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 0, 2024-05-08T06:04:14.776 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 14, 2024-05-08T06:04:14.776 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 83, 2024-05-08T06:04:14.776 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-05-08T06:04:14.776 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-05-08T06:04:14.776 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-05-08T06:04:14.776 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 557, 2024-05-08T06:04:14.776 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 554, 2024-05-08T06:04:14.776 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 3, 2024-05-08T06:04:14.776 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-05-08T06:04:14.776 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-05-08T06:04:14.776 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-05-08T06:04:14.776 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-05-08T06:04:14.776 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-05-08T06:04:14.777 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-05-08T06:04:14.777 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-05-08T06:04:14.777 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-05-08T06:04:14.777 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 10, 2024-05-08T06:04:14.777 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 82, 2024-05-08T06:04:14.777 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-05-08T06:04:14.777 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 14, 2024-05-08T06:04:14.777 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-05-08T06:04:14.777 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-05-08T06:04:14.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.778 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-05-08T06:04:14.778 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-05-08T06:04:14.778 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 34338, 2024-05-08T06:04:14.778 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-05-08T06:04:14.778 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-05-08T06:04:14.778 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 34338, 2024-05-08T06:04:14.778 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-05-08T06:04:14.778 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-05-08T06:04:14.778 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 39, 2024-05-08T06:04:14.778 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-05-08T06:04:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-05-08T06:04:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 39, 2024-05-08T06:04:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 4, 2024-05-08T06:04:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-05-08T06:04:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-05-08T06:04:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 4194304, 2024-05-08T06:04:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 78503147, 2024-05-08T06:04:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 0, 2024-05-08T06:04:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-05-08T06:04:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 24876, 2024-05-08T06:04:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 78.295806246, 2024-05-08T06:04:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003147443 2024-05-08T06:04:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 0 2024-05-08T06:04:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-05-08T06:04:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 30, 2024-05-08T06:04:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 5287, 2024-05-08T06:04:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 5257, 2024-05-08T06:04:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 25, 2024-05-08T06:04:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 287, 2024-05-08T06:04:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 262, 2024-05-08T06:04:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 32, 2024-05-08T06:04:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 6279, 2024-05-08T06:04:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 6247, 2024-05-08T06:04:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 2, 2024-05-08T06:04:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 5143, 2024-05-08T06:04:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 5141, 2024-05-08T06:04:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 86368, 2024-05-08T06:04:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 231800 2024-05-08T06:04:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-05-08T06:04:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 33486, 2024-05-08T06:04:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 563, 2024-05-08T06:04:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 15766, 2024-05-08T06:04:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 95, 2024-05-08T06:04:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 1111, 2024-05-08T06:04:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-05-08T06:04:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4621, 2024-05-08T06:04:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 16.017079120, 2024-05-08T06:04:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003466149 2024-05-08T06:04:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-05-08T06:04:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 236, 2024-05-08T06:04:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1.170551787, 2024-05-08T06:04:14.782 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.004959965 2024-05-08T06:04:14.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.782 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-05-08T06:04:14.782 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.782 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.782 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.782 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-05-08T06:04:14.782 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3, 2024-05-08T06:04:14.782 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.017263943, 2024-05-08T06:04:14.782 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.005754647 2024-05-08T06:04:14.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.782 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-05-08T06:04:14.782 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 449, 2024-05-08T06:04:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.222775747, 2024-05-08T06:04:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000496159 2024-05-08T06:04:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-05-08T06:04:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-05-08T06:04:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1, 2024-05-08T06:04:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000405608, 2024-05-08T06:04:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000405608 2024-05-08T06:04:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-05-08T06:04:14.783 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.784 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.784 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.784 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-05-08T06:04:14.784 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.784 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.784 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.784 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-05-08T06:04:14.784 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.784 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.784 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.785 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-05-08T06:04:14.785 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.785 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.785 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.785 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-05-08T06:04:14.785 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 70, 2024-05-08T06:04:14.785 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.108469829, 2024-05-08T06:04:14.785 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001549568 2024-05-08T06:04:14.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.786 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-05-08T06:04:14.786 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.786 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.786 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.786 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-05-08T06:04:14.786 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.786 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.786 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.786 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-05-08T06:04:14.786 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 308, 2024-05-08T06:04:14.787 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.090040046, 2024-05-08T06:04:14.787 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000292337 2024-05-08T06:04:14.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.787 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-05-08T06:04:14.787 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 77, 2024-05-08T06:04:14.787 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.796369388, 2024-05-08T06:04:14.787 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.010342459 2024-05-08T06:04:14.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-05-08T06:04:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 77, 2024-05-08T06:04:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.313462699, 2024-05-08T06:04:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.004070944 2024-05-08T06:04:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-05-08T06:04:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-05-08T06:04:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-05-08T06:04:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 10, 2024-05-08T06:04:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.153264866, 2024-05-08T06:04:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.015326486 2024-05-08T06:04:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-05-08T06:04:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.789 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-05-08T06:04:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-05-08T06:04:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 8841, 2024-05-08T06:04:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 63.325302197, 2024-05-08T06:04:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.007162685 2024-05-08T06:04:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-05-08T06:04:14.790 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-05-08T06:04:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 8, 2024-05-08T06:04:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.005170737, 2024-05-08T06:04:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000646342 2024-05-08T06:04:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-05-08T06:04:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.791 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-05-08T06:04:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 7, 2024-05-08T06:04:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.030190812, 2024-05-08T06:04:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.004312973 2024-05-08T06:04:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-05-08T06:04:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 5, 2024-05-08T06:04:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.005516681, 2024-05-08T06:04:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001103336 2024-05-08T06:04:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-05-08T06:04:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 488, 2024-05-08T06:04:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1.975474673, 2024-05-08T06:04:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.004048103 2024-05-08T06:04:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-05-08T06:04:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-05-08T06:04:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-05-08T06:04:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-05-08T06:04:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 1, 2024-05-08T06:04:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 2, 2024-05-08T06:04:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 1, 2024-05-08T06:04:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 1, 2024-05-08T06:04:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-05-08T06:04:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 1591, 2024-05-08T06:04:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 1591, 2024-05-08T06:04:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 556, 2024-05-08T06:04:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-05-08T06:04:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-05-08T06:04:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-05-08T06:04:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 303355, 2024-05-08T06:04:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 137, 2024-05-08T06:04:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 1320, 2024-05-08T06:04:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 15, 2024-05-08T06:04:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-05-08T06:04:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-05-08T06:04:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 7728, 2024-05-08T06:04:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 163, 2024-05-08T06:04:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-05-08T06:04:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 125634, 2024-05-08T06:04:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 761, 2024-05-08T06:04:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-05-08T06:04:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-05-08T06:04:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-05-08T06:04:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-05-08T06:04:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-05-08T06:04:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 66586, 2024-05-08T06:04:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 76343429, 2024-05-08T06:04:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 31, 2024-05-08T06:04:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 66586, 2024-05-08T06:04:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-05-08T06:04:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 66586, 2024-05-08T06:04:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 304.335125574, 2024-05-08T06:04:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.004570557 2024-05-08T06:04:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-05-08T06:04:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-05-08T06:04:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 66586, 2024-05-08T06:04:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 66845 2024-05-08T06:04:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 66586, 2024-05-08T06:04:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 3504, 2024-05-08T06:04:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 63082, 2024-05-08T06:04:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-05-08T06:04:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-05-08T06:04:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 0, 2024-05-08T06:04:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 11, 2024-05-08T06:04:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 50, 2024-05-08T06:04:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 24916, 2024-05-08T06:04:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 66, 2024-05-08T06:04:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-05-08T06:04:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-05-08T06:04:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 6, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 1118, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 3454, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 22, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-05-08T06:04:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-05-08T06:04:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 36715, 2024-05-08T06:04:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-05-08T06:04:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-05-08T06:04:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-05-08T06:04:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-05-08T06:04:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-05-08T06:04:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-05-08T06:04:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-05-08T06:04:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-05-08T06:04:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-05-08T06:04:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-05-08T06:04:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-05-08T06:04:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-05-08T06:04:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-05-08T06:04:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 84, 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 4, 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 12, 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 12, 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 438, 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 0, 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 49 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 1, 2024-05-08T06:04:14.803 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 34, 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 10, 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 581, 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 554, 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.804 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:14.805 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-08T06:04:14.805 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-05-08T06:04:14.805 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-08T06:04:14.805 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 102432, 2024-05-08T06:04:14.805 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 21160511, 2024-05-08T06:04:14.805 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:14.805 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 102432, 2024-05-08T06:04:14.805 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-05-08T06:04:14.805 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-05-08T06:04:14.805 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 102432, 2024-05-08T06:04:14.805 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 21160511, 2024-05-08T06:04:14.805 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-08T06:04:14.805 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.805 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.806 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.806 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:14.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.806 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-05-08T06:04:14.806 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-08T06:04:14.806 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-05-08T06:04:14.806 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-08T06:04:14.806 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-05-08T06:04:14.806 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-05-08T06:04:14.806 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:14.806 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:14.806 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 66586, 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 76399383, 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 66586, 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 76399383, 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-05-08T06:04:14.807 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:14.808 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:14.808 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 66586, 2024-05-08T06:04:14.808 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 66586, 2024-05-08T06:04:14.808 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 66586, 2024-05-08T06:04:14.808 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 66586, 2024-05-08T06:04:14.808 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-08T06:04:14.808 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.808 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.808 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.808 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:14.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.809 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-05-08T06:04:14.809 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-08T06:04:14.809 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-05-08T06:04:14.809 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-08T06:04:14.809 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 554, 2024-05-08T06:04:14.809 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 55954, 2024-05-08T06:04:14.809 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:14.809 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 554, 2024-05-08T06:04:14.809 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-05-08T06:04:14.809 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-05-08T06:04:14.809 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 40, 2024-05-08T06:04:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 55954, 2024-05-08T06:04:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-08T06:04:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x556e606ce500": { 2024-05-08T06:04:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-08T06:04:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-05-08T06:04:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-08T06:04:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 34338, 2024-05-08T06:04:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 74308843, 2024-05-08T06:04:14.810 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 34338, 2024-05-08T06:04:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-05-08T06:04:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-05-08T06:04:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 24876, 2024-05-08T06:04:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 74308843, 2024-05-08T06:04:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-08T06:04:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:14.811 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:14.811 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:04:15.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:15 smithi112 ceph-mon[30003]: pgmap v709: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-08T06:04:15.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:15 smithi150 ceph-mon[34315]: pgmap v709: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-08T06:04:15.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:15 smithi162 ceph-mon[34800]: pgmap v709: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-08T06:04:15.818 INFO:tasks.check_counter:Daemon mds.a mds.exported=82 2024-05-08T06:04:15.818 INFO:tasks.check_counter:Daemon mds.a mds.imported=25 2024-05-08T06:04:15.818 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=14 2024-05-08T06:04:15.818 DEBUG:tasks.check_counter:Getting stats from d 2024-05-08T06:04:15.818 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-08T06:04:16.273 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:17.469 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:17 smithi112 ceph-mon[30003]: pgmap v710: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T06:04:17.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:17 smithi150 ceph-mon[34315]: pgmap v710: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T06:04:17.817 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:17 smithi162 ceph-mon[34800]: pgmap v710: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T06:04:18.539 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:04:18.539 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:18.539 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-05-08T06:04:18.539 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-05-08T06:04:18.540 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-05-08T06:04:18.540 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-05-08T06:04:18.540 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:18.540 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-05-08T06:04:18.540 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.004052360, 2024-05-08T06:04:18.540 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T06:04:18.540 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000868228, 2024-05-08T06:04:18.540 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:18.540 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:18.540 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:18.540 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:18.540 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:18.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:18.540 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:18.540 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 5, 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 173, 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3104, 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 109040, 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 3, 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.030151900, 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.023650528, 2024-05-08T06:04:18.541 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.002978790, 2024-05-08T06:04:18.542 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:18.542 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:18.542 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 172, 2024-05-08T06:04:18.542 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.017280586, 2024-05-08T06:04:18.542 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000100468 2024-05-08T06:04:18.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:18.542 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:18.542 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:18.542 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:18.542 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:18.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:18.542 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-08T06:04:18.542 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 109040 2024-05-08T06:04:18.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 227, 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 224, 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 515776, 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 138144, 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.085045751, 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.033157534, 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.050341579, 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.011234240, 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 223, 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.020662186, 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000092655 2024-05-08T06:04:18.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:18.544 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:18.544 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:18.544 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:18.544 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:18.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:18.544 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 515776, 2024-05-08T06:04:18.544 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 138144 2024-05-08T06:04:18.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:18.544 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-05-08T06:04:18.544 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-05-08T06:04:18.544 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-05-08T06:04:18.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:18.544 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-05-08T06:04:18.545 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:18.545 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:18.545 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:18.545 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:18.545 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:18.545 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:18.545 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:18.545 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:18.545 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:18.545 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:18.545 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:18.545 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:18.545 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:18.545 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:18.545 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:18.546 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:18.546 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:18.546 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:18.546 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:18.546 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:18.546 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:18.546 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:18.546 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:18.546 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:18.546 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:18.546 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:18.546 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:18.546 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:18.547 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:18.547 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:18.547 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:18.547 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-05-08T06:04:18.547 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:18.547 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:18.547 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:18.547 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:18.547 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 149998, 2024-05-08T06:04:18.547 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 30, 2024-05-08T06:04:18.547 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:18.547 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:18.547 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-05-08T06:04:18.547 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-05-08T06:04:18.548 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:18.548 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:18.548 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:18.548 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:18.548 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-05-08T06:04:18.548 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-05-08T06:04:18.548 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:18.548 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:18.548 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:18.548 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-05-08T06:04:18.548 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-05-08T06:04:18.548 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-05-08T06:04:18.548 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:18.548 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-05-08T06:04:18.548 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 228, 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 483685, 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 228, 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 228, 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 483685, 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-08T06:04:18.549 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:18.550 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:18.550 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:18.550 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:18.550 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:18.550 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:04:19.326 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-08T06:04:19.326 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-08T06:04:19.326 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-05-08T06:04:19.326 DEBUG:tasks.check_counter:Getting stats from g 2024-05-08T06:04:19.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-08T06:04:19.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:19 smithi112 ceph-mon[30003]: pgmap v711: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:19.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:19 smithi150 ceph-mon[34315]: pgmap v711: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:19.793 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:19.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:19 smithi162 ceph-mon[34800]: pgmap v711: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:21.629 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:21 smithi112 ceph-mon[30003]: pgmap v712: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:04:21.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:21 smithi150 ceph-mon[34315]: pgmap v712: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:04:21.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:21 smithi162 ceph-mon[34800]: pgmap v712: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:04:21.955 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:04:21.955 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:21.955 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 223, 2024-05-08T06:04:21.955 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 220, 2024-05-08T06:04:21.956 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 514688, 2024-05-08T06:04:21.956 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 135584, 2024-05-08T06:04:21.956 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:21.956 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-05-08T06:04:21.956 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.083028997, 2024-05-08T06:04:21.956 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.035312408, 2024-05-08T06:04:21.956 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.036877023, 2024-05-08T06:04:21.956 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.009549833, 2024-05-08T06:04:21.956 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:21.956 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 219, 2024-05-08T06:04:21.956 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.022530947, 2024-05-08T06:04:21.956 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000102881 2024-05-08T06:04:21.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:21.956 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:21.956 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 514688, 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 135584 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 5, 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 170, 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3104, 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 108512, 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 3, 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.028941767, 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.023336731, 2024-05-08T06:04:21.957 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.002179792, 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 169, 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.015547875, 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000091999 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 108512 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:21.958 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-05-08T06:04:21.959 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-05-08T06:04:21.959 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-05-08T06:04:21.959 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-05-08T06:04:21.959 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-05-08T06:04:21.959 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:21.959 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.000950156, 2024-05-08T06:04:21.959 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T06:04:21.959 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000788599, 2024-05-08T06:04:21.959 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:21.959 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:21.959 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:21.959 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:21.959 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:21.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:21.959 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:21.960 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:21.961 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:21.962 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:21.962 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:21.962 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:21.962 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:21.962 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:21.962 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:21.962 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:21.962 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:21.962 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:21.962 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:21.962 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:21.962 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-05-08T06:04:21.962 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:21.962 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 149998, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 30, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:21.963 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 224, 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 483165, 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:21.964 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 224, 2024-05-08T06:04:21.965 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-05-08T06:04:21.965 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-05-08T06:04:21.965 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 224, 2024-05-08T06:04:21.965 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 483165, 2024-05-08T06:04:21.965 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-08T06:04:21.965 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:21.965 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:21.965 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:21.965 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:21.965 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:21.965 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:04:22.768 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-08T06:04:22.768 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-08T06:04:22.768 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-05-08T06:04:22.768 DEBUG:tasks.check_counter:Getting stats from j 2024-05-08T06:04:22.769 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-08T06:04:23.240 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:23.619 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:23 smithi162 ceph-mon[34800]: pgmap v713: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:23.640 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:23 smithi112 ceph-mon[30003]: pgmap v713: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:23.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:23 smithi150 ceph-mon[34315]: pgmap v713: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:25.593 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-05-08T06:04:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-05-08T06:04:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-05-08T06:04:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-05-08T06:04:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-05-08T06:04:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-05-08T06:04:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.003706602, 2024-05-08T06:04:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T06:04:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000816071, 2024-05-08T06:04:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:04:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:04:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 5, 2024-05-08T06:04:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 175, 2024-05-08T06:04:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3104, 2024-05-08T06:04:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 109392, 2024-05-08T06:04:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 3, 2024-05-08T06:04:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-05-08T06:04:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.030959324, 2024-05-08T06:04:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.024550728, 2024-05-08T06:04:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.002839423, 2024-05-08T06:04:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 174, 2024-05-08T06:04:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.017028862, 2024-05-08T06:04:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000097867 2024-05-08T06:04:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-08T06:04:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 109392 2024-05-08T06:04:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 230, 2024-05-08T06:04:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 227, 2024-05-08T06:04:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 516592, 2024-05-08T06:04:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 140064, 2024-05-08T06:04:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-05-08T06:04:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.077570871, 2024-05-08T06:04:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.032002962, 2024-05-08T06:04:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.039334156, 2024-05-08T06:04:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.010234524, 2024-05-08T06:04:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 226, 2024-05-08T06:04:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.021975029, 2024-05-08T06:04:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000097234 2024-05-08T06:04:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 516592, 2024-05-08T06:04:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 140064 2024-05-08T06:04:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-05-08T06:04:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-05-08T06:04:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-05-08T06:04:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-05-08T06:04:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-05-08T06:04:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 149998, 2024-05-08T06:04:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 30, 2024-05-08T06:04:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-05-08T06:04:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-05-08T06:04:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-05-08T06:04:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-05-08T06:04:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-05-08T06:04:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-05-08T06:04:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-05-08T06:04:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-05-08T06:04:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-05-08T06:04:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-05-08T06:04:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-05-08T06:04:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-05-08T06:04:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-05-08T06:04:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 231, 2024-05-08T06:04:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 484075, 2024-05-08T06:04:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 231, 2024-05-08T06:04:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-05-08T06:04:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-05-08T06:04:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 231, 2024-05-08T06:04:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 484075, 2024-05-08T06:04:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-05-08T06:04:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-05-08T06:04:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-05-08T06:04:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-05-08T06:04:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-05-08T06:04:25.605 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-05-08T06:04:25.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:25 smithi150 ceph-mon[34315]: pgmap v714: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:04:25.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:25 smithi112 ceph-mon[30003]: pgmap v714: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:04:25.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:25 smithi162 ceph-mon[34800]: pgmap v714: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:04:26.421 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-08T06:04:26.421 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-08T06:04:26.421 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-05-08T06:04:26.421 DEBUG:tasks.check_counter:Getting stats from b 2024-05-08T06:04:26.422 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-08T06:04:26.676 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:04:27.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:27 smithi150 ceph-mon[34315]: pgmap v715: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T06:04:27.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:27 smithi112 ceph-mon[30003]: pgmap v715: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T06:04:27.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:27 smithi162 ceph-mon[34800]: pgmap v715: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T06:04:28.462 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-08T06:04:28.462 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:28.462 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 8679, 2024-05-08T06:04:28.462 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 10051, 2024-05-08T06:04:28.462 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 5701812, 2024-05-08T06:04:28.462 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 19120896, 2024-05-08T06:04:28.463 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 30, 2024-05-08T06:04:28.463 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 5, 2024-05-08T06:04:28.463 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.588543142, 2024-05-08T06:04:28.463 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.232302879, 2024-05-08T06:04:28.463 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.242386648, 2024-05-08T06:04:28.463 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.118683282, 2024-05-08T06:04:28.463 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:28.463 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 10049, 2024-05-08T06:04:28.463 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.281975733, 2024-05-08T06:04:28.463 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000028060 2024-05-08T06:04:28.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.463 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:28.463 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 7295, 2024-05-08T06:04:28.463 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.001268777, 2024-05-08T06:04:28.464 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000173 2024-05-08T06:04:28.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.464 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:04:28.464 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:04:28.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.464 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:28.464 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 15989, 2024-05-08T06:04:28.464 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 25733, 2024-05-08T06:04:28.464 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 4821952, 2024-05-08T06:04:28.464 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 15741330, 2024-05-08T06:04:28.464 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 12, 2024-05-08T06:04:28.464 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 6, 2024-05-08T06:04:28.464 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 1.047753615, 2024-05-08T06:04:28.464 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.454833883, 2024-05-08T06:04:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.498959312, 2024-05-08T06:04:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.147907467, 2024-05-08T06:04:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 25731, 2024-05-08T06:04:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.604420232, 2024-05-08T06:04:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000023489 2024-05-08T06:04:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 19745, 2024-05-08T06:04:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.003761268, 2024-05-08T06:04:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000190 2024-05-08T06:04:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T06:04:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 371568 2024-05-08T06:04:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.466 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:28.466 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 10797, 2024-05-08T06:04:28.466 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 8789, 2024-05-08T06:04:28.466 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 3651335, 2024-05-08T06:04:28.466 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 6426150, 2024-05-08T06:04:28.466 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 7, 2024-05-08T06:04:28.466 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 6, 2024-05-08T06:04:28.466 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.618017636, 2024-05-08T06:04:28.466 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.220245791, 2024-05-08T06:04:28.466 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.320940281, 2024-05-08T06:04:28.466 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.122326140, 2024-05-08T06:04:28.466 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:28.466 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 8789, 2024-05-08T06:04:28.467 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.297224212, 2024-05-08T06:04:28.467 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000033817 2024-05-08T06:04:28.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.467 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:28.467 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 8024, 2024-05-08T06:04:28.467 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.001206948, 2024-05-08T06:04:28.467 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000150 2024-05-08T06:04:28.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.467 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 607840, 2024-05-08T06:04:28.467 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 149040 2024-05-08T06:04:28.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.467 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-08T06:04:28.468 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-05-08T06:04:28.468 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-08T06:04:28.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.468 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-05-08T06:04:28.468 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-05-08T06:04:28.468 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-05-08T06:04:28.468 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 12143, 2024-05-08T06:04:28.468 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 5.929822398, 2024-05-08T06:04:28.468 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000488332 2024-05-08T06:04:28.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:28.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.468 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-05-08T06:04:28.468 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-05-08T06:04:28.468 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-05-08T06:04:28.469 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 4568, 2024-05-08T06:04:28.469 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.750460631, 2024-05-08T06:04:28.469 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000164286 2024-05-08T06:04:28.469 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:28.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.469 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-05-08T06:04:28.469 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 7008, 2024-05-08T06:04:28.469 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 6735, 2024-05-08T06:04:28.469 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-05-08T06:04:28.469 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 6735, 2024-05-08T06:04:28.469 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 18.783958009, 2024-05-08T06:04:28.469 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002789006 2024-05-08T06:04:28.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.469 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-05-08T06:04:28.469 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 273, 2024-05-08T06:04:28.470 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-05-08T06:04:28.470 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-05-08T06:04:28.470 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 67, 2024-05-08T06:04:28.470 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 7, 2024-05-08T06:04:28.470 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-05-08T06:04:28.470 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 77, 2024-05-08T06:04:28.470 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 41, 2024-05-08T06:04:28.470 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 18, 2024-05-08T06:04:28.470 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 18, 2024-05-08T06:04:28.470 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 18, 2024-05-08T06:04:28.470 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 1497, 2024-05-08T06:04:28.470 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 2, 2024-05-08T06:04:28.471 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 2, 2024-05-08T06:04:28.471 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 3, 2024-05-08T06:04:28.471 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 11059, 2024-05-08T06:04:28.471 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 8661, 2024-05-08T06:04:28.471 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 204, 2024-05-08T06:04:28.471 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 18, 2024-05-08T06:04:28.471 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-05-08T06:04:28.471 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-05-08T06:04:28.471 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 4, 2024-05-08T06:04:28.471 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-05-08T06:04:28.472 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-05-08T06:04:28.472 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 41, 2024-05-08T06:04:28.472 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 1508, 2024-05-08T06:04:28.472 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 105, 2024-05-08T06:04:28.472 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 4063, 2024-05-08T06:04:28.472 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-05-08T06:04:28.472 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 14669, 2024-05-08T06:04:28.472 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-05-08T06:04:28.472 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 4567, 2024-05-08T06:04:28.472 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 87759380, 2024-05-08T06:04:28.472 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-05-08T06:04:28.472 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 14669, 2024-05-08T06:04:28.472 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-05-08T06:04:28.473 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T06:04:28.473 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-05-08T06:04:28.473 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 267, 2024-05-08T06:04:28.473 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T06:04:28.473 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 267, 2024-05-08T06:04:28.473 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 14402, 2024-05-08T06:04:28.473 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 18, 2024-05-08T06:04:28.473 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 195, 2024-05-08T06:04:28.473 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 3267, 2024-05-08T06:04:28.473 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 2, 2024-05-08T06:04:28.473 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T06:04:28.473 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T06:04:28.473 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 1590, 2024-05-08T06:04:28.473 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 1330, 2024-05-08T06:04:28.473 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 55, 2024-05-08T06:04:28.474 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 5449 2024-05-08T06:04:28.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.474 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-05-08T06:04:28.474 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-05-08T06:04:28.474 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-05-08T06:04:28.474 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-05-08T06:04:28.474 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 126, 2024-05-08T06:04:28.474 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 11, 2024-05-08T06:04:28.474 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 4, 2024-05-08T06:04:28.474 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-05-08T06:04:28.474 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-05-08T06:04:28.474 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 2558, 2024-05-08T06:04:28.475 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 2554, 2024-05-08T06:04:28.475 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-05-08T06:04:28.475 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-05-08T06:04:28.475 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-05-08T06:04:28.475 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-05-08T06:04:28.475 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-05-08T06:04:28.475 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-05-08T06:04:28.475 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-05-08T06:04:28.475 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-05-08T06:04:28.475 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-05-08T06:04:28.475 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T06:04:28.475 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 41, 2024-05-08T06:04:28.475 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-05-08T06:04:28.475 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 7, 2024-05-08T06:04:28.475 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-05-08T06:04:28.476 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-05-08T06:04:28.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.476 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-05-08T06:04:28.476 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 1, 2024-05-08T06:04:28.476 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 8787, 2024-05-08T06:04:28.476 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-05-08T06:04:28.476 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-05-08T06:04:28.476 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 8787, 2024-05-08T06:04:28.476 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-05-08T06:04:28.476 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-05-08T06:04:28.476 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 12, 2024-05-08T06:04:28.476 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-05-08T06:04:28.477 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-05-08T06:04:28.477 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 12, 2024-05-08T06:04:28.477 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 2, 2024-05-08T06:04:28.477 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-05-08T06:04:28.477 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-05-08T06:04:28.477 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-05-08T06:04:28.477 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 22881505, 2024-05-08T06:04:28.477 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 0, 2024-05-08T06:04:28.477 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-05-08T06:04:28.477 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 372, 2024-05-08T06:04:28.477 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 11.293260869, 2024-05-08T06:04:28.477 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.030358228 2024-05-08T06:04:28.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.477 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 0 2024-05-08T06:04:28.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.478 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-05-08T06:04:28.478 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 23, 2024-05-08T06:04:28.478 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 3558, 2024-05-08T06:04:28.478 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 3535, 2024-05-08T06:04:28.478 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 36, 2024-05-08T06:04:28.478 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 250, 2024-05-08T06:04:28.478 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 214, 2024-05-08T06:04:28.478 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 77, 2024-05-08T06:04:28.478 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 7191, 2024-05-08T06:04:28.478 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 7114, 2024-05-08T06:04:28.478 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 2, 2024-05-08T06:04:28.478 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 6388, 2024-05-08T06:04:28.479 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 6386, 2024-05-08T06:04:28.479 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 82744, 2024-05-08T06:04:28.479 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 223608 2024-05-08T06:04:28.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.479 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-05-08T06:04:28.479 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 8164, 2024-05-08T06:04:28.479 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 2129, 2024-05-08T06:04:28.479 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 7008, 2024-05-08T06:04:28.479 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 27, 2024-05-08T06:04:28.479 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 4791, 2024-05-08T06:04:28.480 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-05-08T06:04:28.480 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 438, 2024-05-08T06:04:28.480 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.885862187, 2024-05-08T06:04:28.480 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002022516 2024-05-08T06:04:28.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.480 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-05-08T06:04:28.480 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 10, 2024-05-08T06:04:28.480 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.034155550, 2024-05-08T06:04:28.480 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.003415555 2024-05-08T06:04:28.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.480 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-05-08T06:04:28.480 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.480 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.480 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-05-08T06:04:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-05-08T06:04:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3156, 2024-05-08T06:04:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.609961119, 2024-05-08T06:04:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000510127 2024-05-08T06:04:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-05-08T06:04:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-05-08T06:04:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1, 2024-05-08T06:04:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000251773, 2024-05-08T06:04:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000251773 2024-05-08T06:04:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-05-08T06:04:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-05-08T06:04:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-05-08T06:04:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-05-08T06:04:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-05-08T06:04:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3, 2024-05-08T06:04:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.006561733, 2024-05-08T06:04:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002187244 2024-05-08T06:04:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-05-08T06:04:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-05-08T06:04:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-05-08T06:04:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 108, 2024-05-08T06:04:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.036924126, 2024-05-08T06:04:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000341890 2024-05-08T06:04:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-05-08T06:04:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 443, 2024-05-08T06:04:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.345852221, 2024-05-08T06:04:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000780704 2024-05-08T06:04:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-05-08T06:04:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 15, 2024-05-08T06:04:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 9.821817819, 2024-05-08T06:04:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.654787854 2024-05-08T06:04:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-05-08T06:04:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-05-08T06:04:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-05-08T06:04:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 49, 2024-05-08T06:04:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.749926436, 2024-05-08T06:04:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.035712784 2024-05-08T06:04:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-05-08T06:04:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-05-08T06:04:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-05-08T06:04:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2, 2024-05-08T06:04:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.007840670, 2024-05-08T06:04:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.003920335 2024-05-08T06:04:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-05-08T06:04:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-05-08T06:04:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-05-08T06:04:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-05-08T06:04:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-05-08T06:04:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1, 2024-05-08T06:04:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000645654, 2024-05-08T06:04:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000645654 2024-05-08T06:04:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-05-08T06:04:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2509, 2024-05-08T06:04:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 4.284158721, 2024-05-08T06:04:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001707516 2024-05-08T06:04:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-05-08T06:04:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-05-08T06:04:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-05-08T06:04:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-05-08T06:04:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 1, 2024-05-08T06:04:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 1, 2024-05-08T06:04:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 0, 2024-05-08T06:04:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 1, 2024-05-08T06:04:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-05-08T06:04:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 4588, 2024-05-08T06:04:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 4588, 2024-05-08T06:04:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 568, 2024-05-08T06:04:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-05-08T06:04:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-08T06:04:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-08T06:04:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 298144, 2024-05-08T06:04:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 119, 2024-05-08T06:04:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 440, 2024-05-08T06:04:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 5, 2024-05-08T06:04:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-08T06:04:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-08T06:04:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 7728, 2024-05-08T06:04:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 163, 2024-05-08T06:04:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-08T06:04:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 160494, 2024-05-08T06:04:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 1614, 2024-05-08T06:04:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-08T06:04:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-08T06:04:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-05-08T06:04:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-05-08T06:04:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-05-08T06:04:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 18047, 2024-05-08T06:04:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 19891603, 2024-05-08T06:04:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 28, 2024-05-08T06:04:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 18047, 2024-05-08T06:04:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-05-08T06:04:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 18047, 2024-05-08T06:04:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 25.021509307, 2024-05-08T06:04:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001386463 2024-05-08T06:04:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-05-08T06:04:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-05-08T06:04:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 18047, 2024-05-08T06:04:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 18196 2024-05-08T06:04:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 18047, 2024-05-08T06:04:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 14721, 2024-05-08T06:04:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 3326, 2024-05-08T06:04:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-05-08T06:04:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-05-08T06:04:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 0, 2024-05-08T06:04:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 10, 2024-05-08T06:04:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 52, 2024-05-08T06:04:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 424, 2024-05-08T06:04:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 44, 2024-05-08T06:04:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-05-08T06:04:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-05-08T06:04:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-05-08T06:04:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-05-08T06:04:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 2723, 2024-05-08T06:04:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-05-08T06:04:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-05-08T06:04:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-05-08T06:04:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 14669, 2024-05-08T06:04:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 20, 2024-05-08T06:04:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-05-08T06:04:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-05-08T06:04:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-05-08T06:04:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-05-08T06:04:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-05-08T06:04:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-05-08T06:04:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T06:04:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-05-08T06:04:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-05-08T06:04:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 2, 2024-05-08T06:04:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-05-08T06:04:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-05-08T06:04:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-05-08T06:04:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-05-08T06:04:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-05-08T06:04:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-05-08T06:04:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-05-08T06:04:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-05-08T06:04:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-05-08T06:04:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-05-08T06:04:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-05-08T06:04:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-05-08T06:04:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-05-08T06:04:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-05-08T06:04:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-05-08T06:04:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-05-08T06:04:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 84, 2024-05-08T06:04:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-05-08T06:04:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 4, 2024-05-08T06:04:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 12, 2024-05-08T06:04:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 12, 2024-05-08T06:04:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-05-08T06:04:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-05-08T06:04:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 229, 2024-05-08T06:04:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 0, 2024-05-08T06:04:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 23 2024-05-08T06:04:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-05-08T06:04:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 1, 2024-05-08T06:04:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-05-08T06:04:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-05-08T06:04:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-05-08T06:04:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-05-08T06:04:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-05-08T06:04:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-05-08T06:04:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-05-08T06:04:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 65, 2024-05-08T06:04:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-05-08T06:04:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 10, 2024-05-08T06:04:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 2700, 2024-05-08T06:04:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 2554, 2024-05-08T06:04:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-05-08T06:04:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-08T06:04:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-08T06:04:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-08T06:04:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 35461, 2024-05-08T06:04:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 10966005, 2024-05-08T06:04:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 35461, 2024-05-08T06:04:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-08T06:04:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-08T06:04:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 35461, 2024-05-08T06:04:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 10966005, 2024-05-08T06:04:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-08T06:04:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-05-08T06:04:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-08T06:04:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-08T06:04:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-08T06:04:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-08T06:04:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-08T06:04:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 18047, 2024-05-08T06:04:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 20149589, 2024-05-08T06:04:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 18047, 2024-05-08T06:04:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 20149589, 2024-05-08T06:04:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-08T06:04:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-05-08T06:04:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-08T06:04:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-05-08T06:04:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-08T06:04:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-08T06:04:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-08T06:04:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 18047, 2024-05-08T06:04:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 18047, 2024-05-08T06:04:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 18047, 2024-05-08T06:04:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 18047, 2024-05-08T06:04:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-08T06:04:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-05-08T06:04:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-08T06:04:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-05-08T06:04:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-08T06:04:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 2554, 2024-05-08T06:04:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 257986, 2024-05-08T06:04:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 2554, 2024-05-08T06:04:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-08T06:04:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-08T06:04:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 52, 2024-05-08T06:04:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 257986, 2024-05-08T06:04:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-08T06:04:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x5574c4d0c500": { 2024-05-08T06:04:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-08T06:04:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-05-08T06:04:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-08T06:04:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 8787, 2024-05-08T06:04:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 18687201, 2024-05-08T06:04:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 8787, 2024-05-08T06:04:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-08T06:04:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-08T06:04:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 372, 2024-05-08T06:04:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 18687201, 2024-05-08T06:04:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-08T06:04:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:28.514 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-08T06:04:29.299 INFO:tasks.check_counter:Daemon mds.b mds.exported=41 2024-05-08T06:04:29.299 INFO:tasks.check_counter:Daemon mds.b mds.imported=105 2024-05-08T06:04:29.300 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=7 2024-05-08T06:04:29.300 DEBUG:tasks.check_counter:Getting stats from e 2024-05-08T06:04:29.300 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-08T06:04:29.550 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:04:29.592 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:29 smithi150 ceph-mon[34315]: pgmap v716: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:29.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:29 smithi112 ceph-mon[30003]: pgmap v716: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:29.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:29 smithi162 ceph-mon[34800]: pgmap v716: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:31.277 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-08T06:04:31.277 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:31.277 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 0, 2024-05-08T06:04:31.277 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2, 2024-05-08T06:04:31.277 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 0, 2024-05-08T06:04:31.277 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 0, 2024-05-08T06:04:31.277 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:31.277 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-05-08T06:04:31.277 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.000948093, 2024-05-08T06:04:31.277 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T06:04:31.277 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000113052, 2024-05-08T06:04:31.277 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:31.278 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:31.278 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:31.278 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:31.278 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:31.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:31.278 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:31.278 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:31.278 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:31.278 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:31.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:31.278 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:04:31.278 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:04:31.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:31.278 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:31.279 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1, 2024-05-08T06:04:31.279 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 173, 2024-05-08T06:04:31.279 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 160, 2024-05-08T06:04:31.279 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 108944, 2024-05-08T06:04:31.279 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 3, 2024-05-08T06:04:31.279 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-05-08T06:04:31.279 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.009506611, 2024-05-08T06:04:31.279 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.007000381, 2024-05-08T06:04:31.279 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000647894, 2024-05-08T06:04:31.279 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:31.279 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:31.279 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 171, 2024-05-08T06:04:31.279 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.010678015, 2024-05-08T06:04:31.279 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000062444 2024-05-08T06:04:31.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:31.280 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:31.280 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:31.280 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:31.280 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:31.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:31.280 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T06:04:31.280 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 108944 2024-05-08T06:04:31.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:31.280 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:31.280 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 233, 2024-05-08T06:04:31.280 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 228, 2024-05-08T06:04:31.280 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 519264, 2024-05-08T06:04:31.280 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 139872, 2024-05-08T06:04:31.280 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-05-08T06:04:31.281 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:31.281 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.034808931, 2024-05-08T06:04:31.281 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.013236246, 2024-05-08T06:04:31.281 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.015418197, 2024-05-08T06:04:31.281 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.005531478, 2024-05-08T06:04:31.281 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:31.281 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 228, 2024-05-08T06:04:31.281 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.012069461, 2024-05-08T06:04:31.281 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000052936 2024-05-08T06:04:31.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:31.281 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:31.281 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:31.281 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:31.281 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:31.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:31.282 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 519264, 2024-05-08T06:04:31.282 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 139872 2024-05-08T06:04:31.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:31.282 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-08T06:04:31.282 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-05-08T06:04:31.282 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-08T06:04:31.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:31.282 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-08T06:04:31.282 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:31.282 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:31.282 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:31.282 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:31.282 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:31.282 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:31.283 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:31.283 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:31.283 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:31.283 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:31.283 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:31.283 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:31.283 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:31.283 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:31.283 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:31.283 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:31.283 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:31.283 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:31.283 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:31.283 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:31.283 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:31.284 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:31.284 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:31.284 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:31.284 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:31.284 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:31.284 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:31.284 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:31.284 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:31.284 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:31.284 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:31.284 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-08T06:04:31.284 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:31.284 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:31.284 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:31.284 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:31.285 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148945, 2024-05-08T06:04:31.285 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 29, 2024-05-08T06:04:31.285 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:31.285 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:31.285 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-08T06:04:31.285 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-08T06:04:31.285 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:31.285 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:31.285 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:31.285 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:31.285 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-05-08T06:04:31.285 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-05-08T06:04:31.285 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:31.285 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:31.285 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:31.286 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-08T06:04:31.286 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-05-08T06:04:31.286 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-05-08T06:04:31.286 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:31.286 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-08T06:04:31.286 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:31.286 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-08T06:04:31.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:31.286 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:31.286 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-08T06:04:31.286 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-08T06:04:31.286 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-08T06:04:31.286 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 230, 2024-05-08T06:04:31.286 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 483945, 2024-05-08T06:04:31.287 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:31.287 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 230, 2024-05-08T06:04:31.287 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-08T06:04:31.287 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-08T06:04:31.287 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 230, 2024-05-08T06:04:31.287 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 483945, 2024-05-08T06:04:31.287 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-08T06:04:31.287 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:31.287 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:31.287 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:31.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:31.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:31.287 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-08T06:04:31.653 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:31 smithi150 ceph-mon[34315]: pgmap v717: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:04:31.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:31 smithi112 ceph-mon[30003]: pgmap v717: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:04:31.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:31 smithi162 ceph-mon[34800]: pgmap v717: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:04:31.982 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-05-08T06:04:31.982 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-05-08T06:04:31.982 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-05-08T06:04:31.982 DEBUG:tasks.check_counter:Getting stats from h 2024-05-08T06:04:31.983 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-08T06:04:32.224 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:04:33.582 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:33 smithi150 ceph-mon[34315]: pgmap v718: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:33.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:33 smithi112 ceph-mon[30003]: pgmap v718: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:33.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:33 smithi162 ceph-mon[34800]: pgmap v718: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:33.832 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-08T06:04:33.832 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:33.832 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 763, 2024-05-08T06:04:33.832 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 763, 2024-05-08T06:04:33.833 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 4396403, 2024-05-08T06:04:33.833 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 241874, 2024-05-08T06:04:33.833 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 6, 2024-05-08T06:04:33.833 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-05-08T06:04:33.833 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.064416283, 2024-05-08T06:04:33.833 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.025903549, 2024-05-08T06:04:33.833 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.016219204, 2024-05-08T06:04:33.833 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.015739384, 2024-05-08T06:04:33.833 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:33.833 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 762, 2024-05-08T06:04:33.833 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.027261415, 2024-05-08T06:04:33.833 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000035776 2024-05-08T06:04:33.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.834 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:33.834 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.834 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.834 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.834 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-08T06:04:33.834 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T06:04:33.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.834 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:33.834 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 697, 2024-05-08T06:04:33.834 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 868, 2024-05-08T06:04:33.834 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 4365878, 2024-05-08T06:04:33.834 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 593093, 2024-05-08T06:04:33.835 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-05-08T06:04:33.835 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2024-05-08T06:04:33.835 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.067191968, 2024-05-08T06:04:33.835 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.023404927, 2024-05-08T06:04:33.835 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.015244550, 2024-05-08T06:04:33.835 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.021851401, 2024-05-08T06:04:33.835 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:33.835 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 866, 2024-05-08T06:04:33.835 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.033571244, 2024-05-08T06:04:33.835 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000038765 2024-05-08T06:04:33.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.835 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:33.835 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.836 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.836 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.836 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T06:04:33.836 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 371760 2024-05-08T06:04:33.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.836 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:33.836 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1095, 2024-05-08T06:04:33.836 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 1079, 2024-05-08T06:04:33.836 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 2322881, 2024-05-08T06:04:33.836 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 409030, 2024-05-08T06:04:33.836 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:33.836 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-05-08T06:04:33.837 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.129114978, 2024-05-08T06:04:33.837 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.042081646, 2024-05-08T06:04:33.837 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.041382965, 2024-05-08T06:04:33.837 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.037069057, 2024-05-08T06:04:33.837 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:33.837 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1078, 2024-05-08T06:04:33.837 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.055856510, 2024-05-08T06:04:33.837 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000051814 2024-05-08T06:04:33.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.837 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:33.837 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.838 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.838 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.838 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 587648, 2024-05-08T06:04:33.838 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 140320 2024-05-08T06:04:33.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.838 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-08T06:04:33.838 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-05-08T06:04:33.838 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-08T06:04:33.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.838 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-05-08T06:04:33.838 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-05-08T06:04:33.838 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-05-08T06:04:33.839 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3891, 2024-05-08T06:04:33.839 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.106558119, 2024-05-08T06:04:33.839 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000027385 2024-05-08T06:04:33.839 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:33.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.839 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-05-08T06:04:33.839 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-05-08T06:04:33.839 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-05-08T06:04:33.839 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.839 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.839 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.839 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:33.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.839 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-05-08T06:04:33.839 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 0, 2024-05-08T06:04:33.840 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 0, 2024-05-08T06:04:33.840 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-05-08T06:04:33.840 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.840 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.840 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.840 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-05-08T06:04:33.840 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-05-08T06:04:33.840 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-05-08T06:04:33.840 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-05-08T06:04:33.840 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 0, 2024-05-08T06:04:33.840 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-05-08T06:04:33.840 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-05-08T06:04:33.841 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 2061, 2024-05-08T06:04:33.841 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 0, 2024-05-08T06:04:33.841 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 2061, 2024-05-08T06:04:33.841 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-05-08T06:04:33.841 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 2061, 2024-05-08T06:04:33.841 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-05-08T06:04:33.841 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 0, 2024-05-08T06:04:33.841 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 0, 2024-05-08T06:04:33.841 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 1, 2024-05-08T06:04:33.841 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 0, 2024-05-08T06:04:33.841 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 0, 2024-05-08T06:04:33.841 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-05-08T06:04:33.841 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-05-08T06:04:33.841 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-05-08T06:04:33.841 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-05-08T06:04:33.842 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-05-08T06:04:33.842 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-05-08T06:04:33.842 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-05-08T06:04:33.842 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-05-08T06:04:33.842 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-05-08T06:04:33.842 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-05-08T06:04:33.842 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-05-08T06:04:33.842 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-05-08T06:04:33.842 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-05-08T06:04:33.842 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-05-08T06:04:33.842 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 0, 2024-05-08T06:04:33.842 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 0, 2024-05-08T06:04:33.842 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-05-08T06:04:33.842 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T06:04:33.843 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-05-08T06:04:33.843 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T06:04:33.843 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-05-08T06:04:33.843 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-05-08T06:04:33.843 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T06:04:33.843 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T06:04:33.843 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-05-08T06:04:33.843 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-05-08T06:04:33.843 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T06:04:33.843 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 0, 2024-05-08T06:04:33.843 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T06:04:33.843 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T06:04:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T06:04:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 0, 2024-05-08T06:04:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 0, 2024-05-08T06:04:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2024-05-08T06:04:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 0 2024-05-08T06:04:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-05-08T06:04:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-05-08T06:04:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-05-08T06:04:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-05-08T06:04:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-05-08T06:04:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-05-08T06:04:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-05-08T06:04:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-05-08T06:04:33.844 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-05-08T06:04:33.845 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2024-05-08T06:04:33.845 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2024-05-08T06:04:33.845 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-05-08T06:04:33.845 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-05-08T06:04:33.845 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-05-08T06:04:33.845 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-05-08T06:04:33.845 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-05-08T06:04:33.845 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-05-08T06:04:33.845 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-05-08T06:04:33.845 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-05-08T06:04:33.845 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-05-08T06:04:33.845 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T06:04:33.845 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-05-08T06:04:33.845 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-05-08T06:04:33.846 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-05-08T06:04:33.846 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-05-08T06:04:33.846 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-05-08T06:04:33.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.846 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-05-08T06:04:33.846 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-05-08T06:04:33.846 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 0, 2024-05-08T06:04:33.846 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-05-08T06:04:33.846 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-05-08T06:04:33.846 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 5507, 2024-05-08T06:04:33.846 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-05-08T06:04:33.846 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-05-08T06:04:33.846 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 0, 2024-05-08T06:04:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-05-08T06:04:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-05-08T06:04:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 8, 2024-05-08T06:04:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 2, 2024-05-08T06:04:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-05-08T06:04:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-05-08T06:04:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-05-08T06:04:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 14038803, 2024-05-08T06:04:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 14037665, 2024-05-08T06:04:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-05-08T06:04:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 5507 2024-05-08T06:04:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-05-08T06:04:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 10, 2024-05-08T06:04:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 2096, 2024-05-08T06:04:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 2086, 2024-05-08T06:04:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 8, 2024-05-08T06:04:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 110, 2024-05-08T06:04:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 102, 2024-05-08T06:04:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 2061, 2024-05-08T06:04:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 4153, 2024-05-08T06:04:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 2092, 2024-05-08T06:04:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 0, 2024-05-08T06:04:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 0, 2024-05-08T06:04:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 0, 2024-05-08T06:04:33.849 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 58852, 2024-05-08T06:04:33.849 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 215416 2024-05-08T06:04:33.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.849 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-05-08T06:04:33.849 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 0, 2024-05-08T06:04:33.849 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2024-05-08T06:04:33.849 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 0, 2024-05-08T06:04:33.849 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 0, 2024-05-08T06:04:33.849 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2024-05-08T06:04:33.849 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-05-08T06:04:33.849 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.849 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.849 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.851 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-05-08T06:04:33.851 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.851 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.851 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.851 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-05-08T06:04:33.851 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.851 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.851 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.851 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-05-08T06:04:33.851 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.851 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.851 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.852 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-05-08T06:04:33.852 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.852 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.852 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.852 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-05-08T06:04:33.852 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.852 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.852 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.852 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-05-08T06:04:33.852 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.852 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.852 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.853 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-05-08T06:04:33.853 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.853 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.853 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.853 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-05-08T06:04:33.853 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.853 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.853 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.854 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-05-08T06:04:33.854 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.854 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.854 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.854 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-05-08T06:04:33.854 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.854 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.854 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.854 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-05-08T06:04:33.854 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.854 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.854 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.855 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-05-08T06:04:33.855 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.855 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.855 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.855 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-05-08T06:04:33.855 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.855 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.855 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.855 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-05-08T06:04:33.855 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.855 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.855 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.856 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-05-08T06:04:33.856 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.856 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.856 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.856 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-05-08T06:04:33.856 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.856 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.856 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.856 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-05-08T06:04:33.856 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.856 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.856 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.858 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.859 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:33.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.860 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-05-08T06:04:33.860 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 1, 2024-05-08T06:04:33.860 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 2, 2024-05-08T06:04:33.860 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 1, 2024-05-08T06:04:33.860 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 1, 2024-05-08T06:04:33.860 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-05-08T06:04:33.860 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 0, 2024-05-08T06:04:33.860 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 0, 2024-05-08T06:04:33.860 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 2252, 2024-05-08T06:04:33.860 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-05-08T06:04:33.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.860 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-08T06:04:33.860 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:33.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-08T06:04:33.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 197441, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 54, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 7728, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 163, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:33.863 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 1484024, 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 8339, 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 2300, 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 21, 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 3, 2024-05-08T06:04:33.864 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 2300, 2024-05-08T06:04:33.865 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-05-08T06:04:33.865 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2300, 2024-05-08T06:04:33.865 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.601968488, 2024-05-08T06:04:33.865 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000696508 2024-05-08T06:04:33.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.865 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-05-08T06:04:33.865 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-05-08T06:04:33.865 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2300, 2024-05-08T06:04:33.865 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2301 2024-05-08T06:04:33.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.865 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 2300, 2024-05-08T06:04:33.865 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 2300, 2024-05-08T06:04:33.865 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 0, 2024-05-08T06:04:33.865 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-05-08T06:04:33.865 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-05-08T06:04:33.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 1475, 2024-05-08T06:04:33.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 0, 2024-05-08T06:04:33.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 823, 2024-05-08T06:04:33.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 0, 2024-05-08T06:04:33.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 0, 2024-05-08T06:04:33.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-05-08T06:04:33.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-05-08T06:04:33.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-05-08T06:04:33.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-05-08T06:04:33.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 0, 2024-05-08T06:04:33.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-05-08T06:04:33.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-05-08T06:04:33.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-05-08T06:04:33.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 1, 2024-05-08T06:04:33.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 0, 2024-05-08T06:04:33.867 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-05-08T06:04:33.867 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-05-08T06:04:33.867 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-05-08T06:04:33.867 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-05-08T06:04:33.867 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-05-08T06:04:33.867 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-05-08T06:04:33.867 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T06:04:33.867 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-05-08T06:04:33.867 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-05-08T06:04:33.867 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-05-08T06:04:33.867 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-05-08T06:04:33.867 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-05-08T06:04:33.867 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-05-08T06:04:33.867 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-05-08T06:04:33.867 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-05-08T06:04:33.868 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-05-08T06:04:33.868 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-05-08T06:04:33.868 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-05-08T06:04:33.868 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-05-08T06:04:33.868 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-05-08T06:04:33.868 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-05-08T06:04:33.868 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-05-08T06:04:33.868 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-05-08T06:04:33.868 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-05-08T06:04:33.868 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-05-08T06:04:33.868 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-05-08T06:04:33.868 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 84, 2024-05-08T06:04:33.868 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-05-08T06:04:33.868 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 4, 2024-05-08T06:04:33.869 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 6, 2024-05-08T06:04:33.869 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 6, 2024-05-08T06:04:33.869 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-05-08T06:04:33.869 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-05-08T06:04:33.869 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 0, 2024-05-08T06:04:33.869 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 2, 2024-05-08T06:04:33.869 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 0 2024-05-08T06:04:33.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.869 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-05-08T06:04:33.869 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 0, 2024-05-08T06:04:33.869 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-05-08T06:04:33.869 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-05-08T06:04:33.869 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-05-08T06:04:33.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.869 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-05-08T06:04:33.870 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-05-08T06:04:33.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.870 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-05-08T06:04:33.870 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-05-08T06:04:33.870 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T06:04:33.870 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-05-08T06:04:33.870 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2024-05-08T06:04:33.870 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 0, 2024-05-08T06:04:33.870 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2024-05-08T06:04:33.870 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-05-08T06:04:33.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.870 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:33.870 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-08T06:04:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-08T06:04:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-08T06:04:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 2551, 2024-05-08T06:04:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 10839820, 2024-05-08T06:04:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 2551, 2024-05-08T06:04:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-08T06:04:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-08T06:04:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2551, 2024-05-08T06:04:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 10839820, 2024-05-08T06:04:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-08T06:04:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.871 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-05-08T06:04:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-08T06:04:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-08T06:04:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-08T06:04:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-08T06:04:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-08T06:04:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2300, 2024-05-08T06:04:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 9844578, 2024-05-08T06:04:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2300, 2024-05-08T06:04:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 9844578, 2024-05-08T06:04:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-08T06:04:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.873 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.873 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.873 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:33.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.873 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-05-08T06:04:33.873 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-08T06:04:33.873 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-05-08T06:04:33.873 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-08T06:04:33.873 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-08T06:04:33.873 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-08T06:04:33.873 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:33.873 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:33.873 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2300, 2024-05-08T06:04:33.873 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 2300, 2024-05-08T06:04:33.874 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2300, 2024-05-08T06:04:33.874 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2300, 2024-05-08T06:04:33.874 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-08T06:04:33.874 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.874 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.874 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.874 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:33.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.874 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-05-08T06:04:33.874 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-08T06:04:33.874 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-05-08T06:04:33.874 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-08T06:04:33.874 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-08T06:04:33.874 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-08T06:04:33.875 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:33.875 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:33.875 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-08T06:04:33.875 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-08T06:04:33.875 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-05-08T06:04:33.875 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-05-08T06:04:33.875 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-08T06:04:33.875 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.875 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.875 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.875 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:33.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:33.875 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x55d02553c6e0": { 2024-05-08T06:04:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-08T06:04:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-05-08T06:04:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-08T06:04:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-08T06:04:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-08T06:04:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-08T06:04:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-08T06:04:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-05-08T06:04:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-05-08T06:04:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-08T06:04:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:33.877 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:33.877 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:33.877 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:33.877 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:33.877 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-08T06:04:34.614 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-08T06:04:34.614 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-08T06:04:34.614 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-05-08T06:04:34.614 DEBUG:tasks.check_counter:Getting stats from k 2024-05-08T06:04:34.615 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-08T06:04:34.852 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:04:35.643 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:35 smithi150 ceph-mon[34315]: pgmap v719: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:04:35.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:35 smithi112 ceph-mon[30003]: pgmap v719: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:04:35.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:35 smithi162 ceph-mon[34800]: pgmap v719: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:04:36.524 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-08T06:04:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 234, 2024-05-08T06:04:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 231, 2024-05-08T06:04:36.524 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 517680, 2024-05-08T06:04:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 142624, 2024-05-08T06:04:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-05-08T06:04:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.035187838, 2024-05-08T06:04:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.013061588, 2024-05-08T06:04:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.015252327, 2024-05-08T06:04:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.005479907, 2024-05-08T06:04:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 230, 2024-05-08T06:04:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.011432077, 2024-05-08T06:04:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000049704 2024-05-08T06:04:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:36.525 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 517680, 2024-05-08T06:04:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 142624 2024-05-08T06:04:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1, 2024-05-08T06:04:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 177, 2024-05-08T06:04:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 160, 2024-05-08T06:04:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 109648, 2024-05-08T06:04:36.526 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 3, 2024-05-08T06:04:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-05-08T06:04:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.009540079, 2024-05-08T06:04:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.007261473, 2024-05-08T06:04:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000420406, 2024-05-08T06:04:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 175, 2024-05-08T06:04:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.010244247, 2024-05-08T06:04:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000058538 2024-05-08T06:04:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:36.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T06:04:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 109648 2024-05-08T06:04:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 4, 2024-05-08T06:04:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 4, 2024-05-08T06:04:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 2944, 2024-05-08T06:04:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 448, 2024-05-08T06:04:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-05-08T06:04:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.000569837, 2024-05-08T06:04:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000036898, 2024-05-08T06:04:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000495530, 2024-05-08T06:04:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:36.528 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3, 2024-05-08T06:04:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000677707, 2024-05-08T06:04:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000225902 2024-05-08T06:04:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-08T06:04:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T06:04:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-08T06:04:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-05-08T06:04:36.529 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-08T06:04:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-08T06:04:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:36.530 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:36.531 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-08T06:04:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148945, 2024-05-08T06:04:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 29, 2024-05-08T06:04:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:36.532 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:36.533 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-08T06:04:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-08T06:04:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-08T06:04:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-08T06:04:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-08T06:04:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 235, 2024-05-08T06:04:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 484595, 2024-05-08T06:04:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 235, 2024-05-08T06:04:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-08T06:04:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-08T06:04:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 235, 2024-05-08T06:04:36.534 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 484595, 2024-05-08T06:04:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-08T06:04:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-08T06:04:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-08T06:04:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-08T06:04:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:36.535 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-08T06:04:36.535 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-08T06:04:37.128 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-08T06:04:37.128 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-08T06:04:37.128 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-05-08T06:04:37.128 DEBUG:tasks.check_counter:Getting stats from c 2024-05-08T06:04:37.128 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-08T06:04:37.385 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:04:37.505 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:37 smithi162 ceph-mon[34800]: pgmap v720: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T06:04:37.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:37 smithi150 ceph-mon[34315]: pgmap v720: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T06:04:37.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:37 smithi112 ceph-mon[30003]: pgmap v720: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T06:04:39.087 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-05-08T06:04:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 1668, 2024-05-08T06:04:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 1652, 2024-05-08T06:04:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 17793686, 2024-05-08T06:04:39.087 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 591734, 2024-05-08T06:04:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 7, 2024-05-08T06:04:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 3, 2024-05-08T06:04:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.182817013, 2024-05-08T06:04:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.055734040, 2024-05-08T06:04:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.063409238, 2024-05-08T06:04:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.051309359, 2024-05-08T06:04:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 1651, 2024-05-08T06:04:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.072400411, 2024-05-08T06:04:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000043852 2024-05-08T06:04:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.088 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 588352, 2024-05-08T06:04:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 141760 2024-05-08T06:04:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 373, 2024-05-08T06:04:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 546, 2024-05-08T06:04:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 25262484, 2024-05-08T06:04:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 570620, 2024-05-08T06:04:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 7, 2024-05-08T06:04:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 5, 2024-05-08T06:04:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.050138363, 2024-05-08T06:04:39.089 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.014610743, 2024-05-08T06:04:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.023531971, 2024-05-08T06:04:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.008282696, 2024-05-08T06:04:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 543, 2024-05-08T06:04:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.024931152, 2024-05-08T06:04:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000045913 2024-05-08T06:04:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T06:04:39.090 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 452576 2024-05-08T06:04:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 1007, 2024-05-08T06:04:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 1007, 2024-05-08T06:04:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 32621130, 2024-05-08T06:04:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 319409, 2024-05-08T06:04:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 6, 2024-05-08T06:04:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 5, 2024-05-08T06:04:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.098376373, 2024-05-08T06:04:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.028499979, 2024-05-08T06:04:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.037155845, 2024-05-08T06:04:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.025557673, 2024-05-08T06:04:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 1006, 2024-05-08T06:04:39.091 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.034890026, 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000034681 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 1, 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "finisher-MDSRank": { 2024-05-08T06:04:39.092 INFO:teuthology.orchestra.run.smithi162.stdout: "queue_len": 0, 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "complete_latency": { 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 4727, 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.117744162, 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000024908 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "finisher-PurgeQueue": { 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "queue_len": 0, 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "complete_latency": { 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-05-08T06:04:39.093 INFO:teuthology.orchestra.run.smithi162.stdout: "request": 0, 2024-05-08T06:04:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "reply": 0, 2024-05-08T06:04:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "reply_latency": { 2024-05-08T06:04:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "slow_reply": 0, 2024-05-08T06:04:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "forward": 0, 2024-05-08T06:04:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_fetch_complete": 0, 2024-05-08T06:04:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_fetch_keys": 0, 2024-05-08T06:04:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_commit": 0, 2024-05-08T06:04:39.094 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_split": 0, 2024-05-08T06:04:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_merge": 0, 2024-05-08T06:04:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes": 574, 2024-05-08T06:04:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_top": 0, 2024-05-08T06:04:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_bottom": 574, 2024-05-08T06:04:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_pin_tail": 0, 2024-05-08T06:04:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_pinned": 574, 2024-05-08T06:04:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_expired": 0, 2024-05-08T06:04:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_with_caps": 0, 2024-05-08T06:04:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": 0, 2024-05-08T06:04:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "subtrees": 2, 2024-05-08T06:04:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse": 0, 2024-05-08T06:04:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_hit": 0, 2024-05-08T06:04:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_forward": 0, 2024-05-08T06:04:39.095 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_discover": 0, 2024-05-08T06:04:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_dir_fetch": 0, 2024-05-08T06:04:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_remote_ino": 0, 2024-05-08T06:04:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_lock": 0, 2024-05-08T06:04:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "load_cent": 0, 2024-05-08T06:04:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "q": 0, 2024-05-08T06:04:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "exported": 0, 2024-05-08T06:04:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "exported_inodes": 0, 2024-05-08T06:04:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "imported": 0, 2024-05-08T06:04:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "imported_inodes": 0, 2024-05-08T06:04:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_dir_fetch": 0, 2024-05-08T06:04:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_backtrace_fetch": 0, 2024-05-08T06:04:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_peer_discover": 0, 2024-05-08T06:04:39.096 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rfiles": 4, 2024-05-08T06:04:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rbytes": 234, 2024-05-08T06:04:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rsnaps": 0, 2024-05-08T06:04:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T06:04:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_set_tag": 0, 2024-05-08T06:04:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T06:04:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_inotable_repaired": 0, 2024-05-08T06:04:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dir_inodes": 0, 2024-05-08T06:04:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T06:04:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T06:04:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_file_inodes": 0, 2024-05-08T06:04:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_inode_file_caps": 0, 2024-05-08T06:04:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T06:04:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_grant": 0, 2024-05-08T06:04:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T06:04:39.097 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T06:04:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T06:04:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_caps": 0, 2024-05-08T06:04:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_caps_dirty": 0, 2024-05-08T06:04:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_cap_release": 0, 2024-05-08T06:04:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "process_request_cap_release": 0 2024-05-08T06:04:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cache": { 2024-05-08T06:04:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_update": 0, 2024-05-08T06:04:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_update_receipt": 0, 2024-05-08T06:04:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_try_discover": 0, 2024-05-08T06:04:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_send_discover": 0, 2024-05-08T06:04:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_handle_discover": 0, 2024-05-08T06:04:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays": 0, 2024-05-08T06:04:39.098 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays_delayed": 0, 2024-05-08T06:04:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays_enqueuing": 0, 2024-05-08T06:04:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_created": 0, 2024-05-08T06:04:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_enqueued": 0, 2024-05-08T06:04:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_reintegrated": 0, 2024-05-08T06:04:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_migrated": 0, 2024-05-08T06:04:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_processing": 0, 2024-05-08T06:04:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_enqueued": 0, 2024-05-08T06:04:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_prioritized": 0, 2024-05-08T06:04:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_started": 0, 2024-05-08T06:04:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_completed": 0, 2024-05-08T06:04:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_quiesce_path": 0, 2024-05-08T06:04:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_quiesce_inode": 0, 2024-05-08T06:04:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T06:04:39.099 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_exportdir": 0, 2024-05-08T06:04:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_flush": 0, 2024-05-08T06:04:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_fragmentdir": 0, 2024-05-08T06:04:39.100 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_fragstats": 0, 2024-05-08T06:04:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_inodestats": 0 2024-05-08T06:04:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_log": { 2024-05-08T06:04:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "evlrg": 0, 2024-05-08T06:04:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "evadd": 0, 2024-05-08T06:04:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "evex": 0, 2024-05-08T06:04:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "evtrm": 0, 2024-05-08T06:04:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "ev": 34338, 2024-05-08T06:04:39.101 INFO:teuthology.orchestra.run.smithi162.stdout: "evexg": 0, 2024-05-08T06:04:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "evexd": 0, 2024-05-08T06:04:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "segadd": 0, 2024-05-08T06:04:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "segex": 0, 2024-05-08T06:04:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "segtrm": 0, 2024-05-08T06:04:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "seg": 39, 2024-05-08T06:04:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "segmjr": 4, 2024-05-08T06:04:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "segexg": 0, 2024-05-08T06:04:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "segexd": 0, 2024-05-08T06:04:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "expos": 4194304, 2024-05-08T06:04:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "wrpos": 78503147, 2024-05-08T06:04:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "rdpos": 78502548, 2024-05-08T06:04:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "jlat": { 2024-05-08T06:04:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.102 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "replayed": 34338 2024-05-08T06:04:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_mem": { 2024-05-08T06:04:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "ino": 17, 2024-05-08T06:04:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "ino+": 5273, 2024-05-08T06:04:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "ino-": 5256, 2024-05-08T06:04:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "dir": 12, 2024-05-08T06:04:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "dir+": 279, 2024-05-08T06:04:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "dir-": 267, 2024-05-08T06:04:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "dn": 574, 2024-05-08T06:04:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "dn+": 5742, 2024-05-08T06:04:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "dn-": 5168, 2024-05-08T06:04:39.103 INFO:teuthology.orchestra.run.smithi162.stdout: "cap": 0, 2024-05-08T06:04:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "cap+": 0, 2024-05-08T06:04:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "cap-": 0, 2024-05-08T06:04:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "rss": 61196, 2024-05-08T06:04:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "heap": 215416 2024-05-08T06:04:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_server": { 2024-05-08T06:04:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "dispatch_client_request": 0, 2024-05-08T06:04:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "dispatch_server_request": 0, 2024-05-08T06:04:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_request": 0, 2024-05-08T06:04:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_session": 0, 2024-05-08T06:04:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_peer_request": 0, 2024-05-08T06:04:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "req_create_latency": { 2024-05-08T06:04:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.104 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getattr_latency": { 2024-05-08T06:04:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getfilelock_latency": { 2024-05-08T06:04:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.105 INFO:teuthology.orchestra.run.smithi162.stdout: "req_link_latency": { 2024-05-08T06:04:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookup_latency": { 2024-05-08T06:04:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookuphash_latency": { 2024-05-08T06:04:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.106 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupino_latency": { 2024-05-08T06:04:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupname_latency": { 2024-05-08T06:04:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupparent_latency": { 2024-05-08T06:04:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.107 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupsnap_latency": { 2024-05-08T06:04:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lssnap_latency": { 2024-05-08T06:04:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mkdir_latency": { 2024-05-08T06:04:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mknod_latency": { 2024-05-08T06:04:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.109 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mksnap_latency": { 2024-05-08T06:04:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "req_open_latency": { 2024-05-08T06:04:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "req_readdir_latency": { 2024-05-08T06:04:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.110 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rename_latency": { 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "req_renamesnap_latency": { 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "req_snapdiff_latency": { 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmdir_latency": { 2024-05-08T06:04:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmsnap_latency": { 2024-05-08T06:04:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmxattr_latency": { 2024-05-08T06:04:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.112 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setattr_latency": { 2024-05-08T06:04:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setdirlayout_latency": { 2024-05-08T06:04:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setfilelock_latency": { 2024-05-08T06:04:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.113 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setlayout_latency": { 2024-05-08T06:04:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setxattr_latency": { 2024-05-08T06:04:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "req_symlink_latency": { 2024-05-08T06:04:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.114 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "req_unlink_latency": { 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "cap_revoke_eviction": 0, 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "cap_acquisition_throttle": 0, 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getvxattr_latency": { 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_sessions": { 2024-05-08T06:04:39.115 INFO:teuthology.orchestra.run.smithi162.stdout: "session_count": 1, 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "session_add": 2, 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "session_remove": 1, 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "sessions_open": 1, 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "sessions_stale": 0, 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "total_load": 0, 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "average_load": 0, 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "avg_session_uptime": 2272, 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "mdthresh_evicted": 0 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:39.116 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:39.117 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:39.118 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 231329, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 74, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 7728, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 163, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2024-05-08T06:04:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 457466, 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 2519, 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "objecter": { 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "op_active": 0, 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "op_laggy": 0, 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "op_send": 2789, 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "op_send_bytes": 26, 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "op_resend": 4, 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "op_reply": 2789, 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "op_latency": { 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 2789, 2024-05-08T06:04:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 1.770962842, 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000634981 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "op_inflight": 0, 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "oplen_avg": { 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 2789, 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 2790 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "op": 2789, 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "op_r": 2789, 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "op_w": 0, 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "op_rmw": 0, 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "op_pg": 0, 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_stat": 1580, 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_create": 0, 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_read": 1206, 2024-05-08T06:04:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_write": 0, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_writefull": 0, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_writesame": 0, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_append": 0, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_zero": 0, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_truncate": 0, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_delete": 0, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_mapext": 0, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_sparse_read": 0, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_clonerange": 0, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_getxattr": 2, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_setxattr": 0, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_cmpxattr": 0, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_rmxattr": 0, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_resetxattrs": 0, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_call": 0, 2024-05-08T06:04:39.122 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_watch": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_notify": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_pgls": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_pgls_filter": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_other": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_active": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_send": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_resend": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_ping": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_active": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_send": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_resend": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_active": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_send": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_resend": 0, 2024-05-08T06:04:39.123 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_active": 0, 2024-05-08T06:04:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_send": 0, 2024-05-08T06:04:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_resend": 0, 2024-05-08T06:04:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "command_active": 0, 2024-05-08T06:04:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "command_send": 0, 2024-05-08T06:04:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "command_resend": 0, 2024-05-08T06:04:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "map_epoch": 84, 2024-05-08T06:04:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "map_full": 0, 2024-05-08T06:04:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "map_inc": 4, 2024-05-08T06:04:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_sessions": 10, 2024-05-08T06:04:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_session_open": 10, 2024-05-08T06:04:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_session_close": 0, 2024-05-08T06:04:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_laggy": 0, 2024-05-08T06:04:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_wr": 0, 2024-05-08T06:04:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_rd": 2, 2024-05-08T06:04:39.124 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_del": 0 2024-05-08T06:04:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "oft": { 2024-05-08T06:04:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_objs": 0, 2024-05-08T06:04:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_kv_pairs": 0, 2024-05-08T06:04:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_updates": 0, 2024-05-08T06:04:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_removes": 0 2024-05-08T06:04:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "osd-slow-ops": { 2024-05-08T06:04:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "slow_ops_count": 0 2024-05-08T06:04:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "purge_queue": { 2024-05-08T06:04:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_ops": 0, 2024-05-08T06:04:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T06:04:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing": 0, 2024-05-08T06:04:39.125 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_high_water": 0, 2024-05-08T06:04:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executed_ops": 0, 2024-05-08T06:04:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executed": 0, 2024-05-08T06:04:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_item_in_journal": 0 2024-05-08T06:04:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-05-08T06:04:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 3044, 2024-05-08T06:04:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 75387395, 2024-05-08T06:04:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 3044, 2024-05-08T06:04:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-05-08T06:04:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-05-08T06:04:39.126 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 3044, 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 75387395, 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-objecter_bytes": { 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 2789, 2024-05-08T06:04:39.127 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 74312542, 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 2789, 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 74312542, 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-objecter_ops": { 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 1024, 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:39.128 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 2789, 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 2789, 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 2789, 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 2789, 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-write_buf_throttle": { 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 3758096384, 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-05-08T06:04:39.129 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 0, 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 0, 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-write_buf_throttle-0x563419c7e6e0": { 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 3758096384, 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:39.130 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-05-08T06:04:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-05-08T06:04:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-05-08T06:04:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-05-08T06:04:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 0, 2024-05-08T06:04:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 0, 2024-05-08T06:04:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:39.131 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:39.131 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-05-08T06:04:39.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:39 smithi150 ceph-mon[34315]: pgmap v721: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:39.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:39 smithi112 ceph-mon[30003]: pgmap v721: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:39.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:39 smithi162 ceph-mon[34800]: pgmap v721: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:39.875 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-08T06:04:39.876 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-08T06:04:39.876 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-05-08T06:04:39.876 DEBUG:tasks.check_counter:Getting stats from f 2024-05-08T06:04:39.876 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-08T06:04:40.128 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:04:41.513 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:41 smithi162 ceph-mon[34800]: pgmap v722: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:04:41.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:41 smithi150 ceph-mon[34315]: pgmap v722: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:04:41.769 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-05-08T06:04:41.769 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:41.769 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 1045, 2024-05-08T06:04:41.769 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 1029, 2024-05-08T06:04:41.769 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 854576, 2024-05-08T06:04:41.769 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 394096, 2024-05-08T06:04:41.769 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 5, 2024-05-08T06:04:41.769 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:41.769 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.128889112, 2024-05-08T06:04:41.769 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.041299248, 2024-05-08T06:04:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.041895784, 2024-05-08T06:04:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.036523548, 2024-05-08T06:04:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 1028, 2024-05-08T06:04:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.051828044, 2024-05-08T06:04:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000050416 2024-05-08T06:04:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 588464, 2024-05-08T06:04:41.770 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 142240 2024-05-08T06:04:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 19, 2024-05-08T06:04:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 191, 2024-05-08T06:04:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 4201028, 2024-05-08T06:04:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 380482, 2024-05-08T06:04:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 5, 2024-05-08T06:04:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 4, 2024-05-08T06:04:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.012867933, 2024-05-08T06:04:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.007138945, 2024-05-08T06:04:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.003593434, 2024-05-08T06:04:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000409267, 2024-05-08T06:04:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 190, 2024-05-08T06:04:41.771 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.013041158, 2024-05-08T06:04:41.772 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000068637 2024-05-08T06:04:41.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.772 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:41.772 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.772 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.772 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.772 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-08T06:04:41.772 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 375968 2024-05-08T06:04:41.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.772 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:41.772 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 1437, 2024-05-08T06:04:41.772 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 1439, 2024-05-08T06:04:41.772 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 14860084, 2024-05-08T06:04:41.772 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 456973, 2024-05-08T06:04:41.773 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 5, 2024-05-08T06:04:41.773 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 4, 2024-05-08T06:04:41.773 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.104518578, 2024-05-08T06:04:41.773 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.035288521, 2024-05-08T06:04:41.773 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.030252625, 2024-05-08T06:04:41.773 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.031764074, 2024-05-08T06:04:41.773 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:41.773 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 1437, 2024-05-08T06:04:41.773 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.050988814, 2024-05-08T06:04:41.773 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000035482 2024-05-08T06:04:41.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.773 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:41.773 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.773 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.774 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.774 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:04:41.774 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:04:41.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.774 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2024-05-08T06:04:41.774 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 1, 2024-05-08T06:04:41.774 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2024-05-08T06:04:41.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.774 INFO:teuthology.orchestra.run.smithi162.stdout: "finisher-MDSRank": { 2024-05-08T06:04:41.774 INFO:teuthology.orchestra.run.smithi162.stdout: "queue_len": 0, 2024-05-08T06:04:41.774 INFO:teuthology.orchestra.run.smithi162.stdout: "complete_latency": { 2024-05-08T06:04:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 3879, 2024-05-08T06:04:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.104755175, 2024-05-08T06:04:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000027005 2024-05-08T06:04:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: "finisher-PurgeQueue": { 2024-05-08T06:04:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: "queue_len": 0, 2024-05-08T06:04:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: "complete_latency": { 2024-05-08T06:04:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-05-08T06:04:41.775 INFO:teuthology.orchestra.run.smithi162.stdout: "request": 0, 2024-05-08T06:04:41.776 INFO:teuthology.orchestra.run.smithi162.stdout: "reply": 0, 2024-05-08T06:04:41.776 INFO:teuthology.orchestra.run.smithi162.stdout: "reply_latency": { 2024-05-08T06:04:41.776 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.776 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.776 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.776 INFO:teuthology.orchestra.run.smithi162.stdout: "slow_reply": 0, 2024-05-08T06:04:41.776 INFO:teuthology.orchestra.run.smithi162.stdout: "forward": 0, 2024-05-08T06:04:41.776 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_fetch_complete": 0, 2024-05-08T06:04:41.776 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_fetch_keys": 0, 2024-05-08T06:04:41.776 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_commit": 0, 2024-05-08T06:04:41.776 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_split": 0, 2024-05-08T06:04:41.776 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_merge": 0, 2024-05-08T06:04:41.776 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes": 3164, 2024-05-08T06:04:41.777 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_top": 99, 2024-05-08T06:04:41.777 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_bottom": 3065, 2024-05-08T06:04:41.777 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_pin_tail": 0, 2024-05-08T06:04:41.777 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_pinned": 3022, 2024-05-08T06:04:41.777 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_expired": 0, 2024-05-08T06:04:41.777 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_with_caps": 0, 2024-05-08T06:04:41.777 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": 0, 2024-05-08T06:04:41.777 INFO:teuthology.orchestra.run.smithi162.stdout: "subtrees": 1, 2024-05-08T06:04:41.777 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse": 0, 2024-05-08T06:04:41.777 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_hit": 0, 2024-05-08T06:04:41.777 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_forward": 0, 2024-05-08T06:04:41.777 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_discover": 0, 2024-05-08T06:04:41.777 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_dir_fetch": 0, 2024-05-08T06:04:41.777 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_remote_ino": 0, 2024-05-08T06:04:41.777 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_lock": 0, 2024-05-08T06:04:41.778 INFO:teuthology.orchestra.run.smithi162.stdout: "load_cent": 0, 2024-05-08T06:04:41.778 INFO:teuthology.orchestra.run.smithi162.stdout: "q": 0, 2024-05-08T06:04:41.778 INFO:teuthology.orchestra.run.smithi162.stdout: "exported": 0, 2024-05-08T06:04:41.778 INFO:teuthology.orchestra.run.smithi162.stdout: "exported_inodes": 0, 2024-05-08T06:04:41.778 INFO:teuthology.orchestra.run.smithi162.stdout: "imported": 0, 2024-05-08T06:04:41.778 INFO:teuthology.orchestra.run.smithi162.stdout: "imported_inodes": 0, 2024-05-08T06:04:41.778 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_dir_fetch": 0, 2024-05-08T06:04:41.778 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_backtrace_fetch": 0, 2024-05-08T06:04:41.778 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_peer_discover": 0, 2024-05-08T06:04:41.778 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rfiles": 0, 2024-05-08T06:04:41.778 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rbytes": 0, 2024-05-08T06:04:41.778 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rsnaps": 0, 2024-05-08T06:04:41.778 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T06:04:41.778 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_set_tag": 0, 2024-05-08T06:04:41.779 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T06:04:41.779 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_inotable_repaired": 0, 2024-05-08T06:04:41.779 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dir_inodes": 0, 2024-05-08T06:04:41.779 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T06:04:41.779 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T06:04:41.779 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_file_inodes": 0, 2024-05-08T06:04:41.779 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_inode_file_caps": 0, 2024-05-08T06:04:41.779 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T06:04:41.779 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_grant": 0, 2024-05-08T06:04:41.779 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T06:04:41.779 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T06:04:41.779 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T06:04:41.779 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_caps": 0, 2024-05-08T06:04:41.779 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_caps_dirty": 0, 2024-05-08T06:04:41.779 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_cap_release": 0, 2024-05-08T06:04:41.780 INFO:teuthology.orchestra.run.smithi162.stdout: "process_request_cap_release": 0 2024-05-08T06:04:41.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.780 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cache": { 2024-05-08T06:04:41.780 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_update": 0, 2024-05-08T06:04:41.780 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_update_receipt": 0, 2024-05-08T06:04:41.780 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_try_discover": 0, 2024-05-08T06:04:41.780 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_send_discover": 0, 2024-05-08T06:04:41.780 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_handle_discover": 0, 2024-05-08T06:04:41.780 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays": 0, 2024-05-08T06:04:41.780 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays_delayed": 0, 2024-05-08T06:04:41.780 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays_enqueuing": 0, 2024-05-08T06:04:41.780 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_created": 0, 2024-05-08T06:04:41.780 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_enqueued": 0, 2024-05-08T06:04:41.780 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_reintegrated": 0, 2024-05-08T06:04:41.780 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_migrated": 0, 2024-05-08T06:04:41.781 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_processing": 0, 2024-05-08T06:04:41.781 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_enqueued": 0, 2024-05-08T06:04:41.781 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_prioritized": 0, 2024-05-08T06:04:41.781 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_started": 0, 2024-05-08T06:04:41.781 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_completed": 0, 2024-05-08T06:04:41.781 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_quiesce_path": 0, 2024-05-08T06:04:41.781 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_quiesce_inode": 0, 2024-05-08T06:04:41.781 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T06:04:41.781 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_exportdir": 0, 2024-05-08T06:04:41.781 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_flush": 0, 2024-05-08T06:04:41.781 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_fragmentdir": 0, 2024-05-08T06:04:41.781 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_fragstats": 0, 2024-05-08T06:04:41.781 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_inodestats": 0 2024-05-08T06:04:41.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.781 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_log": { 2024-05-08T06:04:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "evlrg": 0, 2024-05-08T06:04:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "evadd": 0, 2024-05-08T06:04:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "evex": 0, 2024-05-08T06:04:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "evtrm": 0, 2024-05-08T06:04:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "ev": 8787, 2024-05-08T06:04:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "evexg": 0, 2024-05-08T06:04:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "evexd": 0, 2024-05-08T06:04:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "segadd": 0, 2024-05-08T06:04:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "segex": 0, 2024-05-08T06:04:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "segtrm": 0, 2024-05-08T06:04:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "seg": 12, 2024-05-08T06:04:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "segmjr": 2, 2024-05-08T06:04:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "segexg": 0, 2024-05-08T06:04:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "segexd": 0, 2024-05-08T06:04:41.782 INFO:teuthology.orchestra.run.smithi162.stdout: "expos": 4194304, 2024-05-08T06:04:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "wrpos": 22881505, 2024-05-08T06:04:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "rdpos": 22879854, 2024-05-08T06:04:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "jlat": { 2024-05-08T06:04:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "replayed": 8787 2024-05-08T06:04:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_mem": { 2024-05-08T06:04:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "ino": 16, 2024-05-08T06:04:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "ino+": 3704, 2024-05-08T06:04:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "ino-": 3688, 2024-05-08T06:04:41.783 INFO:teuthology.orchestra.run.smithi162.stdout: "dir": 28, 2024-05-08T06:04:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "dir+": 262, 2024-05-08T06:04:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "dir-": 234, 2024-05-08T06:04:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "dn": 3164, 2024-05-08T06:04:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "dn+": 7105, 2024-05-08T06:04:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "dn-": 3941, 2024-05-08T06:04:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "cap": 0, 2024-05-08T06:04:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "cap+": 0, 2024-05-08T06:04:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "cap-": 0, 2024-05-08T06:04:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "rss": 60012, 2024-05-08T06:04:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "heap": 215416 2024-05-08T06:04:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_server": { 2024-05-08T06:04:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "dispatch_client_request": 0, 2024-05-08T06:04:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "dispatch_server_request": 0, 2024-05-08T06:04:41.784 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_request": 0, 2024-05-08T06:04:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_session": 0, 2024-05-08T06:04:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_peer_request": 0, 2024-05-08T06:04:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "req_create_latency": { 2024-05-08T06:04:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getattr_latency": { 2024-05-08T06:04:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getfilelock_latency": { 2024-05-08T06:04:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.785 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: "req_link_latency": { 2024-05-08T06:04:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookup_latency": { 2024-05-08T06:04:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookuphash_latency": { 2024-05-08T06:04:41.786 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupino_latency": { 2024-05-08T06:04:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupname_latency": { 2024-05-08T06:04:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.787 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupparent_latency": { 2024-05-08T06:04:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupsnap_latency": { 2024-05-08T06:04:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lssnap_latency": { 2024-05-08T06:04:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.789 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mkdir_latency": { 2024-05-08T06:04:41.789 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.789 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.789 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.789 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mknod_latency": { 2024-05-08T06:04:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mksnap_latency": { 2024-05-08T06:04:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: "req_open_latency": { 2024-05-08T06:04:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.791 INFO:teuthology.orchestra.run.smithi162.stdout: "req_readdir_latency": { 2024-05-08T06:04:41.791 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.791 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.791 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.791 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rename_latency": { 2024-05-08T06:04:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: "req_renamesnap_latency": { 2024-05-08T06:04:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: "req_snapdiff_latency": { 2024-05-08T06:04:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.792 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmdir_latency": { 2024-05-08T06:04:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmsnap_latency": { 2024-05-08T06:04:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmxattr_latency": { 2024-05-08T06:04:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.793 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setattr_latency": { 2024-05-08T06:04:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setdirlayout_latency": { 2024-05-08T06:04:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setfilelock_latency": { 2024-05-08T06:04:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.794 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setlayout_latency": { 2024-05-08T06:04:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setxattr_latency": { 2024-05-08T06:04:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: "req_symlink_latency": { 2024-05-08T06:04:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.795 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: "req_unlink_latency": { 2024-05-08T06:04:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: "cap_revoke_eviction": 0, 2024-05-08T06:04:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: "cap_acquisition_throttle": 0, 2024-05-08T06:04:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getvxattr_latency": { 2024-05-08T06:04:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:41.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_sessions": { 2024-05-08T06:04:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "session_count": 1, 2024-05-08T06:04:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "session_add": 1, 2024-05-08T06:04:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "session_remove": 0, 2024-05-08T06:04:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "sessions_open": 1, 2024-05-08T06:04:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "sessions_stale": 0, 2024-05-08T06:04:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "total_load": 0, 2024-05-08T06:04:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "average_load": 0, 2024-05-08T06:04:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "avg_session_uptime": 2274, 2024-05-08T06:04:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "mdthresh_evicted": 0 2024-05-08T06:04:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2024-05-08T06:04:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:41.797 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:41.798 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:41.799 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 197629, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 59, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 7728, 2024-05-08T06:04:41.800 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 163, 2024-05-08T06:04:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2024-05-08T06:04:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 2302966, 2024-05-08T06:04:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 16173, 2024-05-08T06:04:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2024-05-08T06:04:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2024-05-08T06:04:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "objecter": { 2024-05-08T06:04:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "op_active": 0, 2024-05-08T06:04:41.801 INFO:teuthology.orchestra.run.smithi162.stdout: "op_laggy": 0, 2024-05-08T06:04:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "op_send": 2243, 2024-05-08T06:04:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "op_send_bytes": 21, 2024-05-08T06:04:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "op_resend": 3, 2024-05-08T06:04:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "op_reply": 2243, 2024-05-08T06:04:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "op_latency": { 2024-05-08T06:04:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 2243, 2024-05-08T06:04:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 1.147975964, 2024-05-08T06:04:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000511803 2024-05-08T06:04:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "op_inflight": 0, 2024-05-08T06:04:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "oplen_avg": { 2024-05-08T06:04:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 2243, 2024-05-08T06:04:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 2244 2024-05-08T06:04:41.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "op": 2243, 2024-05-08T06:04:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "op_r": 2243, 2024-05-08T06:04:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "op_w": 0, 2024-05-08T06:04:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "op_rmw": 0, 2024-05-08T06:04:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "op_pg": 0, 2024-05-08T06:04:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_stat": 1384, 2024-05-08T06:04:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_create": 0, 2024-05-08T06:04:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_read": 857, 2024-05-08T06:04:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_write": 0, 2024-05-08T06:04:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_writefull": 0, 2024-05-08T06:04:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_writesame": 0, 2024-05-08T06:04:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_append": 0, 2024-05-08T06:04:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_zero": 0, 2024-05-08T06:04:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_truncate": 0, 2024-05-08T06:04:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_delete": 0, 2024-05-08T06:04:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_mapext": 0, 2024-05-08T06:04:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_sparse_read": 0, 2024-05-08T06:04:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_clonerange": 0, 2024-05-08T06:04:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_getxattr": 1, 2024-05-08T06:04:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_setxattr": 0, 2024-05-08T06:04:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_cmpxattr": 0, 2024-05-08T06:04:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_rmxattr": 0, 2024-05-08T06:04:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_resetxattrs": 0, 2024-05-08T06:04:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_call": 0, 2024-05-08T06:04:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_watch": 0, 2024-05-08T06:04:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_notify": 0, 2024-05-08T06:04:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T06:04:41.804 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_pgls": 0, 2024-05-08T06:04:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_pgls_filter": 0, 2024-05-08T06:04:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_other": 0, 2024-05-08T06:04:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_active": 0, 2024-05-08T06:04:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_send": 0, 2024-05-08T06:04:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_resend": 0, 2024-05-08T06:04:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_ping": 0, 2024-05-08T06:04:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_active": 0, 2024-05-08T06:04:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_send": 0, 2024-05-08T06:04:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_resend": 0, 2024-05-08T06:04:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_active": 0, 2024-05-08T06:04:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_send": 0, 2024-05-08T06:04:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_resend": 0, 2024-05-08T06:04:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_active": 0, 2024-05-08T06:04:41.805 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_send": 0, 2024-05-08T06:04:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_resend": 0, 2024-05-08T06:04:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: "command_active": 0, 2024-05-08T06:04:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: "command_send": 0, 2024-05-08T06:04:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: "command_resend": 0, 2024-05-08T06:04:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: "map_epoch": 84, 2024-05-08T06:04:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: "map_full": 0, 2024-05-08T06:04:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: "map_inc": 4, 2024-05-08T06:04:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_sessions": 6, 2024-05-08T06:04:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_session_open": 6, 2024-05-08T06:04:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_session_close": 0, 2024-05-08T06:04:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_laggy": 0, 2024-05-08T06:04:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_wr": 0, 2024-05-08T06:04:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_rd": 2, 2024-05-08T06:04:41.806 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_del": 0 2024-05-08T06:04:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "oft": { 2024-05-08T06:04:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_objs": 0, 2024-05-08T06:04:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_kv_pairs": 0, 2024-05-08T06:04:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_updates": 0, 2024-05-08T06:04:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_removes": 0 2024-05-08T06:04:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "osd-slow-ops": { 2024-05-08T06:04:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "slow_ops_count": 0 2024-05-08T06:04:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "purge_queue": { 2024-05-08T06:04:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_ops": 0, 2024-05-08T06:04:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T06:04:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing": 0, 2024-05-08T06:04:41.807 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_high_water": 0, 2024-05-08T06:04:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executed_ops": 0, 2024-05-08T06:04:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executed": 0, 2024-05-08T06:04:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_item_in_journal": 0 2024-05-08T06:04:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-05-08T06:04:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 2497, 2024-05-08T06:04:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 19675050, 2024-05-08T06:04:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 2497, 2024-05-08T06:04:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-05-08T06:04:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-05-08T06:04:41.808 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 2497, 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 19675050, 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-objecter_bytes": { 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:41.809 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 2243, 2024-05-08T06:04:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 18687206, 2024-05-08T06:04:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 2243, 2024-05-08T06:04:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 18687206, 2024-05-08T06:04:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-objecter_ops": { 2024-05-08T06:04:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 1024, 2024-05-08T06:04:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:41.810 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-05-08T06:04:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-05-08T06:04:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 2243, 2024-05-08T06:04:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 2243, 2024-05-08T06:04:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 2243, 2024-05-08T06:04:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 2243, 2024-05-08T06:04:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-write_buf_throttle": { 2024-05-08T06:04:41.811 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 3758096384, 2024-05-08T06:04:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-05-08T06:04:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-05-08T06:04:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-05-08T06:04:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-05-08T06:04:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 0, 2024-05-08T06:04:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 0, 2024-05-08T06:04:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.812 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-write_buf_throttle-0x563c5d1d26e0": { 2024-05-08T06:04:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 3758096384, 2024-05-08T06:04:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-05-08T06:04:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-05-08T06:04:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-05-08T06:04:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-05-08T06:04:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 0, 2024-05-08T06:04:41.813 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 0, 2024-05-08T06:04:41.814 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:41.814 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:41.814 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:41.814 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:41.814 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:41.814 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:41.814 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-05-08T06:04:41.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:41 smithi112 ceph-mon[30003]: pgmap v722: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:04:42.634 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-08T06:04:42.634 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-08T06:04:42.634 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-05-08T06:04:42.635 DEBUG:tasks.check_counter:Getting stats from i 2024-05-08T06:04:42.635 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-08T06:04:42.875 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:04:43.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:43 smithi162 ceph-mon[34800]: pgmap v723: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:43.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:43 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:43.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:43 smithi150 ceph-mon[34315]: pgmap v723: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:43.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:43 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:43.815 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:43 smithi112 ceph-mon[30003]: pgmap v723: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:43.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:43 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:44.563 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-05-08T06:04:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 4446, 2024-05-08T06:04:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 4659, 2024-05-08T06:04:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 3187330, 2024-05-08T06:04:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 5340870, 2024-05-08T06:04:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 22, 2024-05-08T06:04:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 7, 2024-05-08T06:04:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.314366231, 2024-05-08T06:04:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.110602098, 2024-05-08T06:04:44.563 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.139376315, 2024-05-08T06:04:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.066561444, 2024-05-08T06:04:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 4662, 2024-05-08T06:04:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.170081006, 2024-05-08T06:04:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000036482 2024-05-08T06:04:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 2919, 2024-05-08T06:04:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000621329, 2024-05-08T06:04:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000212 2024-05-08T06:04:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 603200, 2024-05-08T06:04:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 152896 2024-05-08T06:04:44.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 8634, 2024-05-08T06:04:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 10794, 2024-05-08T06:04:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 3171740, 2024-05-08T06:04:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 6395619, 2024-05-08T06:04:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 9, 2024-05-08T06:04:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 5, 2024-05-08T06:04:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.533589797, 2024-05-08T06:04:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.217806345, 2024-05-08T06:04:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.192852032, 2024-05-08T06:04:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.111383493, 2024-05-08T06:04:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 10793, 2024-05-08T06:04:44.565 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.299946125, 2024-05-08T06:04:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000027790 2024-05-08T06:04:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 8024, 2024-05-08T06:04:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.001129153, 2024-05-08T06:04:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000140 2024-05-08T06:04:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T06:04:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 456352 2024-05-08T06:04:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 11398, 2024-05-08T06:04:44.566 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 17055, 2024-05-08T06:04:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 3510862, 2024-05-08T06:04:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 10852501, 2024-05-08T06:04:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 21, 2024-05-08T06:04:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 8, 2024-05-08T06:04:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.725481316, 2024-05-08T06:04:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.304677232, 2024-05-08T06:04:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.293404990, 2024-05-08T06:04:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.128245065, 2024-05-08T06:04:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 17048, 2024-05-08T06:04:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.429861934, 2024-05-08T06:04:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000025214 2024-05-08T06:04:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.567 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 11282, 2024-05-08T06:04:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.001937124, 2024-05-08T06:04:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000171 2024-05-08T06:04:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 3872, 2024-05-08T06:04:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 1184 2024-05-08T06:04:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2024-05-08T06:04:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 1, 2024-05-08T06:04:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2024-05-08T06:04:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "finisher-MDSRank": { 2024-05-08T06:04:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "queue_len": 0, 2024-05-08T06:04:44.568 INFO:teuthology.orchestra.run.smithi162.stdout: "complete_latency": { 2024-05-08T06:04:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 10098, 2024-05-08T06:04:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 3.657449280, 2024-05-08T06:04:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000362195 2024-05-08T06:04:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "finisher-PurgeQueue": { 2024-05-08T06:04:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "queue_len": 0, 2024-05-08T06:04:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "complete_latency": { 2024-05-08T06:04:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 3587, 2024-05-08T06:04:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.356806094, 2024-05-08T06:04:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000099472 2024-05-08T06:04:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.569 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-05-08T06:04:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "request": 4356, 2024-05-08T06:04:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "reply": 4352, 2024-05-08T06:04:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "reply_latency": { 2024-05-08T06:04:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 4352, 2024-05-08T06:04:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 4.637615113, 2024-05-08T06:04:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.001065628 2024-05-08T06:04:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "slow_reply": 0, 2024-05-08T06:04:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "forward": 4, 2024-05-08T06:04:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_fetch_complete": 0, 2024-05-08T06:04:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_fetch_keys": 0, 2024-05-08T06:04:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_commit": 19, 2024-05-08T06:04:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_split": 1, 2024-05-08T06:04:44.570 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_merge": 0, 2024-05-08T06:04:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes": 16, 2024-05-08T06:04:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_top": 1, 2024-05-08T06:04:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_bottom": 2, 2024-05-08T06:04:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_pin_tail": 13, 2024-05-08T06:04:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_pinned": 13, 2024-05-08T06:04:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_expired": 101, 2024-05-08T06:04:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_with_caps": 0, 2024-05-08T06:04:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": 0, 2024-05-08T06:04:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "subtrees": 4, 2024-05-08T06:04:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse": 8560, 2024-05-08T06:04:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_hit": 6363, 2024-05-08T06:04:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_forward": 4, 2024-05-08T06:04:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_discover": 14, 2024-05-08T06:04:44.571 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_dir_fetch": 0, 2024-05-08T06:04:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_remote_ino": 0, 2024-05-08T06:04:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_lock": 0, 2024-05-08T06:04:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "load_cent": 0, 2024-05-08T06:04:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "q": 0, 2024-05-08T06:04:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "exported": 69, 2024-05-08T06:04:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "exported_inodes": 68, 2024-05-08T06:04:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "imported": 62, 2024-05-08T06:04:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "imported_inodes": 2020, 2024-05-08T06:04:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_dir_fetch": 0, 2024-05-08T06:04:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_backtrace_fetch": 12333, 2024-05-08T06:04:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_peer_discover": 0, 2024-05-08T06:04:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rfiles": 4567, 2024-05-08T06:04:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rbytes": 87759380, 2024-05-08T06:04:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rsnaps": 0, 2024-05-08T06:04:44.572 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_backtrace_fetch": 12333, 2024-05-08T06:04:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_set_tag": 0, 2024-05-08T06:04:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T06:04:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_inotable_repaired": 0, 2024-05-08T06:04:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dir_inodes": 56, 2024-05-08T06:04:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T06:04:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dirfrag_rstats": 56, 2024-05-08T06:04:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_file_inodes": 12277, 2024-05-08T06:04:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_inode_file_caps": 0, 2024-05-08T06:04:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_revoke": 47, 2024-05-08T06:04:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_grant": 2131, 2024-05-08T06:04:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T06:04:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T06:04:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T06:04:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_caps": 398, 2024-05-08T06:04:44.573 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_caps_dirty": 356, 2024-05-08T06:04:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_cap_release": 39, 2024-05-08T06:04:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "process_request_cap_release": 4282 2024-05-08T06:04:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cache": { 2024-05-08T06:04:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_update": 0, 2024-05-08T06:04:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_update_receipt": 0, 2024-05-08T06:04:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_try_discover": 0, 2024-05-08T06:04:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_send_discover": 10, 2024-05-08T06:04:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_handle_discover": 57, 2024-05-08T06:04:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays": 0, 2024-05-08T06:04:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays_delayed": 0, 2024-05-08T06:04:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays_enqueuing": 0, 2024-05-08T06:04:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_created": 2054, 2024-05-08T06:04:44.574 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_enqueued": 2054, 2024-05-08T06:04:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_reintegrated": 0, 2024-05-08T06:04:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_migrated": 0, 2024-05-08T06:04:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_processing": 0, 2024-05-08T06:04:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_enqueued": 0, 2024-05-08T06:04:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_prioritized": 0, 2024-05-08T06:04:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_started": 0, 2024-05-08T06:04:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_completed": 0, 2024-05-08T06:04:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_quiesce_path": 0, 2024-05-08T06:04:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_quiesce_inode": 0, 2024-05-08T06:04:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T06:04:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_exportdir": 69, 2024-05-08T06:04:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_flush": 0, 2024-05-08T06:04:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_fragmentdir": 1, 2024-05-08T06:04:44.575 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_fragstats": 0, 2024-05-08T06:04:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_inodestats": 0 2024-05-08T06:04:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_log": { 2024-05-08T06:04:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "evlrg": 0, 2024-05-08T06:04:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "evadd": 5507, 2024-05-08T06:04:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "evex": 0, 2024-05-08T06:04:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "evtrm": 0, 2024-05-08T06:04:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "ev": 5507, 2024-05-08T06:04:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "evexg": 0, 2024-05-08T06:04:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "evexd": 0, 2024-05-08T06:04:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "segadd": 8, 2024-05-08T06:04:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "segex": 0, 2024-05-08T06:04:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "segtrm": 0, 2024-05-08T06:04:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "seg": 8, 2024-05-08T06:04:44.576 INFO:teuthology.orchestra.run.smithi162.stdout: "segmjr": 2, 2024-05-08T06:04:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "segexg": 0, 2024-05-08T06:04:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "segexd": 0, 2024-05-08T06:04:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "expos": 4194304, 2024-05-08T06:04:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "wrpos": 14038803, 2024-05-08T06:04:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "rdpos": 0, 2024-05-08T06:04:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "jlat": { 2024-05-08T06:04:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 213, 2024-05-08T06:04:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 5.578052590, 2024-05-08T06:04:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.026188040 2024-05-08T06:04:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "replayed": 0 2024-05-08T06:04:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_mem": { 2024-05-08T06:04:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "ino": 21, 2024-05-08T06:04:44.577 INFO:teuthology.orchestra.run.smithi162.stdout: "ino+": 2092, 2024-05-08T06:04:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "ino-": 2071, 2024-05-08T06:04:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "dir": 20, 2024-05-08T06:04:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "dir+": 109, 2024-05-08T06:04:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "dir-": 89, 2024-05-08T06:04:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "dn": 16, 2024-05-08T06:04:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "dn+": 4161, 2024-05-08T06:04:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "dn-": 4145, 2024-05-08T06:04:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "cap": 0, 2024-05-08T06:04:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "cap+": 3596, 2024-05-08T06:04:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "cap-": 3596, 2024-05-08T06:04:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "rss": 76756, 2024-05-08T06:04:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "heap": 223608 2024-05-08T06:04:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.578 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_server": { 2024-05-08T06:04:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "dispatch_client_request": 6482, 2024-05-08T06:04:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "dispatch_server_request": 10, 2024-05-08T06:04:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_request": 4356, 2024-05-08T06:04:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_session": 11, 2024-05-08T06:04:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_peer_request": 2174, 2024-05-08T06:04:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "req_create_latency": { 2024-05-08T06:04:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 38, 2024-05-08T06:04:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.061411234, 2024-05-08T06:04:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.001616085 2024-05-08T06:04:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getattr_latency": { 2024-05-08T06:04:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.579 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getfilelock_latency": { 2024-05-08T06:04:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "req_link_latency": { 2024-05-08T06:04:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookup_latency": { 2024-05-08T06:04:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 2077, 2024-05-08T06:04:44.580 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.743846689, 2024-05-08T06:04:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000358135 2024-05-08T06:04:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookuphash_latency": { 2024-05-08T06:04:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupino_latency": { 2024-05-08T06:04:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupname_latency": { 2024-05-08T06:04:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.581 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupparent_latency": { 2024-05-08T06:04:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupsnap_latency": { 2024-05-08T06:04:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lssnap_latency": { 2024-05-08T06:04:44.582 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mkdir_latency": { 2024-05-08T06:04:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mknod_latency": { 2024-05-08T06:04:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mksnap_latency": { 2024-05-08T06:04:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "req_open_latency": { 2024-05-08T06:04:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 35, 2024-05-08T06:04:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.014295020, 2024-05-08T06:04:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000408429 2024-05-08T06:04:44.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "req_readdir_latency": { 2024-05-08T06:04:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 148, 2024-05-08T06:04:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.130085194, 2024-05-08T06:04:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000878954 2024-05-08T06:04:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rename_latency": { 2024-05-08T06:04:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "req_renamesnap_latency": { 2024-05-08T06:04:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.585 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "req_snapdiff_latency": { 2024-05-08T06:04:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmdir_latency": { 2024-05-08T06:04:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 8, 2024-05-08T06:04:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.056913358, 2024-05-08T06:04:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.007114169 2024-05-08T06:04:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmsnap_latency": { 2024-05-08T06:04:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.586 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmxattr_latency": { 2024-05-08T06:04:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setattr_latency": { 2024-05-08T06:04:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setdirlayout_latency": { 2024-05-08T06:04:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.587 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setfilelock_latency": { 2024-05-08T06:04:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setlayout_latency": { 2024-05-08T06:04:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.588 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setxattr_latency": { 2024-05-08T06:04:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "req_symlink_latency": { 2024-05-08T06:04:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "req_unlink_latency": { 2024-05-08T06:04:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 2046, 2024-05-08T06:04:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 3.631063618, 2024-05-08T06:04:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.001774713 2024-05-08T06:04:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.589 INFO:teuthology.orchestra.run.smithi162.stdout: "cap_revoke_eviction": 0, 2024-05-08T06:04:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "cap_acquisition_throttle": 0, 2024-05-08T06:04:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getvxattr_latency": { 2024-05-08T06:04:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_sessions": { 2024-05-08T06:04:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "session_count": 1, 2024-05-08T06:04:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "session_add": 2, 2024-05-08T06:04:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "session_remove": 1, 2024-05-08T06:04:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "sessions_open": 1, 2024-05-08T06:04:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "sessions_stale": 0, 2024-05-08T06:04:44.590 INFO:teuthology.orchestra.run.smithi162.stdout: "total_load": 3831, 2024-05-08T06:04:44.591 INFO:teuthology.orchestra.run.smithi162.stdout: "average_load": 3831, 2024-05-08T06:04:44.591 INFO:teuthology.orchestra.run.smithi162.stdout: "avg_session_uptime": 2277, 2024-05-08T06:04:44.591 INFO:teuthology.orchestra.run.smithi162.stdout: "mdthresh_evicted": 0 2024-05-08T06:04:44.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.591 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2024-05-08T06:04:44.591 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:44.591 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:44.591 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:44.591 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:44.591 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:44.591 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:44.591 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:44.591 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:44.591 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:44.592 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2024-05-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:44.593 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 281441, 2024-05-08T06:04:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 111, 2024-05-08T06:04:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 528, 2024-05-08T06:04:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 6, 2024-05-08T06:04:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2024-05-08T06:04:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2024-05-08T06:04:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 7728, 2024-05-08T06:04:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 163, 2024-05-08T06:04:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:44.594 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2024-05-08T06:04:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 84637, 2024-05-08T06:04:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 359, 2024-05-08T06:04:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2024-05-08T06:04:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2024-05-08T06:04:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "objecter": { 2024-05-08T06:04:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "op_active": 0, 2024-05-08T06:04:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "op_laggy": 0, 2024-05-08T06:04:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "op_send": 14757, 2024-05-08T06:04:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "op_send_bytes": 10301081, 2024-05-08T06:04:44.595 INFO:teuthology.orchestra.run.smithi162.stdout: "op_resend": 33, 2024-05-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "op_reply": 14757, 2024-05-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "op_latency": { 2024-05-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 14757, 2024-05-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 19.077599659, 2024-05-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.001292783 2024-05-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "op_inflight": 0, 2024-05-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "oplen_avg": { 2024-05-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 14757, 2024-05-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 14828 2024-05-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "op": 14757, 2024-05-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "op_r": 12359, 2024-05-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "op_w": 2398, 2024-05-08T06:04:44.596 INFO:teuthology.orchestra.run.smithi162.stdout: "op_rmw": 0, 2024-05-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "op_pg": 0, 2024-05-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_stat": 0, 2024-05-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_create": 10, 2024-05-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_read": 26, 2024-05-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_write": 239, 2024-05-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_writefull": 26, 2024-05-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_writesame": 0, 2024-05-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_append": 0, 2024-05-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_zero": 0, 2024-05-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_truncate": 0, 2024-05-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_delete": 2069, 2024-05-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_mapext": 0, 2024-05-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_sparse_read": 0, 2024-05-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_clonerange": 0, 2024-05-08T06:04:44.597 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_getxattr": 12333, 2024-05-08T06:04:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_setxattr": 20, 2024-05-08T06:04:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_cmpxattr": 0, 2024-05-08T06:04:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_rmxattr": 0, 2024-05-08T06:04:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_resetxattrs": 0, 2024-05-08T06:04:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_call": 0, 2024-05-08T06:04:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_watch": 0, 2024-05-08T06:04:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_notify": 0, 2024-05-08T06:04:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T06:04:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_pgls": 0, 2024-05-08T06:04:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_pgls_filter": 0, 2024-05-08T06:04:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_other": 0, 2024-05-08T06:04:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_active": 0, 2024-05-08T06:04:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_send": 0, 2024-05-08T06:04:44.598 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_resend": 0, 2024-05-08T06:04:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_ping": 0, 2024-05-08T06:04:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_active": 0, 2024-05-08T06:04:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_send": 0, 2024-05-08T06:04:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_resend": 0, 2024-05-08T06:04:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_active": 0, 2024-05-08T06:04:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_send": 0, 2024-05-08T06:04:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_resend": 0, 2024-05-08T06:04:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_active": 0, 2024-05-08T06:04:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_send": 0, 2024-05-08T06:04:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_resend": 0, 2024-05-08T06:04:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "command_active": 0, 2024-05-08T06:04:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "command_send": 0, 2024-05-08T06:04:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "command_resend": 0, 2024-05-08T06:04:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "map_epoch": 84, 2024-05-08T06:04:44.599 INFO:teuthology.orchestra.run.smithi162.stdout: "map_full": 0, 2024-05-08T06:04:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "map_inc": 4, 2024-05-08T06:04:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_sessions": 12, 2024-05-08T06:04:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_session_open": 12, 2024-05-08T06:04:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_session_close": 0, 2024-05-08T06:04:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_laggy": 0, 2024-05-08T06:04:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_wr": 89, 2024-05-08T06:04:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_rd": 0, 2024-05-08T06:04:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_del": 16 2024-05-08T06:04:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "oft": { 2024-05-08T06:04:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_objs": 1, 2024-05-08T06:04:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_kv_pairs": 0, 2024-05-08T06:04:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_updates": 0, 2024-05-08T06:04:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_removes": 0 2024-05-08T06:04:44.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "osd-slow-ops": { 2024-05-08T06:04:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "slow_ops_count": 0 2024-05-08T06:04:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "purge_queue": { 2024-05-08T06:04:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_ops": 0, 2024-05-08T06:04:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_ops_high_water": 10, 2024-05-08T06:04:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing": 0, 2024-05-08T06:04:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_high_water": 10, 2024-05-08T06:04:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executed_ops": 2054, 2024-05-08T06:04:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executed": 2054, 2024-05-08T06:04:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_item_in_journal": 0 2024-05-08T06:04:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:44.601 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-05-08T06:04:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 24474, 2024-05-08T06:04:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 7649043, 2024-05-08T06:04:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 24474, 2024-05-08T06:04:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-05-08T06:04:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-05-08T06:04:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 24474, 2024-05-08T06:04:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 7649043, 2024-05-08T06:04:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:44.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-objecter_bytes": { 2024-05-08T06:04:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-05-08T06:04:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-05-08T06:04:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-05-08T06:04:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 14757, 2024-05-08T06:04:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 10508535, 2024-05-08T06:04:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 14757, 2024-05-08T06:04:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 10508535, 2024-05-08T06:04:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:44.603 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-objecter_ops": { 2024-05-08T06:04:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 1024, 2024-05-08T06:04:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-05-08T06:04:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-05-08T06:04:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 14757, 2024-05-08T06:04:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 14757, 2024-05-08T06:04:44.604 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 14757, 2024-05-08T06:04:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 14757, 2024-05-08T06:04:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-write_buf_throttle": { 2024-05-08T06:04:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 3758096384, 2024-05-08T06:04:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 2054, 2024-05-08T06:04:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 207454, 2024-05-08T06:04:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:44.605 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 2054, 2024-05-08T06:04:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-05-08T06:04:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-05-08T06:04:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 26, 2024-05-08T06:04:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 207454, 2024-05-08T06:04:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-write_buf_throttle-0x561756b3a500": { 2024-05-08T06:04:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 3758096384, 2024-05-08T06:04:44.606 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 5507, 2024-05-08T06:04:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 9844499, 2024-05-08T06:04:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 5507, 2024-05-08T06:04:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-05-08T06:04:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-05-08T06:04:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 213, 2024-05-08T06:04:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 9844499, 2024-05-08T06:04:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:44.607 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:44.607 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-05-08T06:04:45.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:04:45.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:04:45.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:04:45.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:44 smithi112 ceph-mon[30003]: pgmap v724: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:04:45.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:04:45.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:45.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:45.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:44 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:04:45.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:04:45.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:04:45.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:04:45.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:44 smithi162 ceph-mon[34800]: pgmap v724: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:04:45.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:04:45.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:45.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:45.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:44 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:04:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:04:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:04:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:04:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:44 smithi150 ceph-mon[34315]: pgmap v724: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:04:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:04:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:45.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:44 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' 2024-05-08T06:04:45.345 INFO:tasks.check_counter:Daemon mds.i mds.exported=69 2024-05-08T06:04:45.345 INFO:tasks.check_counter:Daemon mds.i mds.imported=62 2024-05-08T06:04:45.345 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=1 2024-05-08T06:04:45.346 DEBUG:tasks.check_counter:Getting stats from l 2024-05-08T06:04:45.346 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-08T06:04:45.600 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:04:47.020 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:46 smithi162 ceph-mon[34800]: pgmap v725: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T06:04:47.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:46 smithi150 ceph-mon[34315]: pgmap v725: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T06:04:47.297 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-05-08T06:04:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 4, 2024-05-08T06:04:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 4, 2024-05-08T06:04:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 2944, 2024-05-08T06:04:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 448, 2024-05-08T06:04:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 0, 2024-05-08T06:04:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.001281292, 2024-05-08T06:04:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.000033240, 2024-05-08T06:04:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.000441501, 2024-05-08T06:04:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 3, 2024-05-08T06:04:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000727320, 2024-05-08T06:04:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000242440 2024-05-08T06:04:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-08T06:04:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T06:04:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 237, 2024-05-08T06:04:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 234, 2024-05-08T06:04:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 518496, 2024-05-08T06:04:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 144544, 2024-05-08T06:04:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 2, 2024-05-08T06:04:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.036381061, 2024-05-08T06:04:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.014063871, 2024-05-08T06:04:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.016072387, 2024-05-08T06:04:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.005836793, 2024-05-08T06:04:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 233, 2024-05-08T06:04:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.011667960, 2024-05-08T06:04:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000050077 2024-05-08T06:04:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 518496, 2024-05-08T06:04:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 144544 2024-05-08T06:04:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 2, 2024-05-08T06:04:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 181, 2024-05-08T06:04:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 320, 2024-05-08T06:04:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 189200, 2024-05-08T06:04:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 2, 2024-05-08T06:04:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.010373764, 2024-05-08T06:04:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.007611998, 2024-05-08T06:04:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.000766970, 2024-05-08T06:04:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 179, 2024-05-08T06:04:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.012938644, 2024-05-08T06:04:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000072282 2024-05-08T06:04:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T06:04:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 189200 2024-05-08T06:04:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2024-05-08T06:04:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 0, 2024-05-08T06:04:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2024-05-08T06:04:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2024-05-08T06:04:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2024-05-08T06:04:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 148923, 2024-05-08T06:04:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 29, 2024-05-08T06:04:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2024-05-08T06:04:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2024-05-08T06:04:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 0, 2024-05-08T06:04:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 0, 2024-05-08T06:04:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2024-05-08T06:04:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 928, 2024-05-08T06:04:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 2, 2024-05-08T06:04:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2024-05-08T06:04:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2024-05-08T06:04:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-05-08T06:04:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-05-08T06:04:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-05-08T06:04:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-05-08T06:04:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 239, 2024-05-08T06:04:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 485012, 2024-05-08T06:04:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 239, 2024-05-08T06:04:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-05-08T06:04:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-05-08T06:04:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 239, 2024-05-08T06:04:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 485012, 2024-05-08T06:04:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-05-08T06:04:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-05-08T06:04:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-05-08T06:04:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-05-08T06:04:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-05-08T06:04:47.310 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-05-08T06:04:47.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:46 smithi112 ceph-mon[30003]: pgmap v725: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-08T06:04:47.990 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-08T06:04:47.990 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-08T06:04:47.990 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-05-08T06:04:47.990 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-08T06:04:48.000 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-08T06:04:48.000 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T06:04:48.000 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:04:48.060 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-05-08T06:04:48.060 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:04:48.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-08T06:04:48.556 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:48.965 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:48 smithi112 ceph-mon[30003]: pgmap v726: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:49.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:48 smithi150 ceph-mon[34315]: pgmap v726: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:49.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:48 smithi162 ceph-mon[34800]: pgmap v726: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:50.922 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:50 smithi112 ceph-mon[30003]: pgmap v727: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:04:50.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.951+0000 7fe547c00700 1 -- 172.21.15.112:0/301400515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe54006b1b0 msgr2=0x7fe54006b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:50.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.951+0000 7fe547c00700 1 --2- 172.21.15.112:0/301400515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe54006b1b0 0x7fe54006b590 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe53400b0a0 tx=0x7fe53400b3b0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.952+0000 7fe547c00700 1 -- 172.21.15.112:0/301400515 shutdown_connections 2024-05-08T06:04:50.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.952+0000 7fe547c00700 1 --2- 172.21.15.112:0/301400515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe54010ff50 0x7fe540112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.952+0000 7fe547c00700 1 --2- 172.21.15.112:0/301400515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe54006b1b0 0x7fe54006b590 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.952+0000 7fe547c00700 1 --2- 172.21.15.112:0/301400515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe54006bad0 0x7fe54010fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.952+0000 7fe547c00700 1 -- 172.21.15.112:0/301400515 >> 172.21.15.112:0/301400515 conn(0x7fe540079350 msgr2=0x7fe540079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:50.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.952+0000 7fe547c00700 1 -- 172.21.15.112:0/301400515 shutdown_connections 2024-05-08T06:04:50.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.952+0000 7fe547c00700 1 -- 172.21.15.112:0/301400515 wait complete. 2024-05-08T06:04:50.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.953+0000 7fe547c00700 1 Processor -- start 2024-05-08T06:04:50.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.953+0000 7fe547c00700 1 -- start start 2024-05-08T06:04:50.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.953+0000 7fe547c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe54006b1b0 0x7fe54019bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:50.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.953+0000 7fe547c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe54006bad0 0x7fe54019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:50.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.953+0000 7fe547c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe54010ff50 0x7fe5401a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.953+0000 7fe547c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401158c0 con 0x7fe54006b1b0 2024-05-08T06:04:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.953+0000 7fe547c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe540115740 con 0x7fe54010ff50 2024-05-08T06:04:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.953+0000 7fe547c00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe540115a40 con 0x7fe54006bad0 2024-05-08T06:04:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.954+0000 7fe545200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe54006b1b0 0x7fe54019bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.954+0000 7fe53fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe54006bad0 0x7fe54019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.954+0000 7fe545200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe54006b1b0 0x7fe54019bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51484/0 (socket says 172.21.15.112:51484) 2024-05-08T06:04:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.954+0000 7fe545200700 1 -- 172.21.15.112:0/3153144525 learned_addr learned my addr 172.21.15.112:0/3153144525 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.954+0000 7fe545c00700 1 --2- 172.21.15.112:0/3153144525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe54010ff50 0x7fe5401a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.954+0000 7fe53fe00700 1 -- 172.21.15.112:0/3153144525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe54010ff50 msgr2=0x7fe5401a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.954+0000 7fe53fe00700 1 --2- 172.21.15.112:0/3153144525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe54010ff50 0x7fe5401a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.954+0000 7fe53fe00700 1 -- 172.21.15.112:0/3153144525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe54006b1b0 msgr2=0x7fe54019bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.954+0000 7fe53fe00700 1 --2- 172.21.15.112:0/3153144525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe54006b1b0 0x7fe54019bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.954+0000 7fe53fe00700 1 -- 172.21.15.112:0/3153144525 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5401a0de0 con 0x7fe54006bad0 2024-05-08T06:04:50.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.954+0000 7fe53fe00700 1 --2- 172.21.15.112:0/3153144525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe54006bad0 0x7fe54019c310 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00e9f0 tx=0x7fe52c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:50.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.955+0000 7fe53d600700 1 -- 172.21.15.112:0/3153144525 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c00cb70 con 0x7fe54006bad0 2024-05-08T06:04:50.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.955+0000 7fe547c00700 1 -- 172.21.15.112:0/3153144525 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5401a10d0 con 0x7fe54006bad0 2024-05-08T06:04:50.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.955+0000 7fe53d600700 1 -- 172.21.15.112:0/3153144525 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe52c00ee90 con 0x7fe54006bad0 2024-05-08T06:04:50.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.955+0000 7fe53d600700 1 -- 172.21.15.112:0/3153144525 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c003d40 con 0x7fe54006bad0 2024-05-08T06:04:50.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.955+0000 7fe547c00700 1 -- 172.21.15.112:0/3153144525 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5401a8950 con 0x7fe54006bad0 2024-05-08T06:04:50.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.956+0000 7fe547c00700 1 -- 172.21.15.112:0/3153144525 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe54006a190 con 0x7fe54006bad0 2024-05-08T06:04:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.958+0000 7fe53d600700 1 -- 172.21.15.112:0/3153144525 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe52c003ee0 con 0x7fe54006bad0 2024-05-08T06:04:50.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.960+0000 7fe53d600700 1 --2- 172.21.15.112:0/3153144525 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe5280b2880 0x7fe5280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:50.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.960+0000 7fe53d600700 1 -- 172.21.15.112:0/3153144525 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fe52c0e2600 con 0x7fe54006bad0 2024-05-08T06:04:50.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.960+0000 7fe545200700 1 --2- 172.21.15.112:0/3153144525 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe5280b2880 0x7fe5280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:50.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.961+0000 7fe545200700 1 --2- 172.21.15.112:0/3153144525 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe5280b2880 0x7fe5280b4d40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe534009e60 tx=0x7fe534006230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:50.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:50.961+0000 7fe53d600700 1 -- 172.21.15.112:0/3153144525 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe52c054df0 con 0x7fe54006bad0 2024-05-08T06:04:51.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:50 smithi150 ceph-mon[34315]: pgmap v727: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:04:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:51.210+0000 7fe547c00700 1 -- 172.21.15.112:0/3153144525 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe54002e510 con 0x7fe54006bad0 2024-05-08T06:04:51.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:51.211+0000 7fe53d600700 1 -- 172.21.15.112:0/3153144525 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) ==== 81+0+598 (secure 0 0 0) 0x7fe52c0aa290 con 0x7fe54006bad0 2024-05-08T06:04:51.212 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3667148497 2024-05-09T05:43:03.313546+0000 2024-05-08T06:04:51.212 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/164150456 2024-05-09T05:43:03.313546+0000 2024-05-08T06:04:51.212 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3425327158 2024-05-09T05:42:29.819307+0000 2024-05-08T06:04:51.212 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1404611205 2024-05-09T05:43:03.313546+0000 2024-05-08T06:04:51.212 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3765334043 2024-05-09T05:42:29.819307+0000 2024-05-08T06:04:51.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/772108158 2024-05-09T05:42:29.819307+0000 2024-05-08T06:04:51.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2814303790 2024-05-09T05:43:03.313546+0000 2024-05-08T06:04:51.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/164150456 2024-05-09T05:43:03.313546+0000 2024-05-08T06:04:51.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/772108158 2024-05-09T05:42:29.819307+0000 2024-05-08T06:04:51.213 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1091751583 2024-05-09T05:42:29.819307+0000 2024-05-08T06:04:51.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:51.214+0000 7fe547c00700 1 -- 172.21.15.112:0/3153144525 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe5280b2880 msgr2=0x7fe5280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:51.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:51.214+0000 7fe547c00700 1 --2- 172.21.15.112:0/3153144525 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe5280b2880 0x7fe5280b4d40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe534009e60 tx=0x7fe534006230 comp rx=0 tx=0).stop 2024-05-08T06:04:51.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:51.214+0000 7fe547c00700 1 -- 172.21.15.112:0/3153144525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe54006bad0 msgr2=0x7fe54019c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:51.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:51.214+0000 7fe547c00700 1 --2- 172.21.15.112:0/3153144525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe54006bad0 0x7fe54019c310 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00e9f0 tx=0x7fe52c00ed00 comp rx=0 tx=0).stop 2024-05-08T06:04:51.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:51.214+0000 7fe547c00700 1 -- 172.21.15.112:0/3153144525 shutdown_connections 2024-05-08T06:04:51.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:51.214+0000 7fe547c00700 1 --2- 172.21.15.112:0/3153144525 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fe5280b2880 0x7fe5280b4d40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:51.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:51.214+0000 7fe547c00700 1 --2- 172.21.15.112:0/3153144525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe54010ff50 0x7fe5401a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:51.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:51.214+0000 7fe547c00700 1 --2- 172.21.15.112:0/3153144525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe54006b1b0 0x7fe54019bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:51.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:51.214+0000 7fe547c00700 1 --2- 172.21.15.112:0/3153144525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe54006bad0 0x7fe54019c310 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:51.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:51.214+0000 7fe547c00700 1 -- 172.21.15.112:0/3153144525 >> 172.21.15.112:0/3153144525 conn(0x7fe540079350 msgr2=0x7fe540111400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:51.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:51.214+0000 7fe547c00700 1 -- 172.21.15.112:0/3153144525 shutdown_connections 2024-05-08T06:04:51.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:51.215+0000 7fe547c00700 1 -- 172.21.15.112:0/3153144525 wait complete. 2024-05-08T06:04:51.216 INFO:teuthology.orchestra.run.smithi112.stderr:listed 10 entries 2024-05-08T06:04:51.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:50 smithi162 ceph-mon[34800]: pgmap v727: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:04:52.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:51 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3153144525' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T06:04:52.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:51 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3153144525' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T06:04:52.271 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T06:04:52.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:04:52.272 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T06:04:52.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:51 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3153144525' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T06:04:52.388 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-05-08T06:04:52.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:04:52.388 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T06:04:52.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:04:52.480 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-05-08T06:04:52.561 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T06:04:52.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:04:52.562 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T06:04:52.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:04:52.654 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-05-08T06:04:52.767 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-08T06:04:52.777 INFO:tasks.cephadm:Teardown begin 2024-05-08T06:04:52.777 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T06:04:52.777 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:04:53.162 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:53.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:52 smithi150 ceph-mon[34315]: pgmap v728: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:53.217 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:52 smithi162 ceph-mon[34800]: pgmap v728: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:53.308 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:52 smithi112 ceph-mon[30003]: pgmap v728: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:04:55.007 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:54 smithi112 ceph-mon[30003]: pgmap v729: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-08T06:04:55.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:54 smithi150 ceph-mon[34315]: pgmap v729: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-08T06:04:55.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:54 smithi162 ceph-mon[34800]: pgmap v729: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-08T06:04:55.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.563+0000 7f0284200700 1 -- 172.21.15.112:0/2631928367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f027c06b180 msgr2=0x7f027c06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:55.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.563+0000 7f0284200700 1 --2- 172.21.15.112:0/2631928367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f027c06b180 0x7f027c06b600 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0268009990 tx=0x7f0268009ca0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.563+0000 7f0284200700 1 -- 172.21.15.112:0/2631928367 shutdown_connections 2024-05-08T06:04:55.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.563+0000 7f0284200700 1 --2- 172.21.15.112:0/2631928367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f027c06bb40 0x7f027c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.563+0000 7f0284200700 1 --2- 172.21.15.112:0/2631928367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f027c109510 0x7f027c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.563+0000 7f0284200700 1 --2- 172.21.15.112:0/2631928367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f027c06b180 0x7f027c06b600 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.563+0000 7f0284200700 1 -- 172.21.15.112:0/2631928367 >> 172.21.15.112:0/2631928367 conn(0x7f027c079350 msgr2=0x7f027c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:55.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.563+0000 7f0284200700 1 -- 172.21.15.112:0/2631928367 shutdown_connections 2024-05-08T06:04:55.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.563+0000 7f0284200700 1 -- 172.21.15.112:0/2631928367 wait complete. 2024-05-08T06:04:55.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.565+0000 7f0284200700 1 Processor -- start 2024-05-08T06:04:55.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.565+0000 7f0284200700 1 -- start start 2024-05-08T06:04:55.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.565+0000 7f0284200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f027c06b180 0x7f027c19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:55.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.565+0000 7f0284200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f027c06bb40 0x7f027c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:55.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.565+0000 7f0284200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f027c109510 0x7f027c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:55.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.565+0000 7f0284200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f027c113670 con 0x7f027c06bb40 2024-05-08T06:04:55.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.565+0000 7f0284200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1134f0 con 0x7f027c06b180 2024-05-08T06:04:55.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.565+0000 7f0284200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1137f0 con 0x7f027c109510 2024-05-08T06:04:55.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.565+0000 7f0280e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f027c06bb40 0x7f027c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:55.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.565+0000 7f0281800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f027c06b180 0x7f027c19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:55.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.565+0000 7f0281800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f027c06b180 0x7f027c19bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:48644/0 (socket says 172.21.15.112:48644) 2024-05-08T06:04:55.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.566+0000 7f0282200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f027c109510 0x7f027c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:55.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.566+0000 7f0281800700 1 -- 172.21.15.112:0/1018909469 learned_addr learned my addr 172.21.15.112:0/1018909469 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:55.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.565+0000 7f0280e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f027c06bb40 0x7f027c19c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42584/0 (socket says 172.21.15.112:42584) 2024-05-08T06:04:55.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.566+0000 7f0281800700 1 -- 172.21.15.112:0/1018909469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f027c109510 msgr2=0x7f027c1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:55.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.566+0000 7f0281800700 1 --2- 172.21.15.112:0/1018909469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f027c109510 0x7f027c1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.566+0000 7f0281800700 1 -- 172.21.15.112:0/1018909469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f027c06bb40 msgr2=0x7f027c19c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:55.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.566+0000 7f0281800700 1 --2- 172.21.15.112:0/1018909469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f027c06bb40 0x7f027c19c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.566+0000 7f0281800700 1 -- 172.21.15.112:0/1018909469 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027c1a0dd0 con 0x7f027c06b180 2024-05-08T06:04:55.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.566+0000 7f0282200700 1 --2- 172.21.15.112:0/1018909469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f027c109510 0x7f027c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:04:55.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.566+0000 7f0281800700 1 --2- 172.21.15.112:0/1018909469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f027c06b180 0x7f027c19bdc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f027000ef10 tx=0x7f027000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:55.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.566+0000 7f027a000700 1 -- 172.21.15.112:0/1018909469 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027000c9c0 con 0x7f027c06b180 2024-05-08T06:04:55.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.567+0000 7f027a000700 1 -- 172.21.15.112:0/1018909469 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0270019070 con 0x7f027c06b180 2024-05-08T06:04:55.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.567+0000 7f0284200700 1 -- 172.21.15.112:0/1018909469 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f027c1a1060 con 0x7f027c06b180 2024-05-08T06:04:55.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.567+0000 7f027a000700 1 -- 172.21.15.112:0/1018909469 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270014480 con 0x7f027c06b180 2024-05-08T06:04:55.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.567+0000 7f0284200700 1 -- 172.21.15.112:0/1018909469 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f027c1a8940 con 0x7f027c06b180 2024-05-08T06:04:55.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.568+0000 7f0284200700 1 -- 172.21.15.112:0/1018909469 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f027c06a190 con 0x7f027c06b180 2024-05-08T06:04:55.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.570+0000 7f027a000700 1 -- 172.21.15.112:0/1018909469 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0270012050 con 0x7f027c06b180 2024-05-08T06:04:55.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.572+0000 7f027a000700 1 --2- 172.21.15.112:0/1018909469 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f02640b2880 0x7f02640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:55.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.572+0000 7f027a000700 1 -- 172.21.15.112:0/1018909469 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f02700e2880 con 0x7f027c06b180 2024-05-08T06:04:55.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.572+0000 7f0280e00700 1 --2- 172.21.15.112:0/1018909469 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f02640b2880 0x7f02640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.573+0000 7f0280e00700 1 --2- 172.21.15.112:0/1018909469 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f02640b2880 0x7f02640b4d40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f027c104490 tx=0x7f026800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:55.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.574+0000 7f027a000700 1 -- 172.21.15.112:0/1018909469 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0270010040 con 0x7f027c06b180 2024-05-08T06:04:55.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.851+0000 7f0284200700 1 -- 172.21.15.112:0/1018909469 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f027c1a8b10 con 0x7f027c06b180 2024-05-08T06:04:55.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.853+0000 7f027a000700 1 -- 172.21.15.112:0/1018909469 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11894 (secure 0 0 0) 0x7f02700aa510 con 0x7f027c06b180 2024-05-08T06:04:55.853 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T06:04:55.853 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T06:03:33.314029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24511,24518]},"id":1}]} 2024-05-08T06:04:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.856+0000 7f0284200700 1 -- 172.21.15.112:0/1018909469 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f02640b2880 msgr2=0x7f02640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.856+0000 7f0284200700 1 --2- 172.21.15.112:0/1018909469 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f02640b2880 0x7f02640b4d40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f027c104490 tx=0x7f026800b260 comp rx=0 tx=0).stop 2024-05-08T06:04:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.856+0000 7f0284200700 1 -- 172.21.15.112:0/1018909469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f027c06b180 msgr2=0x7f027c19bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.856+0000 7f0284200700 1 --2- 172.21.15.112:0/1018909469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f027c06b180 0x7f027c19bdc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f027000ef10 tx=0x7f027000c3a0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.856+0000 7f0284200700 1 -- 172.21.15.112:0/1018909469 shutdown_connections 2024-05-08T06:04:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.856+0000 7f0284200700 1 --2- 172.21.15.112:0/1018909469 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f02640b2880 0x7f02640b4d40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.856+0000 7f0284200700 1 --2- 172.21.15.112:0/1018909469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f027c109510 0x7f027c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.856+0000 7f0284200700 1 --2- 172.21.15.112:0/1018909469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f027c06b180 0x7f027c19bdc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.856+0000 7f0284200700 1 --2- 172.21.15.112:0/1018909469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f027c06bb40 0x7f027c19c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.856+0000 7f0284200700 1 -- 172.21.15.112:0/1018909469 >> 172.21.15.112:0/1018909469 conn(0x7f027c079350 msgr2=0x7f027c10e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.857+0000 7f0284200700 1 -- 172.21.15.112:0/1018909469 shutdown_connections 2024-05-08T06:04:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:55.857+0000 7f0284200700 1 -- 172.21.15.112:0/1018909469 wait complete. 2024-05-08T06:04:55.857 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 36 2024-05-08T06:04:56.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:55 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1018909469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:04:56.258 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:55 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1018909469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:04:56.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:55 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1018909469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:04:56.764 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:04:57.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:56 smithi112 ceph-mon[30003]: pgmap v730: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-08T06:04:57.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:56 smithi150 ceph-mon[34315]: pgmap v730: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-08T06:04:57.220 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:57.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:56 smithi162 ceph-mon[34800]: pgmap v730: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-08T06:04:59.145 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:58 smithi112 ceph-mon[30003]: pgmap v731: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:04:59.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:58 smithi150 ceph-mon[34315]: pgmap v731: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:04:59.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:58 smithi162 ceph-mon[34800]: pgmap v731: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:04:59.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.459+0000 7fba83800700 1 -- 172.21.15.112:0/946288013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba7c10b6c0 msgr2=0x7fba7c10baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:59.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.459+0000 7fba83800700 1 --2- 172.21.15.112:0/946288013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba7c10b6c0 0x7fba7c10baa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fba700098f0 tx=0x7fba70009c00 comp rx=0 tx=0).stop 2024-05-08T06:04:59.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.459+0000 7fba83800700 1 -- 172.21.15.112:0/946288013 shutdown_connections 2024-05-08T06:04:59.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.459+0000 7fba83800700 1 --2- 172.21.15.112:0/946288013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba7c106080 0x7fba7c112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:59.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.459+0000 7fba83800700 1 --2- 172.21.15.112:0/946288013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba7c10b6c0 0x7fba7c10baa0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:59.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.459+0000 7fba83800700 1 --2- 172.21.15.112:0/946288013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7c1056c0 0x7fba7c105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:59.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.459+0000 7fba83800700 1 -- 172.21.15.112:0/946288013 >> 172.21.15.112:0/946288013 conn(0x7fba7c1011b0 msgr2=0x7fba7c1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:59.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.459+0000 7fba83800700 1 -- 172.21.15.112:0/946288013 shutdown_connections 2024-05-08T06:04:59.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.459+0000 7fba83800700 1 -- 172.21.15.112:0/946288013 wait complete. 2024-05-08T06:04:59.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.460+0000 7fba83800700 1 Processor -- start 2024-05-08T06:04:59.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.460+0000 7fba83800700 1 -- start start 2024-05-08T06:04:59.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.461+0000 7fba83800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba7c1056c0 0x7fba7c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:59.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.461+0000 7fba83800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba7c106080 0x7fba7c19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:59.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.461+0000 7fba83800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7c10b6c0 0x7fba7c1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:59.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.461+0000 7fba83800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1158c0 con 0x7fba7c10b6c0 2024-05-08T06:04:59.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.461+0000 7fba83800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c115740 con 0x7fba7c1056c0 2024-05-08T06:04:59.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.461+0000 7fba83800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c115a40 con 0x7fba7c106080 2024-05-08T06:04:59.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.461+0000 7fba80e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba7c1056c0 0x7fba7c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:59.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.461+0000 7fba81800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7c10b6c0 0x7fba7c1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:59.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.461+0000 7fba80e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba7c1056c0 0x7fba7c19bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:48670/0 (socket says 172.21.15.112:48670) 2024-05-08T06:04:59.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.461+0000 7fba80e00700 1 -- 172.21.15.112:0/1423621754 learned_addr learned my addr 172.21.15.112:0/1423621754 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:04:59.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.461+0000 7fba7be00700 1 --2- 172.21.15.112:0/1423621754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba7c106080 0x7fba7c19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:59.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.462+0000 7fba80e00700 1 -- 172.21.15.112:0/1423621754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba7c106080 msgr2=0x7fba7c19c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:59.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.462+0000 7fba80e00700 1 --2- 172.21.15.112:0/1423621754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba7c106080 0x7fba7c19c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:59.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.462+0000 7fba80e00700 1 -- 172.21.15.112:0/1423621754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7c10b6c0 msgr2=0x7fba7c1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:59.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.462+0000 7fba80e00700 1 --2- 172.21.15.112:0/1423621754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7c10b6c0 0x7fba7c1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:59.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.462+0000 7fba80e00700 1 -- 172.21.15.112:0/1423621754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c1a0dc0 con 0x7fba7c1056c0 2024-05-08T06:04:59.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.462+0000 7fba7be00700 1 --2- 172.21.15.112:0/1423621754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba7c106080 0x7fba7c19c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:04:59.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.462+0000 7fba80e00700 1 --2- 172.21.15.112:0/1423621754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba7c1056c0 0x7fba7c19bdb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fba7000f990 tx=0x7fba7000f9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:59.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.462+0000 7fba79600700 1 -- 172.21.15.112:0/1423621754 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba70027070 con 0x7fba7c1056c0 2024-05-08T06:04:59.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.462+0000 7fba83800700 1 -- 172.21.15.112:0/1423621754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7c1a1050 con 0x7fba7c1056c0 2024-05-08T06:04:59.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.462+0000 7fba79600700 1 -- 172.21.15.112:0/1423621754 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba700055f0 con 0x7fba7c1056c0 2024-05-08T06:04:59.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.462+0000 7fba79600700 1 -- 172.21.15.112:0/1423621754 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba70004930 con 0x7fba7c1056c0 2024-05-08T06:04:59.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.462+0000 7fba83800700 1 -- 172.21.15.112:0/1423621754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7c1a8930 con 0x7fba7c1056c0 2024-05-08T06:04:59.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.464+0000 7fba83800700 1 -- 172.21.15.112:0/1423621754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7c10dc50 con 0x7fba7c1056c0 2024-05-08T06:04:59.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.466+0000 7fba79600700 1 -- 172.21.15.112:0/1423621754 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fba7002a070 con 0x7fba7c1056c0 2024-05-08T06:04:59.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.467+0000 7fba79600700 1 --2- 172.21.15.112:0/1423621754 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fba640b2880 0x7fba640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:59.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.468+0000 7fba79600700 1 -- 172.21.15.112:0/1423621754 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7fba700e32a0 con 0x7fba7c1056c0 2024-05-08T06:04:59.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.468+0000 7fba7be00700 1 --2- 172.21.15.112:0/1423621754 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fba640b2880 0x7fba640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:59.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.468+0000 7fba7be00700 1 --2- 172.21.15.112:0/1423621754 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fba640b2880 0x7fba640b4d40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fba7c1038d0 tx=0x7fba68009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:59.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.470+0000 7fba79600700 1 -- 172.21.15.112:0/1423621754 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba70031070 con 0x7fba7c1056c0 2024-05-08T06:04:59.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.757+0000 7fba83800700 1 -- 172.21.15.112:0/1423621754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba7c06a190 con 0x7fba7c1056c0 2024-05-08T06:04:59.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.758+0000 7fba79600700 1 -- 172.21.15.112:0/1423621754 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11894 (secure 0 0 0) 0x7fba70013030 con 0x7fba7c1056c0 2024-05-08T06:04:59.758 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T06:04:59.758 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3258450798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3258450798},{"type":"v1","addr":"172.21.15.112:6835","nonce":3258450798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2849670124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2849670124},{"type":"v1","addr":"172.21.15.112:6837","nonce":2849670124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14391,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3014879788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3014879788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3014879788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/442303489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":442303489},{"type":"v1","addr":"172.21.15.150:6833","nonce":442303489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3668390959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3668390959},{"type":"v1","addr":"172.21.15.162:6833","nonce":3668390959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1945132252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1945132252},{"type":"v1","addr":"172.21.15.150:6835","nonce":1945132252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:51:07.312542+0000","modified":"2024-05-08T06:03:33.314029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24529,"mds_1":24518,"mds_2":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/4103977076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4103977076},{"type":"v1","addr":"172.21.15.150:6837","nonce":4103977076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6835/3060051414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3060051414},{"type":"v1","addr":"172.21.15.162:6835","nonce":3060051414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6837/3092620994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3092620994},{"type":"v1","addr":"172.21.15.162:6837","nonce":3092620994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.162:6839/539986796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":539986796},{"type":"v1","addr":"172.21.15.162:6839","nonce":539986796}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.150:6839/1284437777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":1284437777},{"type":"v1","addr":"172.21.15.150:6839","nonce":1284437777}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.112:6841/2816461111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":2816461111},{"type":"v1","addr":"172.21.15.112:6841","nonce":2816461111}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24529,"qdb_cluster":[24529,24511,24518]},"id":1}]} 2024-05-08T06:04:59.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.761+0000 7fba83800700 1 -- 172.21.15.112:0/1423621754 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fba640b2880 msgr2=0x7fba640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:59.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.761+0000 7fba83800700 1 --2- 172.21.15.112:0/1423621754 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fba640b2880 0x7fba640b4d40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fba7c1038d0 tx=0x7fba68009310 comp rx=0 tx=0).stop 2024-05-08T06:04:59.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.761+0000 7fba83800700 1 -- 172.21.15.112:0/1423621754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba7c1056c0 msgr2=0x7fba7c19bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:59.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.761+0000 7fba83800700 1 --2- 172.21.15.112:0/1423621754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba7c1056c0 0x7fba7c19bdb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fba7000f990 tx=0x7fba7000f9c0 comp rx=0 tx=0).stop 2024-05-08T06:04:59.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.761+0000 7fba83800700 1 -- 172.21.15.112:0/1423621754 shutdown_connections 2024-05-08T06:04:59.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.761+0000 7fba83800700 1 --2- 172.21.15.112:0/1423621754 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fba640b2880 0x7fba640b4d40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:59.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.761+0000 7fba83800700 1 --2- 172.21.15.112:0/1423621754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba7c10b6c0 0x7fba7c1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:59.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.761+0000 7fba83800700 1 --2- 172.21.15.112:0/1423621754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba7c1056c0 0x7fba7c19bdb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:59.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.761+0000 7fba83800700 1 --2- 172.21.15.112:0/1423621754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba7c106080 0x7fba7c19c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:59.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.761+0000 7fba83800700 1 -- 172.21.15.112:0/1423621754 >> 172.21.15.112:0/1423621754 conn(0x7fba7c1011b0 msgr2=0x7fba7c101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:59.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.762+0000 7fba83800700 1 -- 172.21.15.112:0/1423621754 shutdown_connections 2024-05-08T06:04:59.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:04:59.762+0000 7fba83800700 1 -- 172.21.15.112:0/1423621754 wait complete. 2024-05-08T06:04:59.763 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 36 2024-05-08T06:05:00.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:04:59 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1423621754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:00.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:04:59 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1423621754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:00.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:04:59 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1423621754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:00.715 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T06:05:01.165 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:01.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:00 smithi150 ceph-mon[34315]: pgmap v732: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-08T06:05:01.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:00 smithi112 ceph-mon[30003]: pgmap v732: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-08T06:05:01.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:00 smithi162 ceph-mon[34800]: pgmap v732: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-08T06:05:03.228 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:02 smithi112 ceph-mon[30003]: pgmap v733: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:05:03.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:02 smithi162 ceph-mon[34800]: pgmap v733: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:05:03.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.395+0000 7f73f5800700 1 -- 172.21.15.112:0/3479498213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73f010b6e0 msgr2=0x7f73f010bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:03.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.395+0000 7f73f5800700 1 --2- 172.21.15.112:0/3479498213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73f010b6e0 0x7f73f010bac0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f73e00098f0 tx=0x7f73e0009c00 comp rx=0 tx=0).stop 2024-05-08T06:05:03.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.395+0000 7f73f5800700 1 -- 172.21.15.112:0/3479498213 shutdown_connections 2024-05-08T06:05:03.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.395+0000 7f73f5800700 1 --2- 172.21.15.112:0/3479498213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f01060a0 0x7f73f01125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:03.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.395+0000 7f73f5800700 1 --2- 172.21.15.112:0/3479498213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73f010b6e0 0x7f73f010bac0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:03.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.395+0000 7f73f5800700 1 --2- 172.21.15.112:0/3479498213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73f01056e0 0x7f73f0105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:03.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.395+0000 7f73f5800700 1 -- 172.21.15.112:0/3479498213 >> 172.21.15.112:0/3479498213 conn(0x7f73f01011d0 msgr2=0x7f73f01035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:03.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.395+0000 7f73f5800700 1 -- 172.21.15.112:0/3479498213 shutdown_connections 2024-05-08T06:05:03.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.395+0000 7f73f5800700 1 -- 172.21.15.112:0/3479498213 wait complete. 2024-05-08T06:05:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.397+0000 7f73f5800700 1 Processor -- start 2024-05-08T06:05:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.397+0000 7f73f5800700 1 -- start start 2024-05-08T06:05:03.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.397+0000 7f73f5800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73f01056e0 0x7f73f019be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.397+0000 7f73f5800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f01060a0 0x7f73f019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.397+0000 7f73f5800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73f010b6e0 0x7f73f01a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.397+0000 7f73f5800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0115930 con 0x7f73f01060a0 2024-05-08T06:05:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.397+0000 7f73f5800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f73f01157b0 con 0x7f73f01056e0 2024-05-08T06:05:03.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.397+0000 7f73f5800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0115ab0 con 0x7f73f010b6e0 2024-05-08T06:05:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.398+0000 7f73ee000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f01060a0 0x7f73f019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.398+0000 7f73ef400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73f010b6e0 0x7f73f01a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:03.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.398+0000 7f73ef400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73f010b6e0 0x7f73f01a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:35576/0 (socket says 172.21.15.112:35576) 2024-05-08T06:05:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.398+0000 7f73ef400700 1 -- 172.21.15.112:0/4293354123 learned_addr learned my addr 172.21.15.112:0/4293354123 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:05:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.398+0000 7f73eea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73f01056e0 0x7f73f019be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.398+0000 7f73ef400700 1 -- 172.21.15.112:0/4293354123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73f01056e0 msgr2=0x7f73f019be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.398+0000 7f73ef400700 1 --2- 172.21.15.112:0/4293354123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73f01056e0 0x7f73f019be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.398+0000 7f73ef400700 1 -- 172.21.15.112:0/4293354123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f01060a0 msgr2=0x7f73f019c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.398+0000 7f73ef400700 1 --2- 172.21.15.112:0/4293354123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f01060a0 0x7f73f019c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.398+0000 7f73ef400700 1 -- 172.21.15.112:0/4293354123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f01a0e70 con 0x7f73f010b6e0 2024-05-08T06:05:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.398+0000 7f73ee000700 1 --2- 172.21.15.112:0/4293354123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f01060a0 0x7f73f019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:05:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.399+0000 7f73ef400700 1 --2- 172.21.15.112:0/4293354123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73f010b6e0 0x7f73f01a0730 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f73e800e3c0 tx=0x7f73e800e6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.399+0000 7f73e7400700 1 -- 172.21.15.112:0/4293354123 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e80090d0 con 0x7f73f010b6e0 2024-05-08T06:05:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.399+0000 7f73e7400700 1 -- 172.21.15.112:0/4293354123 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e8009270 con 0x7f73f010b6e0 2024-05-08T06:05:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.399+0000 7f73f5800700 1 -- 172.21.15.112:0/4293354123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f01a1160 con 0x7f73f010b6e0 2024-05-08T06:05:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.399+0000 7f73e7400700 1 -- 172.21.15.112:0/4293354123 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8007df0 con 0x7f73f010b6e0 2024-05-08T06:05:03.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.399+0000 7f73f5800700 1 -- 172.21.15.112:0/4293354123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f01a89e0 con 0x7f73f010b6e0 2024-05-08T06:05:03.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.400+0000 7f73f5800700 1 -- 172.21.15.112:0/4293354123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f006a190 con 0x7f73f010b6e0 2024-05-08T06:05:03.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.402+0000 7f73e7400700 1 -- 172.21.15.112:0/4293354123 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f73e8012050 con 0x7f73f010b6e0 2024-05-08T06:05:03.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.404+0000 7f73e7400700 1 --2- 172.21.15.112:0/4293354123 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f73d40b2880 0x7f73d40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:03.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.404+0000 7f73e7400700 1 -- 172.21.15.112:0/4293354123 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7f73e80e1cf0 con 0x7f73f010b6e0 2024-05-08T06:05:03.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.404+0000 7f73eea00700 1 --2- 172.21.15.112:0/4293354123 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f73d40b2880 0x7f73d40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:03.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.405+0000 7f73eea00700 1 --2- 172.21.15.112:0/4293354123 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f73d40b2880 0x7f73d40b4d40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f73e00098c0 tx=0x7f73e0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:03.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.408+0000 7f73e7400700 1 -- 172.21.15.112:0/4293354123 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f73e8022060 con 0x7f73f010b6e0 2024-05-08T06:05:03.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:02 smithi150 ceph-mon[34315]: pgmap v733: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:05:03.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.645+0000 7f73f5800700 1 -- 172.21.15.112:0/4293354123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73f002e510 con 0x7f73f010b6e0 2024-05-08T06:05:03.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.647+0000 7f73e7400700 1 -- 172.21.15.112:0/4293354123 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22829 (secure 0 0 0) 0x7f73e8010040 con 0x7f73f010b6e0 2024-05-08T06:05:03.648 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-05-08T06:05:03.648 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":84,"fsid":"7a07ff60-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:48.498871+0000","modified":"2024-05-08T05:53:05.251932+0000","last_up_change":"2024-05-08T05:49:54.039380+0000","last_in_change":"2024-05-08T05:49:34.443850+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:47.485105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:50:58.918909+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.175022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b378542-1bbe-4614-8805-8a6b875ada55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6803","nonce":3419091820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6805","nonce":3419091820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6809","nonce":3419091820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3419091820},{"type":"v1","addr":"172.21.15.112:6807","nonce":3419091820}]},"public_addr":"172.21.15.112:6803/3419091820","cluster_addr":"172.21.15.112:6805/3419091820","heartbeat_back_addr":"172.21.15.112:6809/3419091820","heartbeat_front_addr":"172.21.15.112:6807/3419091820","state":["exists","up"]},{"osd":1,"uuid":"826ef036-2db7-470e-9956-d9a58b9181e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6801","nonce":2728419797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6803","nonce":2728419797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6807","nonce":2728419797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2728419797},{"type":"v1","addr":"172.21.15.150:6805","nonce":2728419797}]},"public_addr":"172.21.15.150:6801/2728419797","cluster_addr":"172.21.15.150:6803/2728419797","heartbeat_back_addr":"172.21.15.150:6807/2728419797","heartbeat_front_addr":"172.21.15.150:6805/2728419797","state":["exists","up"]},{"osd":2,"uuid":"32e710e7-2a87-4863-bc7a-906d31aa4ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6801","nonce":1556203215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6803","nonce":1556203215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6807","nonce":1556203215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1556203215},{"type":"v1","addr":"172.21.15.162:6805","nonce":1556203215}]},"public_addr":"172.21.15.162:6801/1556203215","cluster_addr":"172.21.15.162:6803/1556203215","heartbeat_back_addr":"172.21.15.162:6807/1556203215","heartbeat_front_addr":"172.21.15.162:6805/1556203215","state":["exists","up"]},{"osd":3,"uuid":"ba1d4a5f-6a95-4b9e-8a48-8524a899b004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6811","nonce":3325718755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6813","nonce":3325718755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6817","nonce":3325718755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3325718755},{"type":"v1","addr":"172.21.15.112:6815","nonce":3325718755}]},"public_addr":"172.21.15.112:6811/3325718755","cluster_addr":"172.21.15.112:6813/3325718755","heartbeat_back_addr":"172.21.15.112:6817/3325718755","heartbeat_front_addr":"172.21.15.112:6815/3325718755","state":["exists","up"]},{"osd":4,"uuid":"cb4f1a42-8de8-4b50-b3bc-602de9f4803a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6809","nonce":1548914683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6811","nonce":1548914683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6815","nonce":1548914683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1548914683},{"type":"v1","addr":"172.21.15.150:6813","nonce":1548914683}]},"public_addr":"172.21.15.150:6809/1548914683","cluster_addr":"172.21.15.150:6811/1548914683","heartbeat_back_addr":"172.21.15.150:6815/1548914683","heartbeat_front_addr":"172.21.15.150:6813/1548914683","state":["exists","up"]},{"osd":5,"uuid":"a26c03b8-050c-4a2c-8f23-5801fffb7c78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6809","nonce":409836750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6811","nonce":409836750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6815","nonce":409836750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":409836750},{"type":"v1","addr":"172.21.15.162:6813","nonce":409836750}]},"public_addr":"172.21.15.162:6809/409836750","cluster_addr":"172.21.15.162:6811/409836750","heartbeat_back_addr":"172.21.15.162:6815/409836750","heartbeat_front_addr":"172.21.15.162:6813/409836750","state":["exists","up"]},{"osd":6,"uuid":"8ca7ad7f-ffbe-4010-95fe-a0b8eb0eb4a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6819","nonce":1098984186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6821","nonce":1098984186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6825","nonce":1098984186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1098984186},{"type":"v1","addr":"172.21.15.112:6823","nonce":1098984186}]},"public_addr":"172.21.15.112:6819/1098984186","cluster_addr":"172.21.15.112:6821/1098984186","heartbeat_back_addr":"172.21.15.112:6825/1098984186","heartbeat_front_addr":"172.21.15.112:6823/1098984186","state":["exists","up"]},{"osd":7,"uuid":"af3629b4-9bed-4669-97a6-48deba503a96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6817","nonce":4009836564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6819","nonce":4009836564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6823","nonce":4009836564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4009836564},{"type":"v1","addr":"172.21.15.150:6821","nonce":4009836564}]},"public_addr":"172.21.15.150:6817/4009836564","cluster_addr":"172.21.15.150:6819/4009836564","heartbeat_back_addr":"172.21.15.150:6823/4009836564","heartbeat_front_addr":"172.21.15.150:6821/4009836564","state":["exists","up"]},{"osd":8,"uuid":"9333e87d-dd78-4467-bbde-c2b44f4ea07f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6817","nonce":852791606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6819","nonce":852791606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6823","nonce":852791606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":852791606},{"type":"v1","addr":"172.21.15.162:6821","nonce":852791606}]},"public_addr":"172.21.15.162:6817/852791606","cluster_addr":"172.21.15.162:6819/852791606","heartbeat_back_addr":"172.21.15.162:6823/852791606","heartbeat_front_addr":"172.21.15.162:6821/852791606","state":["exists","up"]},{"osd":9,"uuid":"09723581-1995-42e0-8db7-776ea98a918c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6827","nonce":2186764674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6829","nonce":2186764674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6833","nonce":2186764674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2186764674},{"type":"v1","addr":"172.21.15.112:6831","nonce":2186764674}]},"public_addr":"172.21.15.112:6827/2186764674","cluster_addr":"172.21.15.112:6829/2186764674","heartbeat_back_addr":"172.21.15.112:6833/2186764674","heartbeat_front_addr":"172.21.15.112:6831/2186764674","state":["exists","up"]},{"osd":10,"uuid":"34e08e3e-1231-4899-a5b3-76d449105bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6825","nonce":2838995474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6827","nonce":2838995474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6831","nonce":2838995474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2838995474},{"type":"v1","addr":"172.21.15.150:6829","nonce":2838995474}]},"public_addr":"172.21.15.150:6825/2838995474","cluster_addr":"172.21.15.150:6827/2838995474","heartbeat_back_addr":"172.21.15.150:6831/2838995474","heartbeat_front_addr":"172.21.15.150:6829/2838995474","state":["exists","up"]},{"osd":11,"uuid":"6fc34ec0-78ca-403c-acb7-b29d83ff64e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6825","nonce":3702614331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6827","nonce":3702614331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6831","nonce":3702614331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3702614331},{"type":"v1","addr":"172.21.15.162:6829","nonce":3702614331}]},"public_addr":"172.21.15.162:6825/3702614331","cluster_addr":"172.21.15.162:6827/3702614331","heartbeat_back_addr":"172.21.15.162:6831/3702614331","heartbeat_front_addr":"172.21.15.162:6829/3702614331","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:54.518702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:19.939543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:44.425586+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:16.126471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:41.696071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:05.517405+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:38.549318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:03.799649+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:28.007282+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:01.395459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:27.963637+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:49:52.181212+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3667148497":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6800/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3425327158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1404611205":"2024-05-09T05:43:03.313546+0000","172.21.15.112:0/3765334043":"2024-05-09T05:42:29.819307+0000","172.21.15.112:6800/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/2814303790":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/164150456":"2024-05-09T05:43:03.313546+0000","172.21.15.112:6801/772108158":"2024-05-09T05:42:29.819307+0000","172.21.15.112:0/1091751583":"2024-05-09T05:42:29.819307+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:05:03.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.650+0000 7f73f5800700 1 -- 172.21.15.112:0/4293354123 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f73d40b2880 msgr2=0x7f73d40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:03.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.650+0000 7f73f5800700 1 --2- 172.21.15.112:0/4293354123 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f73d40b2880 0x7f73d40b4d40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f73e00098c0 tx=0x7f73e0019040 comp rx=0 tx=0).stop 2024-05-08T06:05:03.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.650+0000 7f73f5800700 1 -- 172.21.15.112:0/4293354123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73f010b6e0 msgr2=0x7f73f01a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:03.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.650+0000 7f73f5800700 1 --2- 172.21.15.112:0/4293354123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73f010b6e0 0x7f73f01a0730 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f73e800e3c0 tx=0x7f73e800e6d0 comp rx=0 tx=0).stop 2024-05-08T06:05:03.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.650+0000 7f73f5800700 1 -- 172.21.15.112:0/4293354123 shutdown_connections 2024-05-08T06:05:03.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.650+0000 7f73f5800700 1 --2- 172.21.15.112:0/4293354123 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f73d40b2880 0x7f73d40b4d40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:03.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.650+0000 7f73f5800700 1 --2- 172.21.15.112:0/4293354123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73f010b6e0 0x7f73f01a0730 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:03.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.650+0000 7f73f5800700 1 --2- 172.21.15.112:0/4293354123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73f01056e0 0x7f73f019be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:03.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.650+0000 7f73f5800700 1 --2- 172.21.15.112:0/4293354123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73f01060a0 0x7f73f019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:03.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.650+0000 7f73f5800700 1 -- 172.21.15.112:0/4293354123 >> 172.21.15.112:0/4293354123 conn(0x7f73f01011d0 msgr2=0x7f73f01019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:03.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.651+0000 7f73f5800700 1 -- 172.21.15.112:0/4293354123 shutdown_connections 2024-05-08T06:05:03.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:03.651+0000 7f73f5800700 1 -- 172.21.15.112:0/4293354123 wait complete. 2024-05-08T06:05:04.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:03 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/4293354123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:05:04.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:03 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/4293354123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:05:04.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:03 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/4293354123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:05:04.681 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-08T06:05:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:05.295 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:04 smithi112 ceph-mon[30003]: pgmap v734: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:05:05.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:04 smithi162 ceph-mon[34800]: pgmap v734: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:05:05.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:04 smithi150 ceph-mon[34315]: pgmap v734: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:05:07.229 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:06 smithi112 ceph-mon[30003]: pgmap v735: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-08T06:05:07.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:06 smithi162 ceph-mon[34800]: pgmap v735: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-08T06:05:07.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:06 smithi150 ceph-mon[34315]: pgmap v735: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-08T06:05:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.459+0000 7ffa3ec00700 1 -- 172.21.15.112:0/260725196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa3810fb60 msgr2=0x7ffa3810ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.459+0000 7ffa3ec00700 1 --2- 172.21.15.112:0/260725196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa3810fb60 0x7ffa3810ff40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009990 tx=0x7ffa28009ca0 comp rx=0 tx=0).stop 2024-05-08T06:05:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.459+0000 7ffa3ec00700 1 -- 172.21.15.112:0/260725196 shutdown_connections 2024-05-08T06:05:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.459+0000 7ffa3ec00700 1 --2- 172.21.15.112:0/260725196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa3810a490 0x7ffa381169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.459+0000 7ffa3ec00700 1 --2- 172.21.15.112:0/260725196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa3810fb60 0x7ffa3810ff40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:07.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.459+0000 7ffa3ec00700 1 --2- 172.21.15.112:0/260725196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa38109ad0 0x7ffa38109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.459+0000 7ffa3ec00700 1 -- 172.21.15.112:0/260725196 >> 172.21.15.112:0/260725196 conn(0x7ffa38076c70 msgr2=0x7ffa381082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.459+0000 7ffa3ec00700 1 -- 172.21.15.112:0/260725196 shutdown_connections 2024-05-08T06:05:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.460+0000 7ffa3ec00700 1 -- 172.21.15.112:0/260725196 wait complete. 2024-05-08T06:05:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.460+0000 7ffa3ec00700 1 Processor -- start 2024-05-08T06:05:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.461+0000 7ffa3ec00700 1 -- start start 2024-05-08T06:05:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.461+0000 7ffa3ec00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa38109ad0 0x7ffa381a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.461+0000 7ffa3ec00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa3810a490 0x7ffa381a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.461+0000 7ffa3ec00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa3810fb60 0x7ffa381a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.461+0000 7ffa3ec00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa38119cc0 con 0x7ffa38109ad0 2024-05-08T06:05:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.461+0000 7ffa3ec00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffa38119b40 con 0x7ffa3810a490 2024-05-08T06:05:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.461+0000 7ffa3ec00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ffa38119e40 con 0x7ffa3810fb60 2024-05-08T06:05:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.461+0000 7ffa37e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa38109ad0 0x7ffa381a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.461+0000 7ffa37e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa38109ad0 0x7ffa381a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57512/0 (socket says 172.21.15.112:57512) 2024-05-08T06:05:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.461+0000 7ffa37e00700 1 -- 172.21.15.112:0/1943631628 learned_addr learned my addr 172.21.15.112:0/1943631628 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:05:07.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.461+0000 7ffa37400700 1 --2- 172.21.15.112:0/1943631628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa3810a490 0x7ffa381a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.461+0000 7ffa3cc00700 1 --2- 172.21.15.112:0/1943631628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa3810fb60 0x7ffa381a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.462+0000 7ffa37400700 1 -- 172.21.15.112:0/1943631628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa3810fb60 msgr2=0x7ffa381a4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.462+0000 7ffa37400700 1 --2- 172.21.15.112:0/1943631628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa3810fb60 0x7ffa381a4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.462+0000 7ffa37400700 1 -- 172.21.15.112:0/1943631628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa38109ad0 msgr2=0x7ffa381a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.462+0000 7ffa37400700 1 --2- 172.21.15.112:0/1943631628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa38109ad0 0x7ffa381a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.462+0000 7ffa37400700 1 -- 172.21.15.112:0/1943631628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa381a5170 con 0x7ffa3810a490 2024-05-08T06:05:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.462+0000 7ffa37400700 1 --2- 172.21.15.112:0/1943631628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa3810a490 0x7ffa381a06f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400e9f0 tx=0x7ffa2400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.462+0000 7ffa34c00700 1 -- 172.21.15.112:0/1943631628 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2400c9c0 con 0x7ffa3810a490 2024-05-08T06:05:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.462+0000 7ffa3ec00700 1 -- 172.21.15.112:0/1943631628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa381a5460 con 0x7ffa3810a490 2024-05-08T06:05:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.463+0000 7ffa34c00700 1 -- 172.21.15.112:0/1943631628 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa2400cb60 con 0x7ffa3810a490 2024-05-08T06:05:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.463+0000 7ffa34c00700 1 -- 172.21.15.112:0/1943631628 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24018680 con 0x7ffa3810a490 2024-05-08T06:05:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.463+0000 7ffa3ec00700 1 -- 172.21.15.112:0/1943631628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa381acce0 con 0x7ffa3810a490 2024-05-08T06:05:07.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.464+0000 7ffa3ec00700 1 -- 172.21.15.112:0/1943631628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3806a190 con 0x7ffa3810a490 2024-05-08T06:05:07.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.467+0000 7ffa34c00700 1 -- 172.21.15.112:0/1943631628 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ffa24010430 con 0x7ffa3810a490 2024-05-08T06:05:07.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.468+0000 7ffa34c00700 1 --2- 172.21.15.112:0/1943631628 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffa1c0b2900 0x7ffa1c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:07.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.468+0000 7ffa34c00700 1 -- 172.21.15.112:0/1943631628 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8277+0+0 (secure 0 0 0) 0x7ffa240e26f0 con 0x7ffa3810a490 2024-05-08T06:05:07.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.469+0000 7ffa37e00700 1 --2- 172.21.15.112:0/1943631628 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffa1c0b2900 0x7ffa1c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:07.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.469+0000 7ffa37e00700 1 --2- 172.21.15.112:0/1943631628 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffa1c0b2900 0x7ffa1c0b4dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ffa28005cd0 tx=0x7ffa28005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:07.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.470+0000 7ffa34c00700 1 -- 172.21.15.112:0/1943631628 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffa240aa380 con 0x7ffa3810a490 2024-05-08T06:05:07.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.749+0000 7ffa3ec00700 1 -- 172.21.15.112:0/1943631628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ffa3802e510 con 0x7ffa3810a490 2024-05-08T06:05:07.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.759+0000 7ffa34c00700 1 -- 172.21.15.112:0/1943631628 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7ffa240b2400 con 0x7ffa3810a490 2024-05-08T06:05:07.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.762+0000 7ffa3ec00700 1 -- 172.21.15.112:0/1943631628 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffa1c0b2900 msgr2=0x7ffa1c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:07.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.762+0000 7ffa3ec00700 1 --2- 172.21.15.112:0/1943631628 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffa1c0b2900 0x7ffa1c0b4dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ffa28005cd0 tx=0x7ffa28005c60 comp rx=0 tx=0).stop 2024-05-08T06:05:07.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.763+0000 7ffa3ec00700 1 -- 172.21.15.112:0/1943631628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa3810a490 msgr2=0x7ffa381a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:07.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.763+0000 7ffa3ec00700 1 --2- 172.21.15.112:0/1943631628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa3810a490 0x7ffa381a06f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400e9f0 tx=0x7ffa2400ed00 comp rx=0 tx=0).stop 2024-05-08T06:05:07.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.763+0000 7ffa3ec00700 1 -- 172.21.15.112:0/1943631628 shutdown_connections 2024-05-08T06:05:07.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.763+0000 7ffa3ec00700 1 --2- 172.21.15.112:0/1943631628 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7ffa1c0b2900 0x7ffa1c0b4dc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:07.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.763+0000 7ffa3ec00700 1 --2- 172.21.15.112:0/1943631628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa3810fb60 0x7ffa381a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:07.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.763+0000 7ffa3ec00700 1 --2- 172.21.15.112:0/1943631628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa38109ad0 0x7ffa381a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:07.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.763+0000 7ffa3ec00700 1 --2- 172.21.15.112:0/1943631628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa3810a490 0x7ffa381a06f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:07.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.763+0000 7ffa3ec00700 1 -- 172.21.15.112:0/1943631628 >> 172.21.15.112:0/1943631628 conn(0x7ffa38076c70 msgr2=0x7ffa380772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:07.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.763+0000 7ffa3ec00700 1 -- 172.21.15.112:0/1943631628 shutdown_connections 2024-05-08T06:05:07.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:07.763+0000 7ffa3ec00700 1 -- 172.21.15.112:0/1943631628 wait complete. 2024-05-08T06:05:07.765 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T06:05:08.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:07 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a[29999]: 2024-05-08T06:05:07.751+0000 7fb71cc00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T06:05:08.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:07 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1943631628' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T06:05:08.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:07 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T06:05:08.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:07 smithi112 ceph-mon[30003]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T06:05:08.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:07 smithi112 ceph-mon[30003]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T06:05:08.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:07 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T06:05:08.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:07 smithi112 ceph-mon[30003]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T06:05:08.067 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:07 smithi112 ceph-mon[30003]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-08T06:05:08.067 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 06:05:07 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-a[69856]: ignoring --setuser ceph since I am not root 2024-05-08T06:05:08.067 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 06:05:07 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-a[69856]: ignoring --setgroup ceph since I am not root 2024-05-08T06:05:08.067 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 06:05:07 smithi112 ceph-mds[69860]: -- 172.21.15.112:0/140549776 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563fe28f9a40 con 0x563fe28cc800 2024-05-08T06:05:08.067 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 06:05:07 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-a[69856]: starting mds.a at 2024-05-08T06:05:08.068 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 06:05:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-i[65579]: ignoring --setuser ceph since I am not root 2024-05-08T06:05:08.068 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 06:05:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-i[65579]: ignoring --setgroup ceph since I am not root 2024-05-08T06:05:08.068 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 06:05:07 smithi162 ceph-mds[65583]: -- 172.21.15.162:0/3169053543 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56422060da40 con 0x5642205e1000 2024-05-08T06:05:08.068 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 06:05:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-i[65579]: starting mds.i at 2024-05-08T06:05:08.068 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 06:05:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-f[64961]: ignoring --setuser ceph since I am not root 2024-05-08T06:05:08.068 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 06:05:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-f[64961]: ignoring --setgroup ceph since I am not root 2024-05-08T06:05:08.068 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 06:05:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-f[64961]: starting mds.f at 2024-05-08T06:05:08.068 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:07 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1943631628' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T06:05:08.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:07 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T06:05:08.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:07 smithi162 ceph-mon[34800]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T06:05:08.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:07 smithi162 ceph-mon[34800]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T06:05:08.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:07 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T06:05:08.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:07 smithi162 ceph-mon[34800]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T06:05:08.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:07 smithi162 ceph-mon[34800]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-08T06:05:08.069 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 06:05:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-c[65272]: ignoring --setuser ceph since I am not root 2024-05-08T06:05:08.069 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 06:05:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-c[65272]: ignoring --setgroup ceph since I am not root 2024-05-08T06:05:08.069 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 06:05:07 smithi162 ceph-mds[65276]: -- 172.21.15.162:0/2137725109 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c868b19a40 con 0x55c868aecc00 2024-05-08T06:05:08.069 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 06:05:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-c[65272]: starting mds.c at 2024-05-08T06:05:08.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:07 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1943631628' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T06:05:08.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:07 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T06:05:08.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:07 smithi150 ceph-mon[34315]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T06:05:08.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:07 smithi150 ceph-mon[34315]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T06:05:08.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:07 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T06:05:08.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:07 smithi150 ceph-mon[34315]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T06:05:08.184 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:07 smithi150 ceph-mon[34315]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-08T06:05:08.184 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 06:05:07 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-b[64165]: ignoring --setuser ceph since I am not root 2024-05-08T06:05:08.184 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 06:05:07 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-b[64165]: ignoring --setgroup ceph since I am not root 2024-05-08T06:05:08.184 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 06:05:07 smithi150 ceph-mds[64169]: -- 172.21.15.150:0/3374243922 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636cfa45a40 con 0x5636cfa18c00 2024-05-08T06:05:08.184 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 06:05:07 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-b[64165]: starting mds.b at 2024-05-08T06:05:08.185 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 06:05:07 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-h[63857]: ignoring --setuser ceph since I am not root 2024-05-08T06:05:08.185 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 06:05:07 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-h[63857]: ignoring --setgroup ceph since I am not root 2024-05-08T06:05:08.185 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 06:05:07 smithi150 ceph-mds[63861]: -- 172.21.15.150:0/1069087259 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557049fada40 con 0x557049f80800 2024-05-08T06:05:08.185 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 06:05:07 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-h[63857]: starting mds.h at 2024-05-08T06:05:08.603 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T06:05:09.029 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:09.300 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:08 smithi112 ceph-mon[30003]: pgmap v737: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T06:05:09.300 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:08 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.162:6834/2128116543,v1:172.21.15.162:6835/2128116543] up:boot 2024-05-08T06:05:09.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:08 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.162:6837/3937093611,v1:172.21.15.162:6838/3937093611] up:boot 2024-05-08T06:05:09.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:08 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.162:6836/4164373803,v1:172.21.15.162:6839/4164373803] up:boot 2024-05-08T06:05:09.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:08 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.150:6836/3346220430,v1:172.21.15.150:6838/3346220430] up:boot 2024-05-08T06:05:09.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:08 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.150:6837/3866775343,v1:172.21.15.150:6839/3866775343] up:boot 2024-05-08T06:05:09.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:08 smithi112 ceph-mon[30003]: mds.? [v2:172.21.15.112:6840/2626085871,v1:172.21.15.112:6841/2626085871] up:boot 2024-05-08T06:05:09.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:08 smithi112 ceph-mon[30003]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-08T06:05:09.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T06:05:09.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T06:05:09.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T06:05:09.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T06:05:09.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T06:05:09.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:08 smithi112 ceph-mon[30003]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T06:05:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:08 smithi162 ceph-mon[34800]: pgmap v737: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T06:05:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:08 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.162:6834/2128116543,v1:172.21.15.162:6835/2128116543] up:boot 2024-05-08T06:05:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:08 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.162:6837/3937093611,v1:172.21.15.162:6838/3937093611] up:boot 2024-05-08T06:05:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:08 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.162:6836/4164373803,v1:172.21.15.162:6839/4164373803] up:boot 2024-05-08T06:05:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:08 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.150:6836/3346220430,v1:172.21.15.150:6838/3346220430] up:boot 2024-05-08T06:05:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:08 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.150:6837/3866775343,v1:172.21.15.150:6839/3866775343] up:boot 2024-05-08T06:05:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:08 smithi162 ceph-mon[34800]: mds.? [v2:172.21.15.112:6840/2626085871,v1:172.21.15.112:6841/2626085871] up:boot 2024-05-08T06:05:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:08 smithi162 ceph-mon[34800]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-08T06:05:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T06:05:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T06:05:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T06:05:09.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T06:05:09.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T06:05:09.318 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:08 smithi162 ceph-mon[34800]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T06:05:09.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:08 smithi150 ceph-mon[34315]: pgmap v737: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T06:05:09.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:08 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.162:6834/2128116543,v1:172.21.15.162:6835/2128116543] up:boot 2024-05-08T06:05:09.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:08 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.162:6837/3937093611,v1:172.21.15.162:6838/3937093611] up:boot 2024-05-08T06:05:09.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:08 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.162:6836/4164373803,v1:172.21.15.162:6839/4164373803] up:boot 2024-05-08T06:05:09.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:08 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.150:6836/3346220430,v1:172.21.15.150:6838/3346220430] up:boot 2024-05-08T06:05:09.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:08 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.150:6837/3866775343,v1:172.21.15.150:6839/3866775343] up:boot 2024-05-08T06:05:09.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:08 smithi150 ceph-mon[34315]: mds.? [v2:172.21.15.112:6840/2626085871,v1:172.21.15.112:6841/2626085871] up:boot 2024-05-08T06:05:09.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:08 smithi150 ceph-mon[34315]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-08T06:05:09.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T06:05:09.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T06:05:09.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T06:05:09.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T06:05:09.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T06:05:09.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:08 smithi150 ceph-mon[34315]: from='mgr.14152 172.21.15.112:0/2408120985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T06:05:11.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.167+0000 7f3f36800700 1 -- 172.21.15.112:0/162154299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f30103750 msgr2=0x7f3f30103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:11.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.167+0000 7f3f36800700 1 --2- 172.21.15.112:0/162154299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f30103750 0x7f3f30103b30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3f200099c0 tx=0x7f3f20009cd0 comp rx=0 tx=0).stop 2024-05-08T06:05:11.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.168+0000 7f3f36800700 1 -- 172.21.15.112:0/162154299 shutdown_connections 2024-05-08T06:05:11.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.168+0000 7f3f36800700 1 --2- 172.21.15.112:0/162154299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f30108850 0x7f3f301103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:11.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.168+0000 7f3f36800700 1 --2- 172.21.15.112:0/162154299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f30103750 0x7f3f30103b30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:11.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.168+0000 7f3f36800700 1 --2- 172.21.15.112:0/162154299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f30104100 0x7f3f30108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:11.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.168+0000 7f3f36800700 1 -- 172.21.15.112:0/162154299 >> 172.21.15.112:0/162154299 conn(0x7f3f3007a600 msgr2=0x7f3f3007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:11.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.168+0000 7f3f36800700 1 -- 172.21.15.112:0/162154299 shutdown_connections 2024-05-08T06:05:11.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.168+0000 7f3f36800700 1 -- 172.21.15.112:0/162154299 wait complete. 2024-05-08T06:05:11.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.169+0000 7f3f36800700 1 Processor -- start 2024-05-08T06:05:11.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.169+0000 7f3f36800700 1 -- start start 2024-05-08T06:05:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.169+0000 7f3f36800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f30103750 0x7f3f3019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.169+0000 7f3f36800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f30104100 0x7f3f3019c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.170+0000 7f3f36800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f30108850 0x7f3f301a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:11.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.170+0000 7f3f36800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30113670 con 0x7f3f30103750 2024-05-08T06:05:11.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.170+0000 7f3f36800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3f301134f0 con 0x7f3f30108850 2024-05-08T06:05:11.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.170+0000 7f3f36800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3f301137f0 con 0x7f3f30104100 2024-05-08T06:05:11.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.170+0000 7f3f2f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f30103750 0x7f3f3019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:11.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.170+0000 7f3f2f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f30103750 0x7f3f3019bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57548/0 (socket says 172.21.15.112:57548) 2024-05-08T06:05:11.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.170+0000 7f3f2f400700 1 -- 172.21.15.112:0/1135505496 learned_addr learned my addr 172.21.15.112:0/1135505496 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:05:11.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.170+0000 7f3f27e00700 1 --2- 172.21.15.112:0/1135505496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f30104100 0x7f3f3019c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:11.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.170+0000 7f3f2fe00700 1 --2- 172.21.15.112:0/1135505496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f30108850 0x7f3f301a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:11.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.170+0000 7f3f2f400700 1 -- 172.21.15.112:0/1135505496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f30104100 msgr2=0x7f3f3019c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:11.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.170+0000 7f3f2f400700 1 --2- 172.21.15.112:0/1135505496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f30104100 0x7f3f3019c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:11.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.170+0000 7f3f2f400700 1 -- 172.21.15.112:0/1135505496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f30108850 msgr2=0x7f3f301a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:11.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.170+0000 7f3f2f400700 1 --2- 172.21.15.112:0/1135505496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f30108850 0x7f3f301a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:11.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.170+0000 7f3f2f400700 1 -- 172.21.15.112:0/1135505496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f301a0d30 con 0x7f3f30103750 2024-05-08T06:05:11.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.170+0000 7f3f27e00700 1 --2- 172.21.15.112:0/1135505496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f30104100 0x7f3f3019c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:05:11.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.171+0000 7f3f2f400700 1 --2- 172.21.15.112:0/1135505496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f30103750 0x7f3f3019bd20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3f20005910 tx=0x7f3f20004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:11.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.171+0000 7f3f2cc00700 1 -- 172.21.15.112:0/1135505496 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20026070 con 0x7f3f30103750 2024-05-08T06:05:11.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.171+0000 7f3f2cc00700 1 -- 172.21.15.112:0/1135505496 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f20003c70 con 0x7f3f30103750 2024-05-08T06:05:11.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.171+0000 7f3f2cc00700 1 -- 172.21.15.112:0/1135505496 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2000fa40 con 0x7f3f30103750 2024-05-08T06:05:11.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.171+0000 7f3f36800700 1 -- 172.21.15.112:0/1135505496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f301a0fc0 con 0x7f3f30103750 2024-05-08T06:05:11.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.171+0000 7f3f36800700 1 -- 172.21.15.112:0/1135505496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f301a88a0 con 0x7f3f30103750 2024-05-08T06:05:11.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.172+0000 7f3f36800700 1 -- 172.21.15.112:0/1135505496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f3006a190 con 0x7f3f30103750 2024-05-08T06:05:11.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.175+0000 7f3f2cc00700 1 -- 172.21.15.112:0/1135505496 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f3f20029070 con 0x7f3f30103750 2024-05-08T06:05:11.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.176+0000 7f3f2cc00700 1 --2- 172.21.15.112:0/1135505496 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3f140b2900 0x7f3f140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:11.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.176+0000 7f3f2cc00700 1 -- 172.21.15.112:0/1135505496 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8535+0+0 (secure 0 0 0) 0x7f3f200e2b10 con 0x7f3f30103750 2024-05-08T06:05:11.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.177+0000 7f3f27e00700 1 --2- 172.21.15.112:0/1135505496 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3f140b2900 0x7f3f140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:11.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.177+0000 7f3f27e00700 1 --2- 172.21.15.112:0/1135505496 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3f140b2900 0x7f3f140b4dc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3f301031f0 tx=0x7f3f18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:11.181 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:10 smithi112 ceph-mon[30003]: pgmap v738: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T06:05:11.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.181+0000 7f3f2cc00700 1 -- 172.21.15.112:0/1135505496 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3f2005edf0 con 0x7f3f30103750 2024-05-08T06:05:11.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:10 smithi162 ceph-mon[34800]: pgmap v738: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T06:05:11.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:10 smithi150 ceph-mon[34315]: pgmap v738: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T06:05:11.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.461+0000 7f3f36800700 1 -- 172.21.15.112:0/1135505496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3f301a8a70 con 0x7f3f30103750 2024-05-08T06:05:11.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.987+0000 7f3f2cc00700 1 -- 172.21.15.112:0/1135505496 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7f3f20013070 con 0x7f3f30103750 2024-05-08T06:05:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.991+0000 7f3f36800700 1 -- 172.21.15.112:0/1135505496 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3f140b2900 msgr2=0x7f3f140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.991+0000 7f3f36800700 1 --2- 172.21.15.112:0/1135505496 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3f140b2900 0x7f3f140b4dc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3f301031f0 tx=0x7f3f18009260 comp rx=0 tx=0).stop 2024-05-08T06:05:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.991+0000 7f3f36800700 1 -- 172.21.15.112:0/1135505496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f30103750 msgr2=0x7f3f3019bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:11.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.991+0000 7f3f36800700 1 --2- 172.21.15.112:0/1135505496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f30103750 0x7f3f3019bd20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3f20005910 tx=0x7f3f20004270 comp rx=0 tx=0).stop 2024-05-08T06:05:11.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.992+0000 7f3f36800700 1 -- 172.21.15.112:0/1135505496 shutdown_connections 2024-05-08T06:05:11.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.992+0000 7f3f36800700 1 --2- 172.21.15.112:0/1135505496 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7f3f140b2900 0x7f3f140b4dc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:11.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.992+0000 7f3f36800700 1 --2- 172.21.15.112:0/1135505496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f30108850 0x7f3f301a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:11.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.992+0000 7f3f36800700 1 --2- 172.21.15.112:0/1135505496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f30103750 0x7f3f3019bd20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:11.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.992+0000 7f3f36800700 1 --2- 172.21.15.112:0/1135505496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f30104100 0x7f3f3019c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:11.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.992+0000 7f3f36800700 1 -- 172.21.15.112:0/1135505496 >> 172.21.15.112:0/1135505496 conn(0x7f3f3007a600 msgr2=0x7f3f301018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:11.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.992+0000 7f3f36800700 1 -- 172.21.15.112:0/1135505496 shutdown_connections 2024-05-08T06:05:11.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:11.992+0000 7f3f36800700 1 -- 172.21.15.112:0/1135505496 wait complete. 2024-05-08T06:05:12.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:11 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1135505496' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:05:12.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:11 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1135505496' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:05:12.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:11 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1135505496' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:05:12.940 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T06:05:13.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:12 smithi112 ceph-mon[30003]: pgmap v739: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-08T06:05:13.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:12 smithi112 ceph-mon[30003]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T06:05:13.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:12 smithi112 ceph-mon[30003]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T06:05:13.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:12 smithi112 ceph-mon[30003]: Cluster is now healthy 2024-05-08T06:05:13.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:12 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/1135505496' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T06:05:13.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:12 smithi112 ceph-mon[30003]: fsmap 12 up:standby 2024-05-08T06:05:13.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:12 smithi162 ceph-mon[34800]: pgmap v739: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-08T06:05:13.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:12 smithi162 ceph-mon[34800]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T06:05:13.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:12 smithi162 ceph-mon[34800]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T06:05:13.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:12 smithi162 ceph-mon[34800]: Cluster is now healthy 2024-05-08T06:05:13.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:12 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/1135505496' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T06:05:13.317 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:12 smithi162 ceph-mon[34800]: fsmap 12 up:standby 2024-05-08T06:05:13.379 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:13.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:12 smithi150 ceph-mon[34315]: pgmap v739: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-08T06:05:13.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:12 smithi150 ceph-mon[34315]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T06:05:13.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:12 smithi150 ceph-mon[34315]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T06:05:13.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:12 smithi150 ceph-mon[34315]: Cluster is now healthy 2024-05-08T06:05:13.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:12 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/1135505496' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T06:05:13.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:12 smithi150 ceph-mon[34315]: fsmap 12 up:standby 2024-05-08T06:05:15.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:14 smithi112 ceph-mon[30003]: pgmap v740: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-08T06:05:15.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:14 smithi162 ceph-mon[34800]: pgmap v740: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-08T06:05:15.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:14 smithi150 ceph-mon[34315]: pgmap v740: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-08T06:05:15.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.571+0000 7fbf70000700 1 -- 172.21.15.112:0/965536009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf68104100 msgr2=0x7fbf68108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:15.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.571+0000 7fbf70000700 1 --2- 172.21.15.112:0/965536009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf68104100 0x7fbf68108120 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbf540099c0 tx=0x7fbf54009cd0 comp rx=0 tx=0).stop 2024-05-08T06:05:15.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.571+0000 7fbf70000700 1 -- 172.21.15.112:0/965536009 shutdown_connections 2024-05-08T06:05:15.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.571+0000 7fbf70000700 1 --2- 172.21.15.112:0/965536009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf68108850 0x7fbf681103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:15.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.571+0000 7fbf70000700 1 --2- 172.21.15.112:0/965536009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf68103750 0x7fbf68103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:15.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.571+0000 7fbf70000700 1 --2- 172.21.15.112:0/965536009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf68104100 0x7fbf68108120 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:15.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.571+0000 7fbf70000700 1 -- 172.21.15.112:0/965536009 >> 172.21.15.112:0/965536009 conn(0x7fbf6807a600 msgr2=0x7fbf6807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:15.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.571+0000 7fbf70000700 1 -- 172.21.15.112:0/965536009 shutdown_connections 2024-05-08T06:05:15.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.571+0000 7fbf70000700 1 -- 172.21.15.112:0/965536009 wait complete. 2024-05-08T06:05:15.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.572+0000 7fbf70000700 1 Processor -- start 2024-05-08T06:05:15.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.573+0000 7fbf70000700 1 -- start start 2024-05-08T06:05:15.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.573+0000 7fbf70000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf68103750 0x7fbf6819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:15.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.573+0000 7fbf70000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf68104100 0x7fbf6819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:15.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.573+0000 7fbf70000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf68108850 0x7fbf68199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:15.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.573+0000 7fbf70000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf68113660 con 0x7fbf68103750 2024-05-08T06:05:15.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.573+0000 7fbf70000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbf681134e0 con 0x7fbf68104100 2024-05-08T06:05:15.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.573+0000 7fbf70000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbf681137e0 con 0x7fbf68108850 2024-05-08T06:05:15.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.573+0000 7fbf6d600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf68103750 0x7fbf6819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:15.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.573+0000 7fbf6d600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf68103750 0x7fbf6819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37008/0 (socket says 172.21.15.112:37008) 2024-05-08T06:05:15.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.573+0000 7fbf6d600700 1 -- 172.21.15.112:0/3420677301 learned_addr learned my addr 172.21.15.112:0/3420677301 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:05:15.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.574+0000 7fbf6e000700 1 --2- 172.21.15.112:0/3420677301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf68108850 0x7fbf68199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:15.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.574+0000 7fbf6cc00700 1 --2- 172.21.15.112:0/3420677301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf68104100 0x7fbf6819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:15.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.574+0000 7fbf6d600700 1 -- 172.21.15.112:0/3420677301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf68108850 msgr2=0x7fbf68199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:15.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.574+0000 7fbf6d600700 1 --2- 172.21.15.112:0/3420677301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf68108850 0x7fbf68199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:15.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.574+0000 7fbf6d600700 1 -- 172.21.15.112:0/3420677301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf68104100 msgr2=0x7fbf6819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:15.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.574+0000 7fbf6d600700 1 --2- 172.21.15.112:0/3420677301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf68104100 0x7fbf6819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:15.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.574+0000 7fbf6d600700 1 -- 172.21.15.112:0/3420677301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6819a0c0 con 0x7fbf68103750 2024-05-08T06:05:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.574+0000 7fbf6d600700 1 --2- 172.21.15.112:0/3420677301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf68103750 0x7fbf6819f7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00e9a0 tx=0x7fbf5c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.574+0000 7fbf66000700 1 -- 172.21.15.112:0/3420677301 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c00ca10 con 0x7fbf68103750 2024-05-08T06:05:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.575+0000 7fbf66000700 1 -- 172.21.15.112:0/3420677301 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf5c00cbb0 con 0x7fbf68103750 2024-05-08T06:05:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.575+0000 7fbf70000700 1 -- 172.21.15.112:0/3420677301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6819a3b0 con 0x7fbf68103750 2024-05-08T06:05:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.575+0000 7fbf66000700 1 -- 172.21.15.112:0/3420677301 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c018680 con 0x7fbf68103750 2024-05-08T06:05:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.575+0000 7fbf70000700 1 -- 172.21.15.112:0/3420677301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf681a6670 con 0x7fbf68103750 2024-05-08T06:05:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.576+0000 7fbf70000700 1 -- 172.21.15.112:0/3420677301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6806a190 con 0x7fbf68103750 2024-05-08T06:05:15.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.579+0000 7fbf66000700 1 -- 172.21.15.112:0/3420677301 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbf5c010430 con 0x7fbf68103750 2024-05-08T06:05:15.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.580+0000 7fbf66000700 1 --2- 172.21.15.112:0/3420677301 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbf500b2900 0x7fbf500b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.581+0000 7fbf66000700 1 -- 172.21.15.112:0/3420677301 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8535+0+0 (secure 0 0 0) 0x7fbf5c0e2770 con 0x7fbf68103750 2024-05-08T06:05:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.581+0000 7fbf6cc00700 1 --2- 172.21.15.112:0/3420677301 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbf500b2900 0x7fbf500b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.581+0000 7fbf6cc00700 1 --2- 172.21.15.112:0/3420677301 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbf500b2900 0x7fbf500b4dc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbf681030b0 tx=0x7fbf54005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:15.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.583+0000 7fbf66000700 1 -- 172.21.15.112:0/3420677301 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf5c0aa360 con 0x7fbf68103750 2024-05-08T06:05:15.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.813+0000 7fbf70000700 1 -- 172.21.15.112:0/3420677301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbf6802e510 con 0x7fbf68103750 2024-05-08T06:05:15.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:15.995+0000 7fbf66000700 1 -- 172.21.15.112:0/3420677301 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) ==== 181+0+0 (secure 0 0 0) 0x7fbf5c0b23e0 con 0x7fbf68103750 2024-05-08T06:05:16.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:16.002+0000 7fbf70000700 1 -- 172.21.15.112:0/3420677301 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbf500b2900 msgr2=0x7fbf500b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:16.002+0000 7fbf70000700 1 --2- 172.21.15.112:0/3420677301 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbf500b2900 0x7fbf500b4dc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbf681030b0 tx=0x7fbf54005fb0 comp rx=0 tx=0).stop 2024-05-08T06:05:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:16.002+0000 7fbf70000700 1 -- 172.21.15.112:0/3420677301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf68103750 msgr2=0x7fbf6819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:16.002+0000 7fbf70000700 1 --2- 172.21.15.112:0/3420677301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf68103750 0x7fbf6819f7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00e9a0 tx=0x7fbf5c00ecb0 comp rx=0 tx=0).stop 2024-05-08T06:05:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:16.002+0000 7fbf70000700 1 -- 172.21.15.112:0/3420677301 shutdown_connections 2024-05-08T06:05:16.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:16.002+0000 7fbf70000700 1 --2- 172.21.15.112:0/3420677301 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fbf500b2900 0x7fbf500b4dc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:16.002+0000 7fbf70000700 1 --2- 172.21.15.112:0/3420677301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf68108850 0x7fbf68199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:16.002+0000 7fbf70000700 1 --2- 172.21.15.112:0/3420677301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf68103750 0x7fbf6819f7a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:16.002+0000 7fbf70000700 1 --2- 172.21.15.112:0/3420677301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf68104100 0x7fbf6819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:16.002+0000 7fbf70000700 1 -- 172.21.15.112:0/3420677301 >> 172.21.15.112:0/3420677301 conn(0x7fbf6807a600 msgr2=0x7fbf681017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:16.003+0000 7fbf70000700 1 -- 172.21.15.112:0/3420677301 shutdown_connections 2024-05-08T06:05:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:16.004+0000 7fbf70000700 1 -- 172.21.15.112:0/3420677301 wait complete. 2024-05-08T06:05:16.007 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-05-08T06:05:16.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:15 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3420677301' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:05:16.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:15 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3420677301' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:05:16.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:15 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3420677301' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:05:16.901 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T06:05:17.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:16 smithi112 ceph-mon[30003]: pgmap v741: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-08T06:05:17.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:16 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/3420677301' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:05:17.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:16 smithi112 ceph-mon[30003]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T06:05:17.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:16 smithi162 ceph-mon[34800]: pgmap v741: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-08T06:05:17.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:16 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/3420677301' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:05:17.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:16 smithi162 ceph-mon[34800]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T06:05:17.345 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:16 smithi150 ceph-mon[34315]: pgmap v741: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-08T06:05:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:16 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/3420677301' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:05:17.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:16 smithi150 ceph-mon[34315]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T06:05:19.230 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:19 smithi112 ceph-mon[30003]: pgmap v743: 65 pgs: 65 active+clean; 577 KiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:19.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:19 smithi162 ceph-mon[34800]: pgmap v743: 65 pgs: 65 active+clean; 577 KiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:19.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:19 smithi150 ceph-mon[34315]: pgmap v743: 65 pgs: 65 active+clean; 577 KiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:19.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.783+0000 7fb7a2e00700 1 -- 172.21.15.112:0/2997623504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c109500 msgr2=0x7fb79c1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:19.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.783+0000 7fb7a2e00700 1 --2- 172.21.15.112:0/2997623504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c109500 0x7fb79c1098e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009990 tx=0x7fb78c009ca0 comp rx=0 tx=0).stop 2024-05-08T06:05:19.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.783+0000 7fb7a2e00700 1 -- 172.21.15.112:0/2997623504 shutdown_connections 2024-05-08T06:05:19.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.783+0000 7fb7a2e00700 1 --2- 172.21.15.112:0/2997623504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79c06c3d0 0x7fb79c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:19.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.783+0000 7fb7a2e00700 1 --2- 172.21.15.112:0/2997623504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c109500 0x7fb79c1098e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:19.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.783+0000 7fb7a2e00700 1 --2- 172.21.15.112:0/2997623504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb79c06ba10 0x7fb79c06be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:19.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.783+0000 7fb7a2e00700 1 -- 172.21.15.112:0/2997623504 >> 172.21.15.112:0/2997623504 conn(0x7fb79c079310 msgr2=0x7fb79c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:19.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.783+0000 7fb7a2e00700 1 -- 172.21.15.112:0/2997623504 shutdown_connections 2024-05-08T06:05:19.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.784+0000 7fb7a2e00700 1 -- 172.21.15.112:0/2997623504 wait complete. 2024-05-08T06:05:19.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.785+0000 7fb7a2e00700 1 Processor -- start 2024-05-08T06:05:19.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.785+0000 7fb7a2e00700 1 -- start start 2024-05-08T06:05:19.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.785+0000 7fb7a2e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb79c06ba10 0x7fb79c19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:19.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.785+0000 7fb7a2e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c06c3d0 0x7fb79c19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:19.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.785+0000 7fb7a2e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79c109500 0x7fb79c1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:19.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.785+0000 7fb7a2e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c113660 con 0x7fb79c109500 2024-05-08T06:05:19.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.785+0000 7fb7a2e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1134e0 con 0x7fb79c06c3d0 2024-05-08T06:05:19.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.785+0000 7fb7a2e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1137e0 con 0x7fb79c06ba10 2024-05-08T06:05:19.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.786+0000 7fb793e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c06c3d0 0x7fb79c19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:19.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.786+0000 7fb79be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb79c06ba10 0x7fb79c19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:19.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.786+0000 7fb7a0e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79c109500 0x7fb79c1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:19.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.786+0000 7fb793e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c06c3d0 0x7fb79c19c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:50508/0 (socket says 172.21.15.112:50508) 2024-05-08T06:05:19.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.786+0000 7fb793e00700 1 -- 172.21.15.112:0/2679342191 learned_addr learned my addr 172.21.15.112:0/2679342191 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-05-08T06:05:19.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.786+0000 7fb79be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb79c06ba10 0x7fb79c19be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.112:56158/0 (socket says 172.21.15.112:56158) 2024-05-08T06:05:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.786+0000 7fb793e00700 1 -- 172.21.15.112:0/2679342191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb79c06ba10 msgr2=0x7fb79c19be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.786+0000 7fb793e00700 1 --2- 172.21.15.112:0/2679342191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb79c06ba10 0x7fb79c19be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.786+0000 7fb793e00700 1 -- 172.21.15.112:0/2679342191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79c109500 msgr2=0x7fb79c1a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.786+0000 7fb793e00700 1 --2- 172.21.15.112:0/2679342191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79c109500 0x7fb79c1a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.786+0000 7fb793e00700 1 -- 172.21.15.112:0/2679342191 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79c1a0e50 con 0x7fb79c06c3d0 2024-05-08T06:05:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.786+0000 7fb793e00700 1 --2- 172.21.15.112:0/2679342191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c06c3d0 0x7fb79c19c3d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb78400e9a0 tx=0x7fb78400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.787+0000 7fb799600700 1 -- 172.21.15.112:0/2679342191 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78400ca10 con 0x7fb79c06c3d0 2024-05-08T06:05:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.787+0000 7fb799600700 1 -- 172.21.15.112:0/2679342191 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb78400cbb0 con 0x7fb79c06c3d0 2024-05-08T06:05:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.787+0000 7fb799600700 1 -- 172.21.15.112:0/2679342191 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784018680 con 0x7fb79c06c3d0 2024-05-08T06:05:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.787+0000 7fb7a2e00700 1 -- 172.21.15.112:0/2679342191 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79c1a1140 con 0x7fb79c06c3d0 2024-05-08T06:05:19.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.787+0000 7fb7a2e00700 1 -- 172.21.15.112:0/2679342191 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79c1a89c0 con 0x7fb79c06c3d0 2024-05-08T06:05:19.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.788+0000 7fb7a2e00700 1 -- 172.21.15.112:0/2679342191 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb79c06a190 con 0x7fb79c06c3d0 2024-05-08T06:05:19.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.790+0000 7fb799600700 1 -- 172.21.15.112:0/2679342191 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb784010430 con 0x7fb79c06c3d0 2024-05-08T06:05:19.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.792+0000 7fb799600700 1 --2- 172.21.15.112:0/2679342191 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb7800b2900 0x7fb7800b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:19.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.792+0000 7fb799600700 1 -- 172.21.15.112:0/2679342191 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7973+0+0 (secure 0 0 0) 0x7fb7840e2590 con 0x7fb79c06c3d0 2024-05-08T06:05:19.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.792+0000 7fb79be00700 1 --2- 172.21.15.112:0/2679342191 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb7800b2900 0x7fb7800b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:19.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.793+0000 7fb79be00700 1 --2- 172.21.15.112:0/2679342191 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb7800b2900 0x7fb7800b4dc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb78c006010 tx=0x7fb78c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:19.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:19.794+0000 7fb799600700 1 -- 172.21.15.112:0/2679342191 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7840aa360 con 0x7fb79c06c3d0 2024-05-08T06:05:20.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:20.024+0000 7fb7a2e00700 1 -- 172.21.15.112:0/2679342191 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb79c1a8bf0 con 0x7fb79c06c3d0 2024-05-08T06:05:21.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:21.024+0000 7fb799600700 1 -- 172.21.15.112:0/2679342191 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) ==== 169+0+0 (secure 0 0 0) 0x7fb7840b23e0 con 0x7fb79c06c3d0 2024-05-08T06:05:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:21.030+0000 7fb7a2e00700 1 -- 172.21.15.112:0/2679342191 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb7800b2900 msgr2=0x7fb7800b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:21.030+0000 7fb7a2e00700 1 --2- 172.21.15.112:0/2679342191 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb7800b2900 0x7fb7800b4dc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb78c006010 tx=0x7fb78c00b260 comp rx=0 tx=0).stop 2024-05-08T06:05:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:21.030+0000 7fb7a2e00700 1 -- 172.21.15.112:0/2679342191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c06c3d0 msgr2=0x7fb79c19c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:21.030+0000 7fb7a2e00700 1 --2- 172.21.15.112:0/2679342191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c06c3d0 0x7fb79c19c3d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb78400e9a0 tx=0x7fb78400ecb0 comp rx=0 tx=0).stop 2024-05-08T06:05:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:21.030+0000 7fb7a2e00700 1 -- 172.21.15.112:0/2679342191 shutdown_connections 2024-05-08T06:05:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:21.030+0000 7fb7a2e00700 1 --2- 172.21.15.112:0/2679342191 >> [v2:172.21.15.112:6800/1211679607,v1:172.21.15.112:6801/1211679607] conn(0x7fb7800b2900 0x7fb7800b4dc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:21.030+0000 7fb7a2e00700 1 --2- 172.21.15.112:0/2679342191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79c109500 0x7fb79c1a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:21.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:21.030+0000 7fb7a2e00700 1 --2- 172.21.15.112:0/2679342191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb79c06ba10 0x7fb79c19be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:21.030+0000 7fb7a2e00700 1 --2- 172.21.15.112:0/2679342191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c06c3d0 0x7fb79c19c3d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:21.030+0000 7fb7a2e00700 1 -- 172.21.15.112:0/2679342191 >> 172.21.15.112:0/2679342191 conn(0x7fb79c079310 msgr2=0x7fb79c100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:21.031+0000 7fb7a2e00700 1 -- 172.21.15.112:0/2679342191 shutdown_connections 2024-05-08T06:05:21.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-05-08T06:05:21.031+0000 7fb7a2e00700 1 -- 172.21.15.112:0/2679342191 wait complete. 2024-05-08T06:05:21.034 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-05-08T06:05:21.182 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:21 smithi112 ceph-mon[30003]: pgmap v744: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:21.182 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:21 smithi112 ceph-mon[30003]: from='client.? 172.21.15.112:0/2679342191' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:05:21.182 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:21 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:05:21.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:21 smithi162 ceph-mon[34800]: pgmap v744: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:21.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:21 smithi162 ceph-mon[34800]: from='client.? 172.21.15.112:0/2679342191' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:05:21.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:21 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:05:21.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:21 smithi150 ceph-mon[34315]: pgmap v744: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:21.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:21 smithi150 ceph-mon[34315]: from='client.? 172.21.15.112:0/2679342191' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:05:21.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:21 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:05:21.979 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:05:22.058 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:05:22.092 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:05:22.125 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-08T06:05:22.125 DEBUG:teuthology.orchestra.run.smithi112:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T06:05:22.169 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T06:05:22.187 DEBUG:teuthology.orchestra.run.smithi162:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T06:05:22.204 INFO:tasks.cephadm:Stopping all daemons... 2024-05-08T06:05:22.205 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-08T06:05:22.205 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mon.a 2024-05-08T06:05:22.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:22 smithi112 ceph-mon[30003]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:05:22.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:22 smithi112 ceph-mon[30003]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T06:05:22.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:22 smithi162 ceph-mon[34800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:05:22.316 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:22 smithi162 ceph-mon[34800]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T06:05:22.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:22 smithi150 ceph-mon[34315]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:05:22.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:22 smithi150 ceph-mon[34315]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T06:05:22.565 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:22 smithi112 systemd[1]: Stopping Ceph mon.a for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:23.235 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:22 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a[29999]: 2024-05-08T06:05:22.851+0000 7fb723a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:23.235 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:22 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a[29999]: 2024-05-08T06:05:22.851+0000 7fb723a00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-08T06:05:23.235 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:22 smithi112 podman[119532]: 2024-05-08 06:05:22.985620516 +0000 UTC m=+0.347547547 container died 11e2e06de9b6c2d3298040188484e2e2f6ab0b727158e999cc81a7947a159be6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:05:23.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:23 smithi112 podman[119532]: 2024-05-08 06:05:23.434269195 +0000 UTC m=+0.796196221 container cleanup 11e2e06de9b6c2d3298040188484e2e2f6ab0b727158e999cc81a7947a159be6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-08T06:05:23.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:23 smithi112 bash[119532]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a 2024-05-08T06:05:24.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:May 08 06:05:23 smithi112 podman[119546]: 2024-05-08 06:05:23.669255197 +0000 UTC m=+0.667520131 container remove 11e2e06de9b6c2d3298040188484e2e2f6ab0b727158e999cc81a7947a159be6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:05:24.121 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mon.a.service' 2024-05-08T06:05:24.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:24.251 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-08T06:05:24.251 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-08T06:05:24.251 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mon.b 2024-05-08T06:05:24.563 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:24 smithi150 systemd[1]: Stopping Ceph mon.b for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:24 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-b[34311]: 2024-05-08T06:05:24.562+0000 7f59f0fa0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:24 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-b[34311]: 2024-05-08T06:05:24.562+0000 7f59f0fa0700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-08T06:05:24.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:24 smithi150 podman[68684]: 2024-05-08 06:05:24.665100608 +0000 UTC m=+0.223919552 container died 7241b4923745792aed2fbaeb9a4c8734678da1f048de5fd08182a2e5becbf5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-05-08T06:05:25.297 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:25 smithi150 podman[68684]: 2024-05-08 06:05:25.012650326 +0000 UTC m=+0.571469277 container cleanup 7241b4923745792aed2fbaeb9a4c8734678da1f048de5fd08182a2e5becbf5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-b, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-08T06:05:25.297 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 08 06:05:25 smithi150 bash[68684]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-b 2024-05-08T06:05:25.432 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mon.b.service' 2024-05-08T06:05:25.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:25.481 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-08T06:05:25.481 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-08T06:05:25.481 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mon.c 2024-05-08T06:05:25.798 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:25 smithi162 systemd[1]: Stopping Ceph mon.c for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:26.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:25 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-c[34796]: 2024-05-08T06:05:25.797+0000 7ff4494bd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:26.066 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:25 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-c[34796]: 2024-05-08T06:05:25.797+0000 7ff4494bd700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-08T06:05:26.067 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:25 smithi162 podman[70560]: 2024-05-08 06:05:25.971941414 +0000 UTC m=+0.286688371 container died 9d8c1bd591f29b7d17c7d36a7f62271b8cbbdfdeffb50e7a17103a35f86fbe3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-c, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-08T06:05:26.566 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:26 smithi162 podman[70560]: 2024-05-08 06:05:26.357320159 +0000 UTC m=+0.672067119 container cleanup 9d8c1bd591f29b7d17c7d36a7f62271b8cbbdfdeffb50e7a17103a35f86fbe3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-08T06:05:26.567 INFO:journalctl@ceph.mon.c.smithi162.stdout:May 08 06:05:26 smithi162 bash[70560]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mon-c 2024-05-08T06:05:26.784 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mon.c.service' 2024-05-08T06:05:26.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:26.827 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-08T06:05:26.827 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-08T06:05:26.828 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.x 2024-05-08T06:05:27.316 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 06:05:26 smithi112 systemd[1]: Stopping Ceph mgr.x for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:27.705 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 06:05:27 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T06:05:27.370+0000 7fa16d400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:27.705 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 06:05:27 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x[30242]: 2024-05-08T06:05:27.370+0000 7fa16d400700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-08T06:05:27.706 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 06:05:27 smithi112 podman[119652]: 2024-05-08 06:05:27.463854673 +0000 UTC m=+0.213731147 container died 935065eb06dfb6b3ad266456e9a5905237da1f4776b221602b5f5a63360d3502 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-08T06:05:28.066 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 06:05:27 smithi112 podman[119652]: 2024-05-08 06:05:27.914638772 +0000 UTC m=+0.664515260 container cleanup 935065eb06dfb6b3ad266456e9a5905237da1f4776b221602b5f5a63360d3502 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-08T06:05:28.066 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 06:05:27 smithi112 bash[119652]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x 2024-05-08T06:05:28.566 INFO:journalctl@ceph.mgr.x.smithi112.stdout:May 08 06:05:28 smithi112 podman[119666]: 2024-05-08 06:05:28.107574958 +0000 UTC m=+0.628883061 container remove 935065eb06dfb6b3ad266456e9a5905237da1f4776b221602b5f5a63360d3502 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-08T06:05:28.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-08T06:05:28.734 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:28.734 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-08T06:05:28.734 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-08T06:05:28.735 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.y 2024-05-08T06:05:29.029 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 06:05:28 smithi150 systemd[1]: Stopping Ceph mgr.y for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:29.318 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 06:05:29 smithi150 podman[68806]: 2024-05-08 06:05:29.029746677 +0000 UTC m=+0.109633884 container died fb487a1da7df7572aa9d385258fef8ff94ea9b2fd94a783c9f659ffeccfbeeae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:05:29.683 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 06:05:29 smithi150 podman[68806]: 2024-05-08 06:05:29.425816318 +0000 UTC m=+0.505703525 container cleanup fb487a1da7df7572aa9d385258fef8ff94ea9b2fd94a783c9f659ffeccfbeeae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True) 2024-05-08T06:05:29.683 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 08 06:05:29 smithi150 bash[68806]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-y 2024-05-08T06:05:29.861 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-08T06:05:29.894 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:29.894 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-08T06:05:29.894 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-08T06:05:29.894 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.z 2024-05-08T06:05:30.190 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 06:05:29 smithi162 systemd[1]: Stopping Ceph mgr.z for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:30.461 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 06:05:30 smithi162 podman[70682]: 2024-05-08 06:05:30.191071874 +0000 UTC m=+0.101965649 container died fadf6de37aa032c0d3b6877da755c6f2edc858fd7842193895770178c3afe504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:05:30.720 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 06:05:30 smithi162 podman[70682]: 2024-05-08 06:05:30.53689059 +0000 UTC m=+0.447784346 container cleanup fadf6de37aa032c0d3b6877da755c6f2edc858fd7842193895770178c3afe504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , ceph=True) 2024-05-08T06:05:30.720 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 06:05:30 smithi162 bash[70682]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z 2024-05-08T06:05:30.956 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-08T06:05:30.977 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 06:05:30 smithi162 podman[70696]: 2024-05-08 06:05:30.721563581 +0000 UTC m=+0.523341072 container remove fadf6de37aa032c0d3b6877da755c6f2edc858fd7842193895770178c3afe504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-08T06:05:30.977 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 06:05:30 smithi162 systemd[1]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-08T06:05:30.977 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 06:05:30 smithi162 systemd[1]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-05-08T06:05:30.977 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 06:05:30 smithi162 systemd[1]: Stopped Ceph mgr.z for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T06:05:30.977 INFO:journalctl@ceph.mgr.z.smithi162.stdout:May 08 06:05:30 smithi162 systemd[1]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mgr.z.service: Consumed 13.479s CPU time. 2024-05-08T06:05:30.990 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:30.990 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-08T06:05:30.990 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-08T06:05:30.990 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.0 2024-05-08T06:05:31.508 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 06:05:31 smithi112 systemd[1]: Stopping Ceph osd.0 for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:31.816 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 06:05:31 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-0[42288]: 2024-05-08T06:05:31.506+0000 7fa4b1c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:31.816 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 06:05:31 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-0[42288]: 2024-05-08T06:05:31.506+0000 7fa4b1c00700 -1 osd.0 87 *** Got signal Terminated *** 2024-05-08T06:05:31.816 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 06:05:31 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-0[42288]: 2024-05-08T06:05:31.506+0000 7fa4b1c00700 -1 osd.0 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:05:37.399 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 06:05:37 smithi112 podman[119771]: 2024-05-08 06:05:37.085277387 +0000 UTC m=+5.694365551 container died c1d9f0ef0df44740e3ffccef51f0670a2db805e508af7a4230d8892e0c681f71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-05-08T06:05:37.657 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 06:05:37 smithi112 podman[119771]: 2024-05-08 06:05:37.551715651 +0000 UTC m=+6.160803847 container cleanup c1d9f0ef0df44740e3ffccef51f0670a2db805e508af7a4230d8892e0c681f71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:05:37.657 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 06:05:37 smithi112 bash[119771]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-0 2024-05-08T06:05:38.066 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 06:05:37 smithi112 podman[119789]: 2024-05-08 06:05:37.803181601 +0000 UTC m=+0.703383654 container remove c1d9f0ef0df44740e3ffccef51f0670a2db805e508af7a4230d8892e0c681f71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-08T06:05:38.909 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 06:05:38 smithi112 podman[119880]: 2024-05-08 06:05:38.620698525 +0000 UTC m=+0.124442889 container create 18dc017a6972060bc2fd3893da5807a5590557ce8aeb42901762870e3a956116 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-08T06:05:38.909 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 06:05:38 smithi112 podman[119880]: 2024-05-08 06:05:38.528919593 +0000 UTC m=+0.032663913 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:38.909 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 06:05:38 smithi112 podman[119880]: 2024-05-08 06:05:38.907988445 +0000 UTC m=+0.411732809 container init 18dc017a6972060bc2fd3893da5807a5590557ce8aeb42901762870e3a956116 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-08T06:05:39.316 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 06:05:38 smithi112 podman[119880]: 2024-05-08 06:05:38.919142167 +0000 UTC m=+0.422886517 container start 18dc017a6972060bc2fd3893da5807a5590557ce8aeb42901762870e3a956116 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-08T06:05:39.316 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 06:05:39 smithi112 podman[119880]: 2024-05-08 06:05:39.006745964 +0000 UTC m=+0.510490375 container attach 18dc017a6972060bc2fd3893da5807a5590557ce8aeb42901762870e3a956116 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:05:39.681 INFO:journalctl@ceph.osd.0.smithi112.stdout:May 08 06:05:39 smithi112 podman[119880]: 2024-05-08 06:05:39.368784273 +0000 UTC m=+0.872528627 container died 18dc017a6972060bc2fd3893da5807a5590557ce8aeb42901762870e3a956116 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux ) 2024-05-08T06:05:40.041 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.0.service' 2024-05-08T06:05:40.145 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:40.146 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-08T06:05:40.146 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-08T06:05:40.146 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.1 2024-05-08T06:05:40.433 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 06:05:40 smithi150 systemd[1]: Stopping Ceph osd.1 for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:40.933 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 06:05:40 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-1[38941]: 2024-05-08T06:05:40.456+0000 7f83cb548700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:40.933 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 06:05:40 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-1[38941]: 2024-05-08T06:05:40.456+0000 7f83cb548700 -1 osd.1 87 *** Got signal Terminated *** 2024-05-08T06:05:40.933 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 06:05:40 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-1[38941]: 2024-05-08T06:05:40.456+0000 7f83cb548700 -1 osd.1 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:05:46.274 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 06:05:45 smithi150 podman[68942]: 2024-05-08 06:05:45.987498638 +0000 UTC m=+5.647538180 container died 4c5a4a89b25455efe905f306012eae5f448a193de9be661647a39d8821b34cf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-1, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:05:46.633 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 06:05:46 smithi150 podman[68942]: 2024-05-08 06:05:46.373687527 +0000 UTC m=+6.033727067 container cleanup 4c5a4a89b25455efe905f306012eae5f448a193de9be661647a39d8821b34cf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-08T06:05:46.633 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 06:05:46 smithi150 bash[68942]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-1 2024-05-08T06:05:46.933 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 06:05:46 smithi150 podman[68959]: 2024-05-08 06:05:46.633247954 +0000 UTC m=+0.641867822 container remove 4c5a4a89b25455efe905f306012eae5f448a193de9be661647a39d8821b34cf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-08T06:05:47.319 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 06:05:46 smithi150 podman[69051]: 2024-05-08 06:05:46.889296998 +0000 UTC m=+0.017769705 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:47.320 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 06:05:46 smithi150 podman[69051]: 2024-05-08 06:05:46.99233917 +0000 UTC m=+0.120811846 container create 3322871aa1d0765aa5ac8530cfde0ccbe03de011071ec7e531cdbd0cc6bdccdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-08T06:05:47.320 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 06:05:47 smithi150 podman[69051]: 2024-05-08 06:05:47.219718722 +0000 UTC m=+0.348191407 container init 3322871aa1d0765aa5ac8530cfde0ccbe03de011071ec7e531cdbd0cc6bdccdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-08T06:05:47.320 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 06:05:47 smithi150 podman[69051]: 2024-05-08 06:05:47.223231788 +0000 UTC m=+0.351704469 container start 3322871aa1d0765aa5ac8530cfde0ccbe03de011071ec7e531cdbd0cc6bdccdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-1-deactivate, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-08T06:05:47.320 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 06:05:47 smithi150 podman[69051]: 2024-05-08 06:05:47.32011871 +0000 UTC m=+0.448591386 container attach 3322871aa1d0765aa5ac8530cfde0ccbe03de011071ec7e531cdbd0cc6bdccdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-1-deactivate, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-08T06:05:47.683 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 08 06:05:47 smithi150 podman[69051]: 2024-05-08 06:05:47.515611933 +0000 UTC m=+0.644084611 container died 3322871aa1d0765aa5ac8530cfde0ccbe03de011071ec7e531cdbd0cc6bdccdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-08T06:05:48.121 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.1.service' 2024-05-08T06:05:48.159 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:48.160 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-08T06:05:48.160 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-08T06:05:48.160 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.2 2024-05-08T06:05:48.472 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 06:05:48 smithi162 systemd[1]: Stopping Ceph osd.2 for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:48.816 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 06:05:48 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-2[39958]: 2024-05-08T06:05:48.472+0000 7f53b612e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:48.817 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 06:05:48 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-2[39958]: 2024-05-08T06:05:48.472+0000 7f53b612e700 -1 osd.2 87 *** Got signal Terminated *** 2024-05-08T06:05:48.817 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 06:05:48 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-2[39958]: 2024-05-08T06:05:48.472+0000 7f53b612e700 -1 osd.2 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:05:54.173 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 06:05:53 smithi162 podman[70807]: 2024-05-08 06:05:53.891231145 +0000 UTC m=+5.537708067 container died e810072f771882b3cdb36f5aadaaba9d7d223e43e76f46af643661348800c856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-2, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:05:54.566 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 06:05:54 smithi162 podman[70807]: 2024-05-08 06:05:54.247107893 +0000 UTC m=+5.893584854 container cleanup e810072f771882b3cdb36f5aadaaba9d7d223e43e76f46af643661348800c856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-08T06:05:54.566 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 06:05:54 smithi162 bash[70807]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-2 2024-05-08T06:05:54.567 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 06:05:54 smithi162 podman[70822]: 2024-05-08 06:05:54.414930629 +0000 UTC m=+0.523136501 container remove e810072f771882b3cdb36f5aadaaba9d7d223e43e76f46af643661348800c856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-08T06:05:55.126 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 06:05:54 smithi162 podman[70915]: 2024-05-08 06:05:54.730132054 +0000 UTC m=+0.017704441 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:55.126 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 06:05:54 smithi162 podman[70915]: 2024-05-08 06:05:54.882518711 +0000 UTC m=+0.170091079 container create 70923369dc49062c75e09e774896198420b6c21af8e1376b558a4458ea0639cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:05:55.127 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 06:05:55 smithi162 podman[70915]: 2024-05-08 06:05:55.127010314 +0000 UTC m=+0.414582692 container init 70923369dc49062c75e09e774896198420b6c21af8e1376b558a4458ea0639cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-2-deactivate, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-08T06:05:55.423 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 06:05:55 smithi162 podman[70915]: 2024-05-08 06:05:55.130573198 +0000 UTC m=+0.418145567 container start 70923369dc49062c75e09e774896198420b6c21af8e1376b558a4458ea0639cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-08T06:05:55.423 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 06:05:55 smithi162 podman[70915]: 2024-05-08 06:05:55.209824782 +0000 UTC m=+0.497397151 container attach 70923369dc49062c75e09e774896198420b6c21af8e1376b558a4458ea0639cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-08T06:05:55.729 INFO:journalctl@ceph.osd.2.smithi162.stdout:May 08 06:05:55 smithi162 podman[70915]: 2024-05-08 06:05:55.423823202 +0000 UTC m=+0.711395577 container died 70923369dc49062c75e09e774896198420b6c21af8e1376b558a4458ea0639cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-08T06:05:56.029 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.2.service' 2024-05-08T06:05:56.068 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:56.069 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-08T06:05:56.069 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-08T06:05:56.069 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.3 2024-05-08T06:05:56.535 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:05:56 smithi112 systemd[1]: Stopping Ceph osd.3 for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:56.816 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:05:56 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3[49242]: 2024-05-08T06:05:56.651+0000 7f0143000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:56.817 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:05:56 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3[49242]: 2024-05-08T06:05:56.651+0000 7f0143000700 -1 osd.3 87 *** Got signal Terminated *** 2024-05-08T06:05:56.817 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:05:56 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3[49242]: 2024-05-08T06:05:56.651+0000 7f0143000700 -1 osd.3 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:05:57.815 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:05:57 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3[49242]: 2024-05-08T06:05:57.477+0000 7f013e800700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:34.401517+0000 front 2024-05-08T06:05:34.401821+0000 (oldest deadline 2024-05-08T06:05:57.301657+0000) 2024-05-08T06:05:58.317 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:05:58 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:05:58.156+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:05:58.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:05:57 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:05:57.948+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:05:58.565 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:05:58 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3[49242]: 2024-05-08T06:05:58.459+0000 7f013e800700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:34.401517+0000 front 2024-05-08T06:05:34.401821+0000 (oldest deadline 2024-05-08T06:05:57.301657+0000) 2024-05-08T06:05:59.316 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:05:59 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:05:59.123+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:05:59.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:05:58 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:05:58.917+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:05:59.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:05:59 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:05:59.414+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:05:59.815 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:05:59 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3[49242]: 2024-05-08T06:05:59.458+0000 7f013e800700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:34.401517+0000 front 2024-05-08T06:05:34.401821+0000 (oldest deadline 2024-05-08T06:05:57.301657+0000) 2024-05-08T06:06:00.316 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:00 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:00.128+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:00.316 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:05:59 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:05:59.909+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:00.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:00 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:00.386+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:00.815 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:06:00 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3[49242]: 2024-05-08T06:06:00.446+0000 7f013e800700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:34.401517+0000 front 2024-05-08T06:05:34.401821+0000 (oldest deadline 2024-05-08T06:05:57.301657+0000) 2024-05-08T06:06:01.177 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:00 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:00.912+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:01.317 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:01 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:01.028+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:01.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:01 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:01.177+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:01.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:01 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:01.403+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:01.815 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:06:01 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3[49242]: 2024-05-08T06:06:01.464+0000 7f013e800700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:34.401517+0000 front 2024-05-08T06:05:34.401821+0000 (oldest deadline 2024-05-08T06:05:57.301657+0000) 2024-05-08T06:06:02.220 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:01 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:01.894+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:02.316 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:06:02 smithi112 podman[119985]: 2024-05-08 06:06:02.054734051 +0000 UTC m=+5.530897135 container died 48d5c4d03c7980c9f21e26c21ca7a3c7e7acf7f865f489cad8f36ca21b1897de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2) 2024-05-08T06:06:02.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:02 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:02.000+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:02.317 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:01 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:01.883+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:02.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:02 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:02.220+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:02.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:02 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:02.429+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:02.816 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:06:02 smithi112 podman[119985]: 2024-05-08 06:06:02.562474268 +0000 UTC m=+6.038637402 container cleanup 48d5c4d03c7980c9f21e26c21ca7a3c7e7acf7f865f489cad8f36ca21b1897de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:06:02.816 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:06:02 smithi112 bash[119985]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3 2024-05-08T06:06:03.247 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:02 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:02.898+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:03.315 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:06:02 smithi112 podman[120016]: 2024-05-08 06:06:02.839090778 +0000 UTC m=+0.769592996 container remove 48d5c4d03c7980c9f21e26c21ca7a3c7e7acf7f865f489cad8f36ca21b1897de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-08T06:06:03.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:02 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:02.977+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:03.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:02 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:02.921+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:03.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:03 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:03.248+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:03.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:03 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:03.443+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:04.035 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:03 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:03.992+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:04.035 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:03 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:03.951+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:04.035 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:06:03 smithi112 podman[120104]: 2024-05-08 06:06:03.670353341 +0000 UTC m=+0.035005439 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:06:04.035 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:06:03 smithi112 podman[120104]: 2024-05-08 06:06:03.773359043 +0000 UTC m=+0.138011161 container create b9ce40a8f206bf96f7b313db7b57639ffb2069a28806e88a3934bd3b3827f2c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.build-date=20240507) 2024-05-08T06:06:04.036 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:06:04 smithi112 podman[120104]: 2024-05-08 06:06:04.035281931 +0000 UTC m=+0.399934073 container init b9ce40a8f206bf96f7b313db7b57639ffb2069a28806e88a3934bd3b3827f2c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:06:04.280 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:03 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:03.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:04.316 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:06:04 smithi112 podman[120104]: 2024-05-08 06:06:04.047919325 +0000 UTC m=+0.412571431 container start b9ce40a8f206bf96f7b313db7b57639ffb2069a28806e88a3934bd3b3827f2c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-08T06:06:04.316 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:06:04 smithi112 podman[120104]: 2024-05-08 06:06:04.134187852 +0000 UTC m=+0.498839993 container attach b9ce40a8f206bf96f7b313db7b57639ffb2069a28806e88a3934bd3b3827f2c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:06:04.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:04 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:04.281+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:04.815 INFO:journalctl@ceph.osd.3.smithi112.stdout:May 08 06:06:04 smithi112 podman[120104]: 2024-05-08 06:06:04.501104123 +0000 UTC m=+0.865756265 container died b9ce40a8f206bf96f7b313db7b57639ffb2069a28806e88a3934bd3b3827f2c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-08T06:06:04.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:04 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:04.434+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:05.272 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:04 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:04.842+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:05.304 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:04 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:04.982+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:05.305 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:04 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:04.963+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:05.364 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.3.service' 2024-05-08T06:06:05.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:05.481 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-08T06:06:05.481 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-08T06:06:05.481 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.4 2024-05-08T06:06:05.567 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:05 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:05.273+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:05.784 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 06:06:05 smithi150 systemd[1]: Stopping Ceph osd.4 for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:05.785 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:05 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:05.434+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:06.183 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 06:06:05 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-4[45481]: 2024-05-08T06:06:05.784+0000 7fc4ce18f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:06.183 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 06:06:05 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-4[45481]: 2024-05-08T06:06:05.784+0000 7fc4ce18f700 -1 osd.4 87 *** Got signal Terminated *** 2024-05-08T06:06:06.183 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 06:06:05 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-4[45481]: 2024-05-08T06:06:05.784+0000 7fc4ce18f700 -1 osd.4 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:06:06.235 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:05 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:05.887+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:06.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:05 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:05.927+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:06.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:05 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:05.977+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:06.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:06 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:06.235+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:06.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:06 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:06.399+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:07.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:06 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:06.926+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:07.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:06 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:06.945+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:07.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:06 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:06.945+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:07.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:06 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:06.945+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:07.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:07.271+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:07.567 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:07.271+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.017575+0000 front 2024-05-08T06:05:41.017652+0000 (oldest deadline 2024-05-08T06:06:06.317636+0000) 2024-05-08T06:06:07.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:07 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:07.394+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:08.308 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:07.926+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:08.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:07 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:07.908+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:08.315 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:07 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:07.899+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:08.315 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:07 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:07.899+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:08.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:08 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:08.309+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:08.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:08 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:08.309+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.017575+0000 front 2024-05-08T06:05:41.017652+0000 (oldest deadline 2024-05-08T06:06:06.317636+0000) 2024-05-08T06:06:08.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:08 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:08.372+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:09.289 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:08 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:08.871+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:09.289 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:08 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:08.871+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:09.290 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:08 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:08.934+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:09.290 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:08 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:08.934+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:09.316 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:08 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:08.882+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:09.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:08 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:08.904+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:09.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:08 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:08.904+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:09.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:09 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:09.395+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:09.816 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:09 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:09.339+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:09.817 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:09 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:09.339+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.017575+0000 front 2024-05-08T06:05:41.017652+0000 (oldest deadline 2024-05-08T06:06:06.317636+0000) 2024-05-08T06:06:10.297 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:09 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:09.891+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:10.297 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:09 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:09.939+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:10.298 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:09 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:09.939+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:10.315 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:09 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:09.910+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:10.315 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:09 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:09.910+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:10.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:09 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:09.897+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:10.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:09 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:09.897+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:10.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:10 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:10.297+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:10.567 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:10 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:10.297+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.017575+0000 front 2024-05-08T06:05:41.017652+0000 (oldest deadline 2024-05-08T06:06:06.317636+0000) 2024-05-08T06:06:10.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:10 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:10.421+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:11.258 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:10 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:10.875+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:11.258 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:10 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:10.950+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:11.258 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:10 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:10.950+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:11.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:10 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:10.896+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:11.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:10 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:10.896+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:11.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:10 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:10.906+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:11.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:10 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:10.906+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:11.393 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 06:06:11 smithi150 podman[69157]: 2024-05-08 06:06:11.125162186 +0000 UTC m=+5.442572556 container died ee01b6b84637979520017ac7f52df674b0a9e597497fae8df492a08503c9b988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-4, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-08T06:06:11.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:11 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:11.258+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:11.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:11 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:11.258+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.017575+0000 front 2024-05-08T06:05:41.017652+0000 (oldest deadline 2024-05-08T06:06:06.317636+0000) 2024-05-08T06:06:11.683 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 06:06:11 smithi150 podman[69157]: 2024-05-08 06:06:11.493800006 +0000 UTC m=+5.811210375 container cleanup ee01b6b84637979520017ac7f52df674b0a9e597497fae8df492a08503c9b988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:06:11.683 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 06:06:11 smithi150 bash[69157]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-4 2024-05-08T06:06:11.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:11 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:11.434+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:11.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:11 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:11.434+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:12.108 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 06:06:11 smithi150 podman[69174]: 2024-05-08 06:06:11.778581759 +0000 UTC m=+0.655360400 container remove ee01b6b84637979520017ac7f52df674b0a9e597497fae8df492a08503c9b988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-08T06:06:12.108 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:11 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:11.957+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:12.227 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:11 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:11.911+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:12.227 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:11 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:11.910+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:12.227 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:11 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:11.910+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:12.315 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:11 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:11.939+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:12.315 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:11 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:11.939+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:12.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:11 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:11.912+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:12.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:11 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:11.912+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:12.412 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 06:06:12 smithi150 podman[69265]: 2024-05-08 06:06:12.011602532 +0000 UTC m=+0.017480707 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:06:12.412 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 06:06:12 smithi150 podman[69265]: 2024-05-08 06:06:12.121573281 +0000 UTC m=+0.127451447 container create effa3ef13705511bcf9526fdb6a7ccb100a3fcc66f1b048dc2e50945d0f53749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-4-deactivate, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:06:12.412 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 06:06:12 smithi150 podman[69265]: 2024-05-08 06:06:12.340544589 +0000 UTC m=+0.346422756 container init effa3ef13705511bcf9526fdb6a7ccb100a3fcc66f1b048dc2e50945d0f53749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507) 2024-05-08T06:06:12.412 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 06:06:12 smithi150 podman[69265]: 2024-05-08 06:06:12.343667989 +0000 UTC m=+0.349546164 container start effa3ef13705511bcf9526fdb6a7ccb100a3fcc66f1b048dc2e50945d0f53749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:06:12.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:12 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:12.227+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:12.567 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:12 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:12.227+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.017575+0000 front 2024-05-08T06:05:41.017652+0000 (oldest deadline 2024-05-08T06:06:06.317636+0000) 2024-05-08T06:06:12.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:12 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:12.412+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:12.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:12 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:12.412+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:12.683 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 06:06:12 smithi150 podman[69265]: 2024-05-08 06:06:12.415530087 +0000 UTC m=+0.421408269 container attach effa3ef13705511bcf9526fdb6a7ccb100a3fcc66f1b048dc2e50945d0f53749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-08T06:06:12.683 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 08 06:06:12 smithi150 podman[69265]: 2024-05-08 06:06:12.639451991 +0000 UTC m=+0.645330157 container died effa3ef13705511bcf9526fdb6a7ccb100a3fcc66f1b048dc2e50945d0f53749 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-08T06:06:13.227 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:12 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:12.983+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:13.249 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.4.service' 2024-05-08T06:06:13.257 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:12 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:12.933+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:13.257 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:12 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:12.937+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:13.257 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:12 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:12.937+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:13.288 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:13.288 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-08T06:06:13.288 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-08T06:06:13.288 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.5 2024-05-08T06:06:13.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:12 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:12.925+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:13.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:12 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:12.925+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:13.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:12 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:12.922+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:13.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:12 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:12.922+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:13.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:13 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:13.258+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:13.589 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:13 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:13.258+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.017575+0000 front 2024-05-08T06:05:41.017652+0000 (oldest deadline 2024-05-08T06:06:06.317636+0000) 2024-05-08T06:06:13.589 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:13 smithi162 systemd[1]: Stopping Ceph osd.5 for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:13.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:13 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:13.458+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:13.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:13 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:13.458+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:13.886 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:13 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:13.597+0000 7f60d936d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:13.886 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:13 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:13.597+0000 7f60d936d700 -1 osd.5 87 *** Got signal Terminated *** 2024-05-08T06:06:13.886 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:13 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:13.597+0000 7f60d936d700 -1 osd.5 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:06:14.268 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:13 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:13.886+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:14.268 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:13 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:13.983+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:14.268 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:13 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:13.983+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:14.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:13 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:13.926+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:14.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:13 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:13.926+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:14.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:13 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:13.901+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:14.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:13 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:13.901+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:14.433 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:13 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:13.957+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:14.567 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:14 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:14.267+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:14.567 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:14 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:14.267+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.017575+0000 front 2024-05-08T06:05:41.017652+0000 (oldest deadline 2024-05-08T06:06:06.317636+0000) 2024-05-08T06:06:14.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:14 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:14.457+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:14.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:14 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:14.457+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:15.249 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:14 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:14.924+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:15.249 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:14 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:14.951+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:15.249 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:14 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:14.951+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:15.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:14 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:14.966+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:15.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:14 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:14.966+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:15.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:14 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:14.878+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:15.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:14 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:14.878+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:15.433 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:14 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:14.950+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:15.433 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:14 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:14.950+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:15.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:15 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:15.248+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:15.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:15 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:15.248+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.017575+0000 front 2024-05-08T06:05:41.017652+0000 (oldest deadline 2024-05-08T06:06:06.317636+0000) 2024-05-08T06:06:15.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:15 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:15.248+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:49.218510+0000 front 2024-05-08T06:05:49.218436+0000 (oldest deadline 2024-05-08T06:06:15.118594+0000) 2024-05-08T06:06:15.925 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:15 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:15.437+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:15.925 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:15 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:15.437+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:15.925 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:15 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:15.437+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:16.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:15 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:15.925+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:16.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:15 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:15.925+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:16.285 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:15 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:15.942+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:16.286 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:15 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:15.962+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:16.286 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:15 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:15.962+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:16.286 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:15 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:15.962+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:16.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:15 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:15.954+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:16.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:15 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:15.954+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:16.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:15 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:15.923+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:16.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:15 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:15.923+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:16.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:16 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:16.285+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:16.567 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:16 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:16.285+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.017575+0000 front 2024-05-08T06:05:41.017652+0000 (oldest deadline 2024-05-08T06:06:06.317636+0000) 2024-05-08T06:06:16.567 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:16 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:16.285+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:49.218510+0000 front 2024-05-08T06:05:49.218436+0000 (oldest deadline 2024-05-08T06:06:15.118594+0000) 2024-05-08T06:06:16.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:16 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:16.429+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:16.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:16 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:16.429+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:16.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:16 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:16.429+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:17.275 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:16 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:16.938+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:17.276 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:16 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:16.938+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:17.276 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:16 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:16.938+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:17.276 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:16 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:16.938+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:17.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:17 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:17.001+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:17.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:17 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:17.001+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:17.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:16 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:16.904+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:17.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:16 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:16.904+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:17.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:16 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:16.904+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:17.411 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:16 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:16.945+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:17.411 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:16 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:16.945+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:17.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:17 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:17.275+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:17.567 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:17 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:17.275+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.017575+0000 front 2024-05-08T06:05:41.017652+0000 (oldest deadline 2024-05-08T06:06:06.317636+0000) 2024-05-08T06:06:17.567 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:17 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:17.275+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:49.218510+0000 front 2024-05-08T06:05:49.218436+0000 (oldest deadline 2024-05-08T06:06:15.118594+0000) 2024-05-08T06:06:17.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:17 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:17.411+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:17.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:17 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:17.411+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:17.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:17 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:17.411+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:18.279 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:17 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:17.976+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:18.279 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:17 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:17.976+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:18.279 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:17 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:17.904+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:18.279 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:17 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:17.904+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:18.279 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:17 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:17.904+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:18.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:17 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:17.994+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:18.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:17 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:17.994+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:18.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:17 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:17.946+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:18.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:17 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:17.946+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:18.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:17 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:17.946+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:18.387 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:17 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:17.935+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:18.387 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:17 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:17.935+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:18.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:18 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:18.278+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.216727+0000 front 2024-05-08T06:05:32.216678+0000 (oldest deadline 2024-05-08T06:05:58.116572+0000) 2024-05-08T06:06:18.567 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:18 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:18.278+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.017575+0000 front 2024-05-08T06:05:41.017652+0000 (oldest deadline 2024-05-08T06:06:06.317636+0000) 2024-05-08T06:06:18.567 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:18 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5[46483]: 2024-05-08T06:06:18.278+0000 7f60d1959700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:49.218510+0000 front 2024-05-08T06:05:49.218436+0000 (oldest deadline 2024-05-08T06:06:15.118594+0000) 2024-05-08T06:06:18.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:18 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:18.387+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:18.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:18 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:18.387+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:18.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:18 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:18.387+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:19.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:18 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:18.896+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:19.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:18 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:18.896+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:19.265 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:19 smithi162 podman[71033]: 2024-05-08 06:06:19.010686986 +0000 UTC m=+5.517072240 container died f10d6d2b9330e2bba8063985fda93db4a03e11584c138bffd738401f4167e435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-05-08T06:06:19.265 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:18 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:18.981+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:19.266 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:18 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:18.981+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:19.266 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:18 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:18.921+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:19.266 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:18 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:18.921+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:19.266 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:18 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:18.921+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:19.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:18 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:18.995+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:19.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:18 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:18.995+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:19.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:18 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:18.936+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:19.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:18 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:18.936+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:19.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:18 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:18.936+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:19.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:19 smithi162 podman[71033]: 2024-05-08 06:06:19.365228671 +0000 UTC m=+5.871613933 container cleanup f10d6d2b9330e2bba8063985fda93db4a03e11584c138bffd738401f4167e435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-08T06:06:19.566 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:19 smithi162 bash[71033]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5 2024-05-08T06:06:19.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:19 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:19.426+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:19.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:19 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:19.426+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:19.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:19 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:19.426+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:19.874 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:19 smithi162 podman[71050]: 2024-05-08 06:06:19.582973895 +0000 UTC m=+0.571228194 container remove f10d6d2b9330e2bba8063985fda93db4a03e11584c138bffd738401f4167e435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:06:20.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:19 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:19.916+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:20.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:19 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:19.916+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:20.219 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:20 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:20.020+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:20.219 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:20 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:20.020+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:20.219 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:19 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:19.874+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:20.219 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:19 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:19.874+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:20.219 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:19 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:19.874+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:20.220 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:19 smithi162 podman[71141]: 2024-05-08 06:06:19.809467883 +0000 UTC m=+0.017882014 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:06:20.220 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:19 smithi162 podman[71141]: 2024-05-08 06:06:19.917394744 +0000 UTC m=+0.125808895 container create fbfb97118b5a237ea285c41815837c31216b3fc5eee8e5049538ae1ef86c6180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-08T06:06:20.220 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:20 smithi162 podman[71141]: 2024-05-08 06:06:20.120006608 +0000 UTC m=+0.328420718 container init fbfb97118b5a237ea285c41815837c31216b3fc5eee8e5049538ae1ef86c6180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5-deactivate, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-08T06:06:20.220 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:20 smithi162 podman[71141]: 2024-05-08 06:06:20.123150245 +0000 UTC m=+0.331564355 container start fbfb97118b5a237ea285c41815837c31216b3fc5eee8e5049538ae1ef86c6180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-08T06:06:20.220 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:20 smithi162 podman[71141]: 2024-05-08 06:06:20.219789778 +0000 UTC m=+0.428203901 container attach fbfb97118b5a237ea285c41815837c31216b3fc5eee8e5049538ae1ef86c6180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-05-08T06:06:20.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:19 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:19.997+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:20.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:19 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:19.997+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:20.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:19 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:19.997+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:53.357604+0000 front 2024-05-08T06:05:53.357267+0000 (oldest deadline 2024-05-08T06:06:19.158030+0000) 2024-05-08T06:06:20.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:19 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:19.966+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:20.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:19 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:19.966+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:20.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:19 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:19.966+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:20.567 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:20 smithi162 podman[71141]: 2024-05-08 06:06:20.416376158 +0000 UTC m=+0.624790264 container died fbfb97118b5a237ea285c41815837c31216b3fc5eee8e5049538ae1ef86c6180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:06:20.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:20 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:20.430+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:20.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:20 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:20.430+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:20.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:20 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:20.430+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:20.970 INFO:journalctl@ceph.osd.5.smithi162.stdout:May 08 06:06:20 smithi162 podman[71167]: 2024-05-08 06:06:20.948616408 +0000 UTC m=+0.526465735 container remove fbfb97118b5a237ea285c41815837c31216b3fc5eee8e5049538ae1ef86c6180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:06:20.971 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:20 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:20.903+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:20.971 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:20 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:20.903+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:20.971 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:20 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:20.903+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:20.975 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.5.service' 2024-05-08T06:06:21.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:21.012 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-08T06:06:21.012 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-08T06:06:21.013 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.6 2024-05-08T06:06:21.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:20 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:20.867+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:21.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:20 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:20.867+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:21.315 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:20 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:20.938+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:21.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:20 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:20.938+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:21.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:20 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:20.938+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:21.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:21 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:21.041+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:21.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:21 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:21.041+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:21.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:21 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:21.041+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:53.357604+0000 front 2024-05-08T06:05:53.357267+0000 (oldest deadline 2024-05-08T06:06:19.158030+0000) 2024-05-08T06:06:21.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:21 smithi112 systemd[1]: Stopping Ceph osd.6 for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:21.317 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:21 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:21.027+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:21.317 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:21 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:21.027+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:21.819 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:21 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:21.478+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:21.819 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:21 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:21.478+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:21.819 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:21 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:21.478+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:21.987 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:21 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:21.571+0000 7f6859000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:21.987 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:21 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:21.571+0000 7f6859000700 -1 osd.6 87 *** Got signal Terminated *** 2024-05-08T06:06:21.987 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:21 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:21.571+0000 7f6859000700 -1 osd.6 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:06:22.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:21 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:21.819+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:22.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:21 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:21.819+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:22.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:22 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:22.053+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:22.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:22 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:22.053+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:22.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:22 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:22.053+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:53.357604+0000 front 2024-05-08T06:05:53.357267+0000 (oldest deadline 2024-05-08T06:06:19.158030+0000) 2024-05-08T06:06:22.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:21 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:21.986+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:22.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:21 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:21.986+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:22.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:21 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:21.986+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:22.317 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:22 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:22.005+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:22.317 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:22 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:22.005+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:22.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:21 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:21.896+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:22.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:21 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:21.896+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:22.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:21 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:21.896+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:22.841 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:22 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:22.520+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:22.841 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:22 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:22.520+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:22.841 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:22 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:22.520+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:23.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:22 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:22.840+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:23.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:22 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:22.840+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:23.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:23 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:23.070+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:23.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:23 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:23.070+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:23.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:23 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:23.070+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:53.357604+0000 front 2024-05-08T06:05:53.357267+0000 (oldest deadline 2024-05-08T06:06:19.158030+0000) 2024-05-08T06:06:23.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:23 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:22.998+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:23.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:23 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:22.998+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:23.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:23 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:22.998+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:23.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:23 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:22.998+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:23.317 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:23 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:23.029+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:23.317 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:23 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:23.029+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:23.317 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:23 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:23.029+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:23.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:22 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:22.891+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:23.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:22 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:22.891+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:23.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:22 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:22.891+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:23.842 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:23 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:23.502+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:23.842 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:23 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:23.502+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:23.843 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:23 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:23.502+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:24.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:23 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:23.842+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:24.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:23 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:23.842+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:24.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:23 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:23.842+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.727382+0000 front 2024-05-08T06:05:59.727122+0000 (oldest deadline 2024-05-08T06:06:23.226911+0000) 2024-05-08T06:06:24.315 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:24 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:24.082+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:24.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:24 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:24.082+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:24.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:24 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:24.082+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:53.357604+0000 front 2024-05-08T06:05:53.357267+0000 (oldest deadline 2024-05-08T06:06:19.158030+0000) 2024-05-08T06:06:24.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:23 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:23.979+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:24.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:23 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:23.979+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:24.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:23 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:23.979+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:24.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:23 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:23.979+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:24.317 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:24 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:24.016+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:24.317 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:24 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:24.016+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:24.317 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:24 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:24.016+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:24.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:23 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:23.914+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:24.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:23 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:23.914+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:24.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:23 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:23.914+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:24.797 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:24 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:24.535+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:24.797 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:24 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:24.535+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:24.797 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:24 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:24.535+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:25.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:24 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:24.797+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:25.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:24 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:24.797+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:25.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:24 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:24.797+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.727382+0000 front 2024-05-08T06:05:59.727122+0000 (oldest deadline 2024-05-08T06:06:23.226911+0000) 2024-05-08T06:06:25.218 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:24 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:24.878+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:25.218 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:24 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:24.878+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:25.218 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:24 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:24.878+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:25.218 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:25 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:25.060+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:25.218 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:25 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:25.060+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:25.218 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:25 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:25.060+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:25.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:25 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:25.047+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:25.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:25 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:25.047+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:25.316 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:25 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:25.047+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:53.357604+0000 front 2024-05-08T06:05:53.357267+0000 (oldest deadline 2024-05-08T06:06:19.158030+0000) 2024-05-08T06:06:25.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:24 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:24.944+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:25.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:24 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:24.944+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:25.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:24 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:24.944+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:25.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:24 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:24.944+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:25.755 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:25 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:25.487+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:25.756 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:25 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:25.487+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:25.756 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:25 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:25.487+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:26.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:25 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:25.756+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:26.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:25 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:25.756+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:26.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:25 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:25.756+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.727382+0000 front 2024-05-08T06:05:59.727122+0000 (oldest deadline 2024-05-08T06:06:23.226911+0000) 2024-05-08T06:06:26.251 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:25 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:25.975+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:26.251 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:25 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:25.975+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:26.251 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:25 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:25.975+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:26.251 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:25 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:25.975+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:26.251 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:26 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:26.030+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.353906+0000 front 2024-05-08T06:05:36.354481+0000 (oldest deadline 2024-05-08T06:06:01.053808+0000) 2024-05-08T06:06:26.251 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:26 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:26.030+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.155849+0000 front 2024-05-08T06:05:45.155673+0000 (oldest deadline 2024-05-08T06:06:08.655809+0000) 2024-05-08T06:06:26.251 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:26 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:26.030+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:53.357604+0000 front 2024-05-08T06:05:53.357267+0000 (oldest deadline 2024-05-08T06:06:19.158030+0000) 2024-05-08T06:06:26.251 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:26 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6[56073]: 2024-05-08T06:06:26.030+0000 7f6854800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.158967+0000 front 2024-05-08T06:05:59.158943+0000 (oldest deadline 2024-05-08T06:06:25.058823+0000) 2024-05-08T06:06:26.316 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:25 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:25.853+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:26.316 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:25 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:25.853+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:26.316 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:25 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:25.853+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:26.316 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:26 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:26.085+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:26.317 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:26 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:26.085+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:26.317 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:26 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:26.085+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:26.765 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:26 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:26.511+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:26.765 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:26 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:26.511+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:26.765 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:26 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:26.511+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:26.765 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:26 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:26.511+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:27.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:26 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:26.765+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:27.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:26 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:26.765+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:27.183 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:26 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:26.765+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.727382+0000 front 2024-05-08T06:05:59.727122+0000 (oldest deadline 2024-05-08T06:06:23.226911+0000) 2024-05-08T06:06:27.222 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:26 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:26.964+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:27.222 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:26 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:26.964+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:27.222 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:26 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:26.964+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:27.222 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:26 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:26.964+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:27.226 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:26 smithi112 podman[120210]: 2024-05-08 06:06:26.964740851 +0000 UTC m=+5.592679374 container died a97c5b3e459251bc17a2814d230756aed781be6286a1f11ba91faf324e12ab0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-08T06:06:27.316 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:26 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:26.882+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:27.316 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:26 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:26.882+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:27.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:26 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:26.882+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:27.317 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:27 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:27.078+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:27.317 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:27 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:27.078+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:27.317 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:27 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:27.078+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:27.729 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:27 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:27.532+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:27.729 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:27 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:27.532+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:27.729 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:27 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:27.532+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:27.729 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:27 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:27.532+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:27.730 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:27 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:27.728+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:27.730 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:27 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:27.728+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:27.730 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:27 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:27.728+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.727382+0000 front 2024-05-08T06:05:59.727122+0000 (oldest deadline 2024-05-08T06:06:23.226911+0000) 2024-05-08T06:06:27.816 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:27 smithi112 podman[120210]: 2024-05-08 06:06:27.48266019 +0000 UTC m=+6.110598766 container cleanup a97c5b3e459251bc17a2814d230756aed781be6286a1f11ba91faf324e12ab0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-08T06:06:27.816 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:27 smithi112 bash[120210]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6 2024-05-08T06:06:27.816 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:27 smithi112 podman[120238]: 2024-05-08 06:06:27.700712236 +0000 UTC m=+0.717849338 container remove a97c5b3e459251bc17a2814d230756aed781be6286a1f11ba91faf324e12ab0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-08T06:06:28.315 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:27 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:27.995+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:28.315 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:27 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:27.995+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:28.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:27 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:27.995+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:28.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:27 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:27.995+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:28.316 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:28 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:28.113+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:28.316 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:28 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:28.113+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:28.316 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:28 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:28.113+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:28.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:27 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:27.865+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:28.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:27 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:27.865+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:28.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:27 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:27.865+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:28.749 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:28 smithi112 podman[120330]: 2024-05-08 06:06:28.351664939 +0000 UTC m=+0.031657689 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:06:28.749 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:28 smithi112 podman[120330]: 2024-05-08 06:06:28.476808264 +0000 UTC m=+0.156801049 container create ecd0ceaa60e52e22aa9657a46282cb5cffa9622d711aa17a4f0334bceacef0b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:06:28.750 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:28 smithi112 podman[120330]: 2024-05-08 06:06:28.748731041 +0000 UTC m=+0.428723853 container init ecd0ceaa60e52e22aa9657a46282cb5cffa9622d711aa17a4f0334bceacef0b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-08T06:06:28.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:28 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:28.756+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:28.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:28 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:28.756+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:28.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:28 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:28.756+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.727382+0000 front 2024-05-08T06:05:59.727122+0000 (oldest deadline 2024-05-08T06:06:23.226911+0000) 2024-05-08T06:06:28.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:28 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:28.511+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:28.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:28 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:28.511+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:28.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:28 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:28.511+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:28.934 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:28 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:28.511+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:29.066 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:28 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:28.956+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:29.066 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:28 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:28.956+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:29.066 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:28 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:28.956+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:29.066 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:28 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:28.956+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:29.066 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:28 smithi112 podman[120330]: 2024-05-08 06:06:28.760598456 +0000 UTC m=+0.440591233 container start ecd0ceaa60e52e22aa9657a46282cb5cffa9622d711aa17a4f0334bceacef0b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-08T06:06:29.066 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:28 smithi112 podman[120330]: 2024-05-08 06:06:28.821158029 +0000 UTC m=+0.501150874 container attach ecd0ceaa60e52e22aa9657a46282cb5cffa9622d711aa17a4f0334bceacef0b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-05-08T06:06:29.081 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:28 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:28.820+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:29.081 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:28 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:28.820+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:29.081 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:28 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:28.820+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:29.510 INFO:journalctl@ceph.osd.6.smithi112.stdout:May 08 06:06:29 smithi112 podman[120330]: 2024-05-08 06:06:29.192651852 +0000 UTC m=+0.872644643 container died ecd0ceaa60e52e22aa9657a46282cb5cffa9622d711aa17a4f0334bceacef0b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-05-08T06:06:29.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:29 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:29.081+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:29.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:29 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:29.081+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:29.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:29 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:29.081+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:29.846 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.6.service' 2024-05-08T06:06:29.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:29 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:29.713+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:29.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:29 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:29.713+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:29.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:29 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:29.713+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.727382+0000 front 2024-05-08T06:05:59.727122+0000 (oldest deadline 2024-05-08T06:06:23.226911+0000) 2024-05-08T06:06:29.934 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:29 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:29.531+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:29.934 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:29 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:29.531+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:29.934 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:29 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:29.531+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:29.934 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:29 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:29.531+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:29.959 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:29.959 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-08T06:06:29.959 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-08T06:06:29.959 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.7 2024-05-08T06:06:30.066 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:29 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:29.991+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:30.066 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:29 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:29.991+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:30.066 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:29 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:29.991+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:30.066 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:29 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:29.991+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:30.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:29 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:29.797+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:30.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:29 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:29.797+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:30.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:29 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:29.797+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:30.255 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:30 smithi150 systemd[1]: Stopping Ceph osd.7 for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:30.552 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:30 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:30.254+0000 7f2b41895700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:30.552 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:30 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:30.254+0000 7f2b41895700 -1 osd.7 87 *** Got signal Terminated *** 2024-05-08T06:06:30.552 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:30 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:30.254+0000 7f2b41895700 -1 osd.7 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:06:30.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:30 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:30.071+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:30.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:30 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:30.071+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:30.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:30 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:30.071+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:30.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:30 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:30.552+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:30.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:30 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:30.552+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:30.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:30 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:30.552+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:30.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:30 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:30.552+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:30.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:30 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:30.708+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:30.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:30 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:30.708+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:30.934 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:30 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:30.708+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.727382+0000 front 2024-05-08T06:05:59.727122+0000 (oldest deadline 2024-05-08T06:06:23.226911+0000) 2024-05-08T06:06:31.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:30 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:30.767+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:31.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:30 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:30.767+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:31.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:30 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:30.767+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:31.315 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:31 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:31.038+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:31.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:31 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:31.038+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:31.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:31 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:31.038+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:31.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:31 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:31.038+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:31.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:31 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:31.104+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:31.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:31 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:31.104+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:31.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:31 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:31.104+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:31.934 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:31 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:31.684+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:31.935 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:31 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:31.684+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:31.935 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:31 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:31.684+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.727382+0000 front 2024-05-08T06:05:59.727122+0000 (oldest deadline 2024-05-08T06:06:23.226911+0000) 2024-05-08T06:06:31.935 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:31 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:31.541+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:31.935 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:31 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:31.541+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:31.935 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:31 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:31.541+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:31.935 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:31 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:31.541+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:32.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:31 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:31.809+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:32.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:31 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:31.809+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:32.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:31 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:31.809+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:32.315 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:32 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:32.043+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:32.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:32 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:32.043+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:32.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:32 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:32.043+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:32.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:32 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:32.043+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:32.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:32 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:32.136+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:32.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:32 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:32.136+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:32.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:32 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:32.136+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:32.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:32 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:32.491+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:32.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:32 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:32.491+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:32.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:32 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:32.491+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:32.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:32 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:32.491+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:32.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:32 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:32.731+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:32.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:32 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:32.731+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:32.934 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:32 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:32.731+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.727382+0000 front 2024-05-08T06:05:59.727122+0000 (oldest deadline 2024-05-08T06:06:23.226911+0000) 2024-05-08T06:06:32.934 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:32 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:32.731+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.927721+0000 front 2024-05-08T06:06:07.927655+0000 (oldest deadline 2024-05-08T06:06:32.627612+0000) 2024-05-08T06:06:33.179 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:32 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:32.831+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:33.179 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:32 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:32.831+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:33.179 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:32 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:32.831+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:33.179 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:32 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:32.831+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:33.566 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:33 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:33.081+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:33.566 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:33 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:33.081+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:33.566 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:33 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:33.081+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:33.566 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:33 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:33.081+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:33.566 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:33 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:33.081+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:33.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:33 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:33.179+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:33.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:33 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:33.179+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:33.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:33 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:33.179+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:33.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:33 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:33.710+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:33.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:33 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:33.710+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:33.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:33 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:33.710+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.727382+0000 front 2024-05-08T06:05:59.727122+0000 (oldest deadline 2024-05-08T06:06:23.226911+0000) 2024-05-08T06:06:33.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:33 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:33.710+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.927721+0000 front 2024-05-08T06:06:07.927655+0000 (oldest deadline 2024-05-08T06:06:32.627612+0000) 2024-05-08T06:06:33.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:33 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:33.487+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:33.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:33 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:33.487+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:33.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:33 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:33.487+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:33.934 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:33 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:33.487+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:33.934 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:33 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:33.487+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:34.177 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:33 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:33.844+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:34.177 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:33 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:33.844+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:34.178 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:33 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:33.844+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:34.178 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:33 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:33.844+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:34.315 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:34 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:34.058+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:34.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:34 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:34.058+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:34.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:34 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:34.058+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:34.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:34 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:34.058+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:34.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:34 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:34.058+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:34.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:34 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:34.177+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:34.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:34 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:34.177+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:34.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:34 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:34.177+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:34.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:34 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:34.689+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:45.124359+0000 front 2024-05-08T06:05:45.124388+0000 (oldest deadline 2024-05-08T06:06:11.024576+0000) 2024-05-08T06:06:34.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:34 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:34.689+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.525785+0000 front 2024-05-08T06:05:51.525772+0000 (oldest deadline 2024-05-08T06:06:14.425714+0000) 2024-05-08T06:06:34.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:34 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:34.689+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.727382+0000 front 2024-05-08T06:05:59.727122+0000 (oldest deadline 2024-05-08T06:06:23.226911+0000) 2024-05-08T06:06:34.933 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:34 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7[52351]: 2024-05-08T06:06:34.689+0000 7f2b3de89700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.927721+0000 front 2024-05-08T06:06:07.927655+0000 (oldest deadline 2024-05-08T06:06:32.627612+0000) 2024-05-08T06:06:34.934 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:34 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:34.532+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:34.934 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:34 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:34.532+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:34.934 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:34 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:34.532+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:34.934 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:34 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:34.532+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:34.934 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:34 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:34.532+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:35.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:34 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:34.816+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:35.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:34 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:34.816+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:35.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:34 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:34.816+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:35.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:34 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:34.816+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:35.565 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:35 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:35.078+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:35.566 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:35 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:35.078+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:35.566 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:35 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:35.078+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:35.566 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:35 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:35.078+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:35.566 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:35 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:35.078+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:35.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:35 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:35.182+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:35.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:35 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:35.182+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:35.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:35 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:35.182+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:35.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:35 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:35.182+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:09.247857+0000 front 2024-05-08T06:06:09.247979+0000 (oldest deadline 2024-05-08T06:06:34.547774+0000) 2024-05-08T06:06:35.889 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:35 smithi150 podman[69370]: 2024-05-08 06:06:35.626700441 +0000 UTC m=+5.472531598 container died 94f25f634d848bc5cc9ad5b338a8540f3f910804a3abbfa46f931f70369a1cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-08T06:06:35.889 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:35 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:35.534+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:35.889 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:35 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:35.534+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:35.889 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:35 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:35.534+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:35.889 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:35 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:35.534+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:35.889 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:35 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:35.534+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:36.147 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:35 smithi150 podman[69370]: 2024-05-08 06:06:35.963352084 +0000 UTC m=+5.809183237 container cleanup 94f25f634d848bc5cc9ad5b338a8540f3f910804a3abbfa46f931f70369a1cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-08T06:06:36.148 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:35 smithi150 bash[69370]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7 2024-05-08T06:06:36.166 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:35 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:35.849+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:36.166 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:35 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:35.849+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:36.166 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:35 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:35.849+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:36.166 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:35 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:35.849+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:36.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:36.042+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:36.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:36.042+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:36.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:36.042+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:36.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:36.042+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:36.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:36 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:36.042+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:36.433 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:36 smithi150 podman[69387]: 2024-05-08 06:06:36.148416725 +0000 UTC m=+0.524199734 container remove 94f25f634d848bc5cc9ad5b338a8540f3f910804a3abbfa46f931f70369a1cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:06:36.487 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:36 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:36.166+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:36.487 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:36 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:36.166+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:36.487 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:36 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:36.166+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:36.487 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:36 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:36.166+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:09.247857+0000 front 2024-05-08T06:06:09.247979+0000 (oldest deadline 2024-05-08T06:06:34.547774+0000) 2024-05-08T06:06:36.852 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:36 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:36.497+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:36.852 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:36 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:36.497+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:36.852 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:36 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:36.497+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:36.852 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:36 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:36.497+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:36.852 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:36 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:36.497+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:36.852 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:36 smithi150 podman[69480]: 2024-05-08 06:06:36.47539732 +0000 UTC m=+0.018114065 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:06:36.852 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:36 smithi150 podman[69480]: 2024-05-08 06:06:36.615912581 +0000 UTC m=+0.158629319 container create 91003825deed795cdba000c2395bbc8e5dc40d0882f5cfe2aafd6ca887250e98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-05-08T06:06:36.852 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:36 smithi150 podman[69480]: 2024-05-08 06:06:36.852309867 +0000 UTC m=+0.395026600 container init 91003825deed795cdba000c2395bbc8e5dc40d0882f5cfe2aafd6ca887250e98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True) 2024-05-08T06:06:37.148 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:36 smithi150 podman[69480]: 2024-05-08 06:06:36.855722497 +0000 UTC m=+0.398439237 container start 91003825deed795cdba000c2395bbc8e5dc40d0882f5cfe2aafd6ca887250e98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2) 2024-05-08T06:06:37.149 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:36 smithi150 podman[69480]: 2024-05-08 06:06:36.935319047 +0000 UTC m=+0.478035789 container attach 91003825deed795cdba000c2395bbc8e5dc40d0882f5cfe2aafd6ca887250e98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-08T06:06:37.197 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:36 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:36.818+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:37.198 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:36 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:36.818+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:37.198 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:36 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:36.818+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:37.198 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:36 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:36.818+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:37.310 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:37 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:37.030+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:37.310 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:37 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:37.030+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:37.310 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:37 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:37.030+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:37.310 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:37 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:37.030+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:37.310 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:37 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:37.030+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:37.426 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 08 06:06:37 smithi150 podman[69480]: 2024-05-08 06:06:37.149455452 +0000 UTC m=+0.692172192 container died 91003825deed795cdba000c2395bbc8e5dc40d0882f5cfe2aafd6ca887250e98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:06:37.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:37 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:37.198+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:37.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:37 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:37.198+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:37.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:37 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:37.198+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:37.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:37 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:37.198+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:09.247857+0000 front 2024-05-08T06:06:09.247979+0000 (oldest deadline 2024-05-08T06:06:34.547774+0000) 2024-05-08T06:06:37.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:37 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:37.540+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:37.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:37 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:37.540+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:37.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:37 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:37.540+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:37.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:37 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:37.540+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:37.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:37 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:37.540+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:37.784 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.7.service' 2024-05-08T06:06:37.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:37.820 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-08T06:06:37.821 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-08T06:06:37.821 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.8 2024-05-08T06:06:38.067 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:37 smithi162 systemd[1]: Stopping Ceph osd.8 for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:38.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:37 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:37.780+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:38.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:37 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:37.780+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:38.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:37 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:37.780+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:38.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:37 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:37.780+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:38.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:37 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:37.988+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:38.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:37 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:37.988+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:38.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:37 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:37.988+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:38.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:37 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:37.988+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:38.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:37 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:37.988+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:38.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:38 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:38.108+0000 7f51c5c98700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:38.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:38 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:38.108+0000 7f51c5c98700 -1 osd.8 87 *** Got signal Terminated *** 2024-05-08T06:06:38.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:38 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:38.108+0000 7f51c5c98700 -1 osd.8 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:06:38.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:38 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:38.155+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:38.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:38 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:38.155+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:38.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:38 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:38.155+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:38.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:38 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:38.155+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:09.247857+0000 front 2024-05-08T06:06:09.247979+0000 (oldest deadline 2024-05-08T06:06:34.547774+0000) 2024-05-08T06:06:38.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:38 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:38.559+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:38.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:38 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:38.559+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:38.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:38 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:38.559+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:38.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:38 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:38.559+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:38.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:38 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:38.559+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:39.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:38 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:38.739+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:39.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:38 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:38.739+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:39.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:38 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:38.739+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:39.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:38 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:38.739+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:39.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:38 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:38.977+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:39.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:38 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:38.977+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:39.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:38 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:38.977+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:39.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:38 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:38.977+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:39.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:38 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:38.977+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:39.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:39 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:39.181+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:39.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:39 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:39.181+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:39.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:39 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:39.181+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:39.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:39 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:39.181+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:09.247857+0000 front 2024-05-08T06:06:09.247979+0000 (oldest deadline 2024-05-08T06:06:34.547774+0000) 2024-05-08T06:06:39.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:39 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:39.181+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.549270+0000 front 2024-05-08T06:06:18.549289+0000 (oldest deadline 2024-05-08T06:06:39.049479+0000) 2024-05-08T06:06:39.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:39 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:39.517+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:39.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:39 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:39.517+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:39.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:39 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:39.517+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:39.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:39 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:39.517+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:39.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:39 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:39.517+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:40.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:39 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:39.698+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:40.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:39 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:39.698+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:40.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:39 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:39.698+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:40.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:39 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:39.698+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:40.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:39 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:39.978+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:40.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:39 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:39.978+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:40.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:39 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:39.978+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:40.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:39 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:39.978+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:40.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:39 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:39.978+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:40.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:39 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:39.978+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:16.398434+0000 front 2024-05-08T06:06:16.398364+0000 (oldest deadline 2024-05-08T06:06:39.898103+0000) 2024-05-08T06:06:40.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:40 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:40.223+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:40.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:40 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:40.223+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:40.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:40 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:40.223+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:40.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:40 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:40.223+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:09.247857+0000 front 2024-05-08T06:06:09.247979+0000 (oldest deadline 2024-05-08T06:06:34.547774+0000) 2024-05-08T06:06:40.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:40 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:40.223+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.549270+0000 front 2024-05-08T06:06:18.549289+0000 (oldest deadline 2024-05-08T06:06:39.049479+0000) 2024-05-08T06:06:40.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:40 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:40.547+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:40.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:40 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:40.547+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:40.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:40 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:40.547+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:40.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:40 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:40.547+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:40.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:40 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:40.547+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:41.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:40 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:40.723+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:41.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:40 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:40.723+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:41.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:40 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:40.723+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:41.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:40 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:40.723+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:41.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:40 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:40.982+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:41.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:40 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:40.982+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:41.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:40 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:40.982+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:41.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:40 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:40.982+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:41.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:40 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:40.982+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:41.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:40 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:40.982+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:16.398434+0000 front 2024-05-08T06:06:16.398364+0000 (oldest deadline 2024-05-08T06:06:39.898103+0000) 2024-05-08T06:06:41.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:41 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:41.237+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:41.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:41 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:41.237+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:41.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:41 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:41.237+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:41.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:41 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:41.237+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:09.247857+0000 front 2024-05-08T06:06:09.247979+0000 (oldest deadline 2024-05-08T06:06:34.547774+0000) 2024-05-08T06:06:41.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:41 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:41.237+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.549270+0000 front 2024-05-08T06:06:18.549289+0000 (oldest deadline 2024-05-08T06:06:39.049479+0000) 2024-05-08T06:06:41.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:41 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:41.572+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:41.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:41 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:41.572+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:41.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:41 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:41.572+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:41.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:41 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:41.572+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:41.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:41 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:41.572+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:42.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:41 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:41.703+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:42.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:41 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:41.703+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:42.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:41 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:41.703+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:42.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:41 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:41.703+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:42.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:41 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:41.948+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:42.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:41 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:41.948+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:42.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:41 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:41.948+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:42.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:41 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:41.948+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:42.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:41 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:41.948+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:42.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:41 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:41.948+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:16.398434+0000 front 2024-05-08T06:06:16.398364+0000 (oldest deadline 2024-05-08T06:06:39.898103+0000) 2024-05-08T06:06:42.566 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:42 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:42.262+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.843814+0000 front 2024-05-08T06:05:44.843795+0000 (oldest deadline 2024-05-08T06:06:08.343903+0000) 2024-05-08T06:06:42.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:42 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:42.262+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.444741+0000 front 2024-05-08T06:05:52.444824+0000 (oldest deadline 2024-05-08T06:06:17.144687+0000) 2024-05-08T06:06:42.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:42 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:42.262+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:57.645762+0000 front 2024-05-08T06:05:57.645612+0000 (oldest deadline 2024-05-08T06:06:22.246095+0000) 2024-05-08T06:06:42.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:42 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:42.262+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:09.247857+0000 front 2024-05-08T06:06:09.247979+0000 (oldest deadline 2024-05-08T06:06:34.547774+0000) 2024-05-08T06:06:42.567 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:42 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8[53322]: 2024-05-08T06:06:42.262+0000 7f51be284700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.549270+0000 front 2024-05-08T06:06:18.549289+0000 (oldest deadline 2024-05-08T06:06:39.049479+0000) 2024-05-08T06:06:42.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:42 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:42.539+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:42.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:42 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:42.539+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:42.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:42 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:42.539+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:42.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:42 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:42.539+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:42.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:42 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:42.539+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:42.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:42 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:42.539+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:43.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:42 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:42.729+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:43.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:42 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:42.729+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:43.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:42 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:42.729+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:43.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:42 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:42.729+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:43.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:42 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:42.985+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:43.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:42 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:42.985+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:43.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:42 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:42.985+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:43.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:42 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:42.985+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:43.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:42 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:42.985+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:43.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:42 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:42.985+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:16.398434+0000 front 2024-05-08T06:06:16.398364+0000 (oldest deadline 2024-05-08T06:06:39.898103+0000) 2024-05-08T06:06:43.833 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:43 smithi162 podman[71259]: 2024-05-08 06:06:43.578949485 +0000 UTC m=+5.564780742 container died 4853414ed854775ae1122d89917e4cca2a0c93b3b20638edd67817bde130c717 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-08T06:06:43.833 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:43 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:43.767+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:43.833 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:43 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:43.767+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:43.833 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:43 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:43.767+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:43.833 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:43 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:43.767+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:43.833 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:43 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:43.767+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:43.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:43 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:43.585+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:43.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:43 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:43.585+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:43.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:43 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:43.585+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:43.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:43 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:43.585+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:43.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:43 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:43.585+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:43.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:43 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:43.585+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:44.216 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:43 smithi162 podman[71259]: 2024-05-08 06:06:43.93230952 +0000 UTC m=+5.918140784 container cleanup 4853414ed854775ae1122d89917e4cca2a0c93b3b20638edd67817bde130c717 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:06:44.216 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:43 smithi162 bash[71259]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8 2024-05-08T06:06:44.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:43 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:43.972+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:44.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:43 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:43.972+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:44.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:43 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:43.972+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:44.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:43 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:43.972+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:44.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:43 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:43.972+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:44.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:43 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:43.972+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:16.398434+0000 front 2024-05-08T06:06:16.398364+0000 (oldest deadline 2024-05-08T06:06:39.898103+0000) 2024-05-08T06:06:44.549 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:44 smithi162 podman[71276]: 2024-05-08 06:06:44.217126795 +0000 UTC m=+0.632863343 container remove 4853414ed854775ae1122d89917e4cca2a0c93b3b20638edd67817bde130c717 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-08T06:06:44.808 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:44 smithi162 podman[71366]: 2024-05-08 06:06:44.452304103 +0000 UTC m=+0.017939490 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:06:44.809 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:44 smithi162 podman[71366]: 2024-05-08 06:06:44.551100222 +0000 UTC m=+0.116735597 container create c57743b6d34d395174a0a82ce60eb95ba75ac30e3dcb84d1a84a2a5ce88ca61d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:06:44.809 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:44 smithi162 podman[71366]: 2024-05-08 06:06:44.780393684 +0000 UTC m=+0.346029064 container init c57743b6d34d395174a0a82ce60eb95ba75ac30e3dcb84d1a84a2a5ce88ca61d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8-deactivate, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS) 2024-05-08T06:06:44.809 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:44 smithi162 podman[71366]: 2024-05-08 06:06:44.783705297 +0000 UTC m=+0.349340673 container start c57743b6d34d395174a0a82ce60eb95ba75ac30e3dcb84d1a84a2a5ce88ca61d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-08T06:06:44.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:44 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:44.552+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:44.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:44 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:44.552+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:44.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:44 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:44.552+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:44.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:44 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:44.552+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:44.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:44 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:44.552+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:44.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:44 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:44.552+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:45.066 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:44 smithi162 podman[71366]: 2024-05-08 06:06:44.871613095 +0000 UTC m=+0.437248483 container attach c57743b6d34d395174a0a82ce60eb95ba75ac30e3dcb84d1a84a2a5ce88ca61d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True) 2024-05-08T06:06:45.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:44 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:44.808+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:45.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:44 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:44.808+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:45.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:44 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:44.808+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:45.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:44 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:44.808+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:45.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:44 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:44.808+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:45.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:44 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:44.941+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:45.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:44 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:44.941+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:45.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:44 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:44.941+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:45.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:44 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:44.941+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:45.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:44 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:44.941+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:45.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:44 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:44.941+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:16.398434+0000 front 2024-05-08T06:06:16.398364+0000 (oldest deadline 2024-05-08T06:06:39.898103+0000) 2024-05-08T06:06:45.340 INFO:journalctl@ceph.osd.8.smithi162.stdout:May 08 06:06:45 smithi162 podman[71366]: 2024-05-08 06:06:45.077215891 +0000 UTC m=+0.642851267 container died c57743b6d34d395174a0a82ce60eb95ba75ac30e3dcb84d1a84a2a5ce88ca61d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-08T06:06:45.688 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.8.service' 2024-05-08T06:06:45.725 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:45.725 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-08T06:06:45.726 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-08T06:06:45.726 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.9 2024-05-08T06:06:45.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:45 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:45.504+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:45.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:45 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:45.504+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:45.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:45 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:45.504+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:45.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:45 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:45.504+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:45.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:45 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:45.504+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:45.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:45 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:45.504+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:46.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:45 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:45.820+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:46.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:45 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:45.820+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:46.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:45 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:45.820+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:46.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:45 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:45.820+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:46.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:45 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:45.820+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:46.067 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:45 smithi112 systemd[1]: Stopping Ceph osd.9 for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:46.067 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:45 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:45.974+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:46.067 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:45 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:45.974+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:46.067 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:45 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:45.974+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:46.067 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:45 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:45.974+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:46.067 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:45 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:45.974+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:46.067 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:45 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:45.974+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:16.398434+0000 front 2024-05-08T06:06:16.398364+0000 (oldest deadline 2024-05-08T06:06:39.898103+0000) 2024-05-08T06:06:46.565 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:46 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:46.308+0000 7f5d92c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:46.566 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:46 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:46.308+0000 7f5d92c00700 -1 osd.9 87 *** Got signal Terminated *** 2024-05-08T06:06:46.566 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:46 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:46.308+0000 7f5d92c00700 -1 osd.9 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:06:46.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:46 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:46.528+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:46.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:46 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:46.528+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:46.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:46 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:46.528+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:46.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:46 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:46.528+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:46.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:46 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:46.528+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:46.934 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:46 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:46.528+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:47.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:46 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:46.782+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:47.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:46 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:46.782+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:47.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:46 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:46.782+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:47.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:46 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:46.782+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:47.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:46 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:46.782+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:47.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:47 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:46.998+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:47.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:47 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:46.998+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:47.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:47 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:46.998+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:47.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:47 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:46.998+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:47.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:47 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:46.998+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:47.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:47 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:46.998+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:16.398434+0000 front 2024-05-08T06:06:16.398364+0000 (oldest deadline 2024-05-08T06:06:39.898103+0000) 2024-05-08T06:06:47.870 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:47 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:47.541+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:47.870 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:47 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:47.541+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:47.870 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:47 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:47.541+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:47.870 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:47 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:47.541+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:47.870 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:47 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:47.541+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:47.870 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:47 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:47.541+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:48.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:47 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:47.795+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:48.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:47 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:47.795+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:48.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:47 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:47.795+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:48.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:47 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:47.795+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:48.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:47 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:47.795+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:48.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:48 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:48.028+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:48.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:48 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:48.028+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:48.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:48 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:48.028+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:48.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:48 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:48.028+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:48.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:48 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:48.028+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:48.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:48 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:48.028+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:16.398434+0000 front 2024-05-08T06:06:16.398364+0000 (oldest deadline 2024-05-08T06:06:39.898103+0000) 2024-05-08T06:06:48.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:48 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:48.534+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:48.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:48 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:48.534+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:48.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:48 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:48.534+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:48.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:48 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:48.534+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:48.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:48 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:48.534+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:48.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:48 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:48.534+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:48.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:48 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:48.534+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:25.292034+0000 front 2024-05-08T06:06:25.291890+0000 (oldest deadline 2024-05-08T06:06:47.591718+0000) 2024-05-08T06:06:49.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:48 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:48.755+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:49.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:48 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:48.755+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:49.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:48 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:48.755+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:49.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:48 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:48.755+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:49.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:48 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:48.755+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:49.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:49 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:49.011+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:49.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:49 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:49.011+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:49.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:49 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:49.011+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:49.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:49 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:49.011+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:49.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:49 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:49.011+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:49.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:49 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:49.011+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:16.398434+0000 front 2024-05-08T06:06:16.398364+0000 (oldest deadline 2024-05-08T06:06:39.898103+0000) 2024-05-08T06:06:49.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:49 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:49.495+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:49.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:49 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:49.495+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:49.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:49 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:49.495+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:49.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:49 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:49.495+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:49.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:49 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:49.495+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:49.934 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:49 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:49.495+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:49.934 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:49 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:49.495+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:25.292034+0000 front 2024-05-08T06:06:25.291890+0000 (oldest deadline 2024-05-08T06:06:47.591718+0000) 2024-05-08T06:06:50.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:49 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:49.755+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:50.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:49 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:49.755+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:50.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:49 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:49.755+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:50.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:49 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:49.755+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:50.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:49 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:49.755+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:50.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:49 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:49.755+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:06:50.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:50 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:50.016+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:50.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:50 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:50.016+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:50.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:50 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:50.016+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:50.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:50 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:50.016+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:50.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:50 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:50.016+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:50.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:50 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:50.016+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:16.398434+0000 front 2024-05-08T06:06:16.398364+0000 (oldest deadline 2024-05-08T06:06:39.898103+0000) 2024-05-08T06:06:50.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:50 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:50.016+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:25.200086+0000 front 2024-05-08T06:06:25.199927+0000 (oldest deadline 2024-05-08T06:06:49.899835+0000) 2024-05-08T06:06:50.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:50 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:50.449+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:50.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:50 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:50.449+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:50.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:50 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:50.449+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:50.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:50 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:50.449+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:50.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:50 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:50.449+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:50.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:50 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:50.449+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:50.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:50 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:50.449+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:25.292034+0000 front 2024-05-08T06:06:25.291890+0000 (oldest deadline 2024-05-08T06:06:47.591718+0000) 2024-05-08T06:06:51.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:50 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:50.738+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:51.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:50 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:50.738+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:51.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:50 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:50.738+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:51.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:50 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:50.738+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:51.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:50 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:50.738+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:51.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:50 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:50.738+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:06:51.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:51 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:51.001+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:35.087164+0000 front 2024-05-08T06:05:35.087528+0000 (oldest deadline 2024-05-08T06:06:00.986619+0000) 2024-05-08T06:06:51.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:51 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:51.001+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:41.488623+0000 front 2024-05-08T06:05:41.488864+0000 (oldest deadline 2024-05-08T06:06:06.188817+0000) 2024-05-08T06:06:51.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:51 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:51.001+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:51.391580+0000 front 2024-05-08T06:05:51.391541+0000 (oldest deadline 2024-05-08T06:06:16.091481+0000) 2024-05-08T06:06:51.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:51 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:51.001+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:58.393238+0000 front 2024-05-08T06:05:58.393460+0000 (oldest deadline 2024-05-08T06:06:22.493277+0000) 2024-05-08T06:06:51.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:51 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:51.001+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:08.796311+0000 front 2024-05-08T06:06:08.796640+0000 (oldest deadline 2024-05-08T06:06:32.896516+0000) 2024-05-08T06:06:51.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:51 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:51.001+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:16.398434+0000 front 2024-05-08T06:06:16.398364+0000 (oldest deadline 2024-05-08T06:06:39.898103+0000) 2024-05-08T06:06:51.316 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:51 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9[62649]: 2024-05-08T06:06:51.001+0000 7f5d8e400700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:25.200086+0000 front 2024-05-08T06:06:25.199927+0000 (oldest deadline 2024-05-08T06:06:49.899835+0000) 2024-05-08T06:06:51.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:51 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:51.444+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:51.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:51 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:51.444+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:51.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:51 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:51.444+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:51.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:51 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:51.444+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:51.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:51 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:51.444+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:51.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:51 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:51.444+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:51.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:51 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:51.444+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:25.292034+0000 front 2024-05-08T06:06:25.291890+0000 (oldest deadline 2024-05-08T06:06:47.591718+0000) 2024-05-08T06:06:52.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:51 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:51.779+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:52.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:51 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:51.779+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:52.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:51 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:51.779+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:52.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:51 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:51.779+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:52.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:51 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:51.779+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:52.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:51 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:51.779+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:06:52.206 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:51 smithi112 podman[120434]: 2024-05-08 06:06:51.898181627 +0000 UTC m=+5.763279479 container died e0450fefd79ce9f5168c631c7d181ec5a3a7c14359e1981280a5ec81a10b4bd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5) 2024-05-08T06:06:52.565 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:52 smithi112 podman[120434]: 2024-05-08 06:06:52.384463171 +0000 UTC m=+6.249561048 container cleanup e0450fefd79ce9f5168c631c7d181ec5a3a7c14359e1981280a5ec81a10b4bd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-08T06:06:52.565 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:52 smithi112 bash[120434]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9 2024-05-08T06:06:52.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:52 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:52.409+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:52.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:52 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:52.409+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:52.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:52 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:52.409+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:52.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:52 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:52.409+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:52.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:52 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:52.409+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:52.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:52 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:52.409+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:52.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:52 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:52.409+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:25.292034+0000 front 2024-05-08T06:06:25.291890+0000 (oldest deadline 2024-05-08T06:06:47.591718+0000) 2024-05-08T06:06:53.065 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:52 smithi112 podman[120454]: 2024-05-08 06:06:52.669171871 +0000 UTC m=+0.758247764 container remove e0450fefd79ce9f5168c631c7d181ec5a3a7c14359e1981280a5ec81a10b4bd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-08T06:06:53.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:52 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:52.742+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:53.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:52 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:52.742+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:53.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:52 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:52.742+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:53.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:52 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:52.742+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:53.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:52 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:52.742+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:53.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:52 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:52.742+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:06:53.566 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:53 smithi112 podman[120539]: 2024-05-08 06:06:53.198108755 +0000 UTC m=+0.031548595 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:06:53.566 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:53 smithi112 podman[120539]: 2024-05-08 06:06:53.342515165 +0000 UTC m=+0.175955012 container create d69cfc49f9eab3c87d531428562fb432feac7698858498d1e9d8bf8d2b98f3ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-08T06:06:53.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:53 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:53.366+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:53.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:53 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:53.366+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:53.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:53 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:53.366+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:53.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:53 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:53.366+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:53.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:53 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:53.366+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:53.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:53 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:53.366+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:53.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:53 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:53.366+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:25.292034+0000 front 2024-05-08T06:06:25.291890+0000 (oldest deadline 2024-05-08T06:06:47.591718+0000) 2024-05-08T06:06:54.065 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:53 smithi112 podman[120539]: 2024-05-08 06:06:53.624108093 +0000 UTC m=+0.457547976 container init d69cfc49f9eab3c87d531428562fb432feac7698858498d1e9d8bf8d2b98f3ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-05-08T06:06:54.066 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:53 smithi112 podman[120539]: 2024-05-08 06:06:53.635088738 +0000 UTC m=+0.468528614 container start d69cfc49f9eab3c87d531428562fb432feac7698858498d1e9d8bf8d2b98f3ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9-deactivate, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:06:54.066 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:53 smithi112 podman[120539]: 2024-05-08 06:06:53.714641383 +0000 UTC m=+0.548081272 container attach d69cfc49f9eab3c87d531428562fb432feac7698858498d1e9d8bf8d2b98f3ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-08T06:06:54.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:53 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:53.729+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:54.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:53 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:53.729+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:54.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:53 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:53.729+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:54.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:53 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:53.729+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:54.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:53 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:53.729+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:54.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:53 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:53.729+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:06:54.453 INFO:journalctl@ceph.osd.9.smithi112.stdout:May 08 06:06:54 smithi112 podman[120539]: 2024-05-08 06:06:54.084804649 +0000 UTC m=+0.918244488 container died d69cfc49f9eab3c87d531428562fb432feac7698858498d1e9d8bf8d2b98f3ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-9-deactivate, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:06:54.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:54 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:54.403+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:54.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:54 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:54.403+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:54.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:54 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:54.403+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:54.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:54 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:54.403+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:54.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:54 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:54.403+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:54.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:54 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:54.403+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:54.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:54 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:54.403+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:25.292034+0000 front 2024-05-08T06:06:25.291890+0000 (oldest deadline 2024-05-08T06:06:47.591718+0000) 2024-05-08T06:06:54.947 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.9.service' 2024-05-08T06:06:55.044 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:55.045 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-08T06:06:55.045 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-08T06:06:55.045 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.10 2024-05-08T06:06:55.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:54 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:54.694+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:55.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:54 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:54.694+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:55.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:54 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:54.694+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:55.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:54 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:54.694+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:55.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:54 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:54.694+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:55.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:54 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:54.694+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:06:55.342 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:55 smithi150 systemd[1]: Stopping Ceph osd.10 for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:55.342 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:55 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:55.341+0000 7f307bc96700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:55.342 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:55 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:55.341+0000 7f307bc96700 -1 osd.10 87 *** Got signal Terminated *** 2024-05-08T06:06:55.342 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:55 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:55.341+0000 7f307bc96700 -1 osd.10 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:06:55.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:55 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:55.366+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:55.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:55 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:55.366+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:55.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:55 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:55.366+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:55.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:55 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:55.366+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:55.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:55 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:55.366+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:55.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:55 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:55.366+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:55.684 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:55 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:55.366+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:25.292034+0000 front 2024-05-08T06:06:25.291890+0000 (oldest deadline 2024-05-08T06:06:47.591718+0000) 2024-05-08T06:06:55.970 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:55 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:55.669+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:55.971 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:55 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:55.669+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:55.971 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:55 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:55.669+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:55.971 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:55 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:55.669+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:55.971 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:55 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:55.669+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:55.971 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:55 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:55.669+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:06:56.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:56 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:56.413+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:56.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:56 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:56.413+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:56.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:56 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:56.413+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:56.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:56 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:56.413+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:56.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:56 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:56.413+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:56.684 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:56 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:56.413+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:56.684 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:56 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:56.413+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:25.292034+0000 front 2024-05-08T06:06:25.291890+0000 (oldest deadline 2024-05-08T06:06:47.591718+0000) 2024-05-08T06:06:57.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:56 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:56.663+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:57.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:56 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:56.663+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:57.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:56 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:56.663+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:57.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:56 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:56.663+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:57.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:56 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:56.663+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:57.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:56 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:56.663+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:06:57.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:57 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:57.372+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:57.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:57 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:57.372+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:57.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:57 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:57.372+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:57.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:57 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:57.372+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:57.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:57 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:57.372+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:57.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:57 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:57.372+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:57.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:57 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:57.372+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:25.292034+0000 front 2024-05-08T06:06:25.291890+0000 (oldest deadline 2024-05-08T06:06:47.591718+0000) 2024-05-08T06:06:58.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:57 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:57.697+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:58.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:57 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:57.697+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:58.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:57 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:57.697+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:58.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:57 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:57.697+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:58.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:57 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:57.697+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:58.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:57 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:57.697+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:06:58.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:58 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:58.396+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:58.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:58 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:58.396+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:58.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:58 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:58.396+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:58.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:58 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:58.396+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:58.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:58 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:58.396+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:58.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:58 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:58.396+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:58.683 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:58 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:58.396+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:25.292034+0000 front 2024-05-08T06:06:25.291890+0000 (oldest deadline 2024-05-08T06:06:47.591718+0000) 2024-05-08T06:06:59.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:58 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:58.717+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:06:59.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:58 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:58.717+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:06:59.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:58 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:58.717+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:06:59.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:58 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:58.717+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:06:59.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:58 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:58.717+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:06:59.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:58 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:58.717+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:06:59.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:59 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:59.443+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:36.384677+0000 front 2024-05-08T06:05:36.384762+0000 (oldest deadline 2024-05-08T06:05:58.684667+0000) 2024-05-08T06:06:59.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:59 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:59.443+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:44.585397+0000 front 2024-05-08T06:05:44.585562+0000 (oldest deadline 2024-05-08T06:06:10.485563+0000) 2024-05-08T06:06:59.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:59 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:59.443+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:50.486412+0000 front 2024-05-08T06:05:50.486347+0000 (oldest deadline 2024-05-08T06:06:15.186128+0000) 2024-05-08T06:06:59.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:59 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:59.443+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-05-08T06:05:59.787800+0000 front 2024-05-08T06:05:59.787939+0000 (oldest deadline 2024-05-08T06:06:25.687524+0000) 2024-05-08T06:06:59.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:59 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:59.443+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.388381+0000 front 2024-05-08T06:06:07.388420+0000 (oldest deadline 2024-05-08T06:06:33.188888+0000) 2024-05-08T06:06:59.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:59 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:59.443+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:17.690655+0000 front 2024-05-08T06:06:17.690772+0000 (oldest deadline 2024-05-08T06:06:42.390563+0000) 2024-05-08T06:06:59.933 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:06:59 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10[58938]: 2024-05-08T06:06:59.443+0000 7f307828a700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:25.292034+0000 front 2024-05-08T06:06:25.291890+0000 (oldest deadline 2024-05-08T06:06:47.591718+0000) 2024-05-08T06:07:00.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:59 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:59.743+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:07:00.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:59 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:59.743+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:07:00.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:59 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:59.743+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:07:00.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:59 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:59.743+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:07:00.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:59 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:59.743+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:07:00.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:59 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:59.743+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:07:00.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:06:59 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:06:59.743+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-08T06:06:33.121193+0000 front 2024-05-08T06:06:33.121124+0000 (oldest deadline 2024-05-08T06:06:59.020982+0000) 2024-05-08T06:07:01.065 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:07:00 smithi150 podman[69595]: 2024-05-08 06:07:00.751358538 +0000 UTC m=+5.519945969 container died b0b81ec35de9654b133ec555250a8f9e77d3e7713dcb46d87c4f0a5b2f4af914 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-08T06:07:01.066 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:07:01 smithi150 podman[69595]: 2024-05-08 06:07:01.066452509 +0000 UTC m=+5.835039948 container cleanup b0b81ec35de9654b133ec555250a8f9e77d3e7713dcb46d87c4f0a5b2f4af914 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:07:01.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:00 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:00.791+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:07:01.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:00 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:00.791+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:07:01.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:00 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:00.791+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:07:01.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:00 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:00.791+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:07:01.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:00 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:00.791+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:07:01.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:00 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:00.791+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:07:01.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:00 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:00.791+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-08T06:06:33.121193+0000 front 2024-05-08T06:06:33.121124+0000 (oldest deadline 2024-05-08T06:06:59.020982+0000) 2024-05-08T06:07:01.433 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:07:01 smithi150 bash[69595]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10 2024-05-08T06:07:01.433 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:07:01 smithi150 podman[69612]: 2024-05-08 06:07:01.267673148 +0000 UTC m=+0.511200743 container remove b0b81ec35de9654b133ec555250a8f9e77d3e7713dcb46d87c4f0a5b2f4af914 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-08T06:07:01.979 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:07:01 smithi150 podman[69704]: 2024-05-08 06:07:01.587389021 +0000 UTC m=+0.018063804 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:07:01.979 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:07:01 smithi150 podman[69704]: 2024-05-08 06:07:01.684880996 +0000 UTC m=+0.115555778 container create d607eed4711006ad061f7b1ca9ea813d21ba5dd37b93feacea6aaa1757c2d2ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:07:01.979 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:07:01 smithi150 podman[69704]: 2024-05-08 06:07:01.904288745 +0000 UTC m=+0.334963527 container init d607eed4711006ad061f7b1ca9ea813d21ba5dd37b93feacea6aaa1757c2d2ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-08T06:07:01.979 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:07:01 smithi150 podman[69704]: 2024-05-08 06:07:01.907343776 +0000 UTC m=+0.338018566 container start d607eed4711006ad061f7b1ca9ea813d21ba5dd37b93feacea6aaa1757c2d2ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-08T06:07:01.979 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:07:01 smithi150 podman[69704]: 2024-05-08 06:07:01.979048208 +0000 UTC m=+0.409723013 container attach d607eed4711006ad061f7b1ca9ea813d21ba5dd37b93feacea6aaa1757c2d2ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-08T06:07:02.316 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:01 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:01.819+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:07:02.316 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:01 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:01.819+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:07:02.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:01 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:01.819+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:07:02.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:01 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:01.819+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:07:02.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:01 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:01.819+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:07:02.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:01 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:01.819+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:07:02.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:01 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:01.819+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-08T06:06:33.121193+0000 front 2024-05-08T06:06:33.121124+0000 (oldest deadline 2024-05-08T06:06:59.020982+0000) 2024-05-08T06:07:02.433 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 08 06:07:02 smithi150 podman[69704]: 2024-05-08 06:07:02.201035955 +0000 UTC m=+0.631710738 container died d607eed4711006ad061f7b1ca9ea813d21ba5dd37b93feacea6aaa1757c2d2ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-08T06:07:02.812 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.10.service' 2024-05-08T06:07:02.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:02.851 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-08T06:07:02.851 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-08T06:07:02.851 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.11 2024-05-08T06:07:03.144 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:02 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:02.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:07:03.144 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:02 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:02.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:07:03.144 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:02 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:02.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:07:03.144 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:02 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:02.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:07:03.144 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:02 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:02.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:07:03.144 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:02 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:02.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:07:03.144 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:02 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:02.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-08T06:06:33.121193+0000 front 2024-05-08T06:06:33.121124+0000 (oldest deadline 2024-05-08T06:06:59.020982+0000) 2024-05-08T06:07:03.145 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:02 smithi162 systemd[1]: Stopping Ceph osd.11 for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:03.145 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:03 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:03.142+0000 7f4e91fec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:03.145 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:03 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:03.142+0000 7f4e91fec700 -1 osd.11 87 *** Got signal Terminated *** 2024-05-08T06:07:03.145 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:03 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:03.142+0000 7f4e91fec700 -1 osd.11 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:07:04.316 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:03 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:03.910+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:07:04.316 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:03 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:03.910+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:07:04.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:03 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:03.910+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:07:04.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:03 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:03.910+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:07:04.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:03 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:03.910+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:07:04.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:03 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:03.910+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:07:04.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:03 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:03.910+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-08T06:06:33.121193+0000 front 2024-05-08T06:06:33.121124+0000 (oldest deadline 2024-05-08T06:06:59.020982+0000) 2024-05-08T06:07:04.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:03 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:03.910+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6820 osd.8 since back 2024-05-08T06:06:39.021481+0000 front 2024-05-08T06:06:39.021547+0000 (oldest deadline 2024-05-08T06:07:03.721614+0000) 2024-05-08T06:07:05.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:04 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:04.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:07:05.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:04 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:04.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:07:05.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:04 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:04.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:07:05.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:04 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:04.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:07:05.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:04 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:04.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:07:05.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:04 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:04.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:07:05.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:04 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:04.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-08T06:06:33.121193+0000 front 2024-05-08T06:06:33.121124+0000 (oldest deadline 2024-05-08T06:06:59.020982+0000) 2024-05-08T06:07:05.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:04 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:04.864+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6820 osd.8 since back 2024-05-08T06:06:39.021481+0000 front 2024-05-08T06:06:39.021547+0000 (oldest deadline 2024-05-08T06:07:03.721614+0000) 2024-05-08T06:07:06.316 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:05 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:05.826+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:07:06.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:05 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:05.826+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:07:06.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:05 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:05.826+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:07:06.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:05 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:05.826+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:07:06.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:05 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:05.826+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:07:06.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:05 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:05.826+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:07:06.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:05 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:05.826+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-08T06:06:33.121193+0000 front 2024-05-08T06:06:33.121124+0000 (oldest deadline 2024-05-08T06:06:59.020982+0000) 2024-05-08T06:07:06.317 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:05 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:05.826+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6820 osd.8 since back 2024-05-08T06:06:39.021481+0000 front 2024-05-08T06:06:39.021547+0000 (oldest deadline 2024-05-08T06:07:03.721614+0000) 2024-05-08T06:07:07.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:06 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:06.784+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:07:07.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:06 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:06.784+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:07:07.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:06 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:06.784+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:07:07.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:06 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:06.784+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:07:07.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:06 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:06.784+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:07:07.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:06 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:06.784+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:07:07.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:06 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:06.784+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-08T06:06:33.121193+0000 front 2024-05-08T06:06:33.121124+0000 (oldest deadline 2024-05-08T06:06:59.020982+0000) 2024-05-08T06:07:07.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:06 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:06.784+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6820 osd.8 since back 2024-05-08T06:06:39.021481+0000 front 2024-05-08T06:06:39.021547+0000 (oldest deadline 2024-05-08T06:07:03.721614+0000) 2024-05-08T06:07:08.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:07.737+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-05-08T06:05:32.913064+0000 front 2024-05-08T06:05:32.912819+0000 (oldest deadline 2024-05-08T06:05:57.612901+0000) 2024-05-08T06:07:08.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:07.737+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-08T06:05:42.914113+0000 front 2024-05-08T06:05:42.914188+0000 (oldest deadline 2024-05-08T06:06:08.213972+0000) 2024-05-08T06:07:08.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:07.737+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-05-08T06:05:52.914936+0000 front 2024-05-08T06:05:52.915157+0000 (oldest deadline 2024-05-08T06:06:15.215017+0000) 2024-05-08T06:07:08.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:07.737+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-08T06:06:07.818377+0000 front 2024-05-08T06:06:07.818390+0000 (oldest deadline 2024-05-08T06:06:32.518400+0000) 2024-05-08T06:07:08.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:07.737+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-05-08T06:06:18.419189+0000 front 2024-05-08T06:06:18.419306+0000 (oldest deadline 2024-05-08T06:06:43.119318+0000) 2024-05-08T06:07:08.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:07.737+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-05-08T06:06:23.119934+0000 front 2024-05-08T06:06:23.119859+0000 (oldest deadline 2024-05-08T06:06:49.019741+0000) 2024-05-08T06:07:08.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:07.737+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-08T06:06:33.121193+0000 front 2024-05-08T06:06:33.121124+0000 (oldest deadline 2024-05-08T06:06:59.020982+0000) 2024-05-08T06:07:08.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:07 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11[60116]: 2024-05-08T06:07:07.737+0000 7f4e8e5e0700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.162:6820 osd.8 since back 2024-05-08T06:06:39.021481+0000 front 2024-05-08T06:06:39.021547+0000 (oldest deadline 2024-05-08T06:07:03.721614+0000) 2024-05-08T06:07:08.935 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:08 smithi162 podman[71496]: 2024-05-08 06:07:08.684694602 +0000 UTC m=+5.630500436 container died 0e0f0951de729389e0d208f2531e3641c3f70c6f184549af344ef1adc0efef66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-08T06:07:08.935 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:08 smithi162 podman[71496]: 2024-05-08 06:07:08.935890982 +0000 UTC m=+5.881696815 container cleanup 0e0f0951de729389e0d208f2531e3641c3f70c6f184549af344ef1adc0efef66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True) 2024-05-08T06:07:09.203 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:08 smithi162 bash[71496]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11 2024-05-08T06:07:09.565 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:09 smithi162 podman[71515]: 2024-05-08 06:07:09.204059477 +0000 UTC m=+0.514293654 container remove 0e0f0951de729389e0d208f2531e3641c3f70c6f184549af344ef1adc0efef66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:07:09.815 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:09 smithi162 podman[71604]: 2024-05-08 06:07:09.468014153 +0000 UTC m=+0.017800792 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:07:09.815 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:09 smithi162 podman[71604]: 2024-05-08 06:07:09.571979703 +0000 UTC m=+0.121766326 container create a11e486a592fac03c260d15ca02dccc4f8bcb77465976e74e4b433eba3f04c23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11-deactivate, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-05-08T06:07:09.815 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:09 smithi162 podman[71604]: 2024-05-08 06:07:09.81624312 +0000 UTC m=+0.366029757 container init a11e486a592fac03c260d15ca02dccc4f8bcb77465976e74e4b433eba3f04c23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, ceph=True) 2024-05-08T06:07:10.066 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:09 smithi162 podman[71604]: 2024-05-08 06:07:09.819438299 +0000 UTC m=+0.369224924 container start a11e486a592fac03c260d15ca02dccc4f8bcb77465976e74e4b433eba3f04c23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2) 2024-05-08T06:07:10.067 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:09 smithi162 podman[71604]: 2024-05-08 06:07:09.89950297 +0000 UTC m=+0.449289613 container attach a11e486a592fac03c260d15ca02dccc4f8bcb77465976e74e4b433eba3f04c23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:07:10.393 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:10 smithi162 podman[71604]: 2024-05-08 06:07:10.113874087 +0000 UTC m=+0.663660720 container died a11e486a592fac03c260d15ca02dccc4f8bcb77465976e74e4b433eba3f04c23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11-deactivate, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:07:10.620 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.11.service' 2024-05-08T06:07:10.644 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:10 smithi162 podman[71631]: 2024-05-08 06:07:10.602586861 +0000 UTC m=+0.479235452 container remove a11e486a592fac03c260d15ca02dccc4f8bcb77465976e74e4b433eba3f04c23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-05-08T06:07:10.644 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:10 smithi162 systemd[1]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.11.service: Deactivated successfully. 2024-05-08T06:07:10.644 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:10 smithi162 systemd[1]: Stopped Ceph osd.11 for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T06:07:10.645 INFO:journalctl@ceph.osd.11.smithi162.stdout:May 08 06:07:10 smithi162 systemd[1]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@osd.11.service: Consumed 46.547s CPU time. 2024-05-08T06:07:10.657 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:10.657 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-08T06:07:10.657 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-08T06:07:10.657 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.a 2024-05-08T06:07:11.065 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 06:07:10 smithi112 systemd[1]: Stopping Ceph mds.a for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:11.565 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 06:07:11 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-a[69856]: 2024-05-08T06:07:11.209+0000 7fa70f200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:11.566 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 06:07:11 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-a[69856]: 2024-05-08T06:07:11.209+0000 7fa70f200700 -1 mds.a *** got signal Terminated *** 2024-05-08T06:07:15.491 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 06:07:15 smithi112 podman[120655]: 2024-05-08 06:07:15.222964183 +0000 UTC m=+4.138598694 container died b7a24dd92d926e507edb14783a33c4ddf6f15db875377ad0659cc98f04929c70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , ceph=True) 2024-05-08T06:07:15.743 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 06:07:15 smithi112 podman[120655]: 2024-05-08 06:07:15.736493933 +0000 UTC m=+4.652128467 container cleanup b7a24dd92d926e507edb14783a33c4ddf6f15db875377ad0659cc98f04929c70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:07:15.743 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 06:07:15 smithi112 bash[120655]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-a 2024-05-08T06:07:16.315 INFO:journalctl@ceph.mds.a.smithi112.stdout:May 08 06:07:15 smithi112 podman[120669]: 2024-05-08 06:07:15.91314895 +0000 UTC m=+0.674770428 container remove b7a24dd92d926e507edb14783a33c4ddf6f15db875377ad0659cc98f04929c70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-a, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-08T06:07:16.410 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.a.service' 2024-05-08T06:07:16.519 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:16.519 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-08T06:07:16.519 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-08T06:07:16.519 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.d 2024-05-08T06:07:17.066 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 06:07:16 smithi112 systemd[1]: Stopping Ceph mds.d for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:17.565 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 06:07:17 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-d[68674]: 2024-05-08T06:07:17.088+0000 7fec56000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:17.565 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 06:07:17 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-d[68674]: 2024-05-08T06:07:17.088+0000 7fec56000700 -1 mds.d *** got signal Terminated *** 2024-05-08T06:07:20.970 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 06:07:20 smithi112 podman[120776]: 2024-05-08 06:07:20.68867075 +0000 UTC m=+3.733792323 container died cb81d29a3c70fe71970fa25cfeff0d7060ce0d5e81d73335d687234a5f8d9d99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS) 2024-05-08T06:07:21.315 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 06:07:21 smithi112 podman[120776]: 2024-05-08 06:07:21.199694354 +0000 UTC m=+4.244815874 container cleanup cb81d29a3c70fe71970fa25cfeff0d7060ce0d5e81d73335d687234a5f8d9d99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-08T06:07:21.315 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 06:07:21 smithi112 bash[120776]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-d 2024-05-08T06:07:21.815 INFO:journalctl@ceph.mds.d.smithi112.stdout:May 08 06:07:21 smithi112 podman[120788]: 2024-05-08 06:07:21.525920204 +0000 UTC m=+0.820182089 container remove cb81d29a3c70fe71970fa25cfeff0d7060ce0d5e81d73335d687234a5f8d9d99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-08T06:07:22.081 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.d.service' 2024-05-08T06:07:22.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:22.185 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-08T06:07:22.185 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-08T06:07:22.185 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.g 2024-05-08T06:07:22.784 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 08 06:07:22 smithi112 systemd[1]: Stopping Ceph mds.g for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:23.066 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 08 06:07:22 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-g[69455]: 2024-05-08T06:07:22.783+0000 7f4634400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:23.066 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 08 06:07:22 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-g[69455]: 2024-05-08T06:07:22.783+0000 7f4634400700 -1 mds.g *** got signal Terminated *** 2024-05-08T06:07:27.065 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 08 06:07:26 smithi112 podman[120897]: 2024-05-08 06:07:26.795510083 +0000 UTC m=+4.189563175 container died f5f0933b3f22d0b6350fa5001b8d3fa50efa351f2482d84157cf6b805e50749c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-g, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-05-08T06:07:27.371 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 08 06:07:27 smithi112 podman[120897]: 2024-05-08 06:07:27.178123766 +0000 UTC m=+4.572176859 container cleanup f5f0933b3f22d0b6350fa5001b8d3fa50efa351f2482d84157cf6b805e50749c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True) 2024-05-08T06:07:27.371 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 08 06:07:27 smithi112 bash[120897]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-g 2024-05-08T06:07:27.779 INFO:journalctl@ceph.mds.g.smithi112.stdout:May 08 06:07:27 smithi112 podman[120910]: 2024-05-08 06:07:27.371240815 +0000 UTC m=+0.558480684 container remove f5f0933b3f22d0b6350fa5001b8d3fa50efa351f2482d84157cf6b805e50749c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-g, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-08T06:07:27.815 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.g.service' 2024-05-08T06:07:27.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:27.915 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-08T06:07:27.915 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-08T06:07:27.915 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.j 2024-05-08T06:07:28.496 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 08 06:07:28 smithi112 systemd[1]: Stopping Ceph mds.j for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:28.816 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 08 06:07:28 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-j[68256]: 2024-05-08T06:07:28.497+0000 7f5f5dc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:28.816 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 08 06:07:28 smithi112 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-j[68256]: 2024-05-08T06:07:28.497+0000 7f5f5dc00700 -1 mds.j *** got signal Terminated *** 2024-05-08T06:07:32.370 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 08 06:07:32 smithi112 podman[121015]: 2024-05-08 06:07:32.095371745 +0000 UTC m=+3.736281049 container died 340658b200f7231df1bc24a3ad398a1d167c68dd82f0b8f38cb9dcfc79ff2412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:07:32.657 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 08 06:07:32 smithi112 podman[121015]: 2024-05-08 06:07:32.473454835 +0000 UTC m=+4.114364135 container cleanup 340658b200f7231df1bc24a3ad398a1d167c68dd82f0b8f38cb9dcfc79ff2412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-j, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-05-08T06:07:32.658 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 08 06:07:32 smithi112 bash[121015]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-j 2024-05-08T06:07:33.065 INFO:journalctl@ceph.mds.j.smithi112.stdout:May 08 06:07:32 smithi112 podman[121029]: 2024-05-08 06:07:32.657542478 +0000 UTC m=+0.542467631 container remove 340658b200f7231df1bc24a3ad398a1d167c68dd82f0b8f38cb9dcfc79ff2412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-j, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-08T06:07:33.211 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.j.service' 2024-05-08T06:07:33.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:33.316 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-08T06:07:33.316 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-08T06:07:33.316 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.b 2024-05-08T06:07:33.608 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 06:07:33 smithi150 systemd[1]: Stopping Ceph mds.b for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:33.933 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 06:07:33 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-b[64165]: 2024-05-08T06:07:33.607+0000 7f0574fdb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:33.933 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 06:07:33 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-b[64165]: 2024-05-08T06:07:33.607+0000 7f0574fdb700 -1 mds.b *** got signal Terminated *** 2024-05-08T06:07:36.183 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 06:07:35 smithi150 podman[69808]: 2024-05-08 06:07:35.830406307 +0000 UTC m=+2.321089648 container died 7dc66aadba577ce6bf384b27537676732d371887822376268dc4ef039052680e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:07:36.458 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 06:07:36 smithi150 podman[69808]: 2024-05-08 06:07:36.257698069 +0000 UTC m=+2.748381407 container cleanup 7dc66aadba577ce6bf384b27537676732d371887822376268dc4ef039052680e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=) 2024-05-08T06:07:36.458 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 08 06:07:36 smithi150 bash[69808]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-b 2024-05-08T06:07:36.677 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.b.service' 2024-05-08T06:07:36.712 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:36.712 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-08T06:07:36.712 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-08T06:07:36.712 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.e 2024-05-08T06:07:37.036 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 06:07:36 smithi150 systemd[1]: Stopping Ceph mds.e for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:37.433 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 06:07:37 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-e[63542]: 2024-05-08T06:07:37.036+0000 7fe27fdf7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:37.433 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 06:07:37 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-e[63542]: 2024-05-08T06:07:37.036+0000 7fe27fdf7700 -1 mds.e *** got signal Terminated *** 2024-05-08T06:07:39.728 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 06:07:39 smithi150 podman[69942]: 2024-05-08 06:07:39.474559854 +0000 UTC m=+2.540758057 container died 92a9e02d224d2bdf587e48bd5ecd3b19384c26ae0506e4e5b38aac74ba199bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:07:40.003 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 06:07:39 smithi150 podman[69942]: 2024-05-08 06:07:39.819517801 +0000 UTC m=+2.885716003 container cleanup 92a9e02d224d2bdf587e48bd5ecd3b19384c26ae0506e4e5b38aac74ba199bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-e, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-08T06:07:40.003 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 06:07:39 smithi150 bash[69942]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-e 2024-05-08T06:07:40.243 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.e.service' 2024-05-08T06:07:40.263 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 06:07:40 smithi150 podman[69957]: 2024-05-08 06:07:40.004142348 +0000 UTC m=+0.525869279 container remove 92a9e02d224d2bdf587e48bd5ecd3b19384c26ae0506e4e5b38aac74ba199bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:07:40.263 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 06:07:40 smithi150 systemd[1]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.e.service: Deactivated successfully. 2024-05-08T06:07:40.263 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 06:07:40 smithi150 systemd[1]: Stopped Ceph mds.e for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T06:07:40.263 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 08 06:07:40 smithi150 systemd[1]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.e.service: Consumed 1.059s CPU time. 2024-05-08T06:07:40.275 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:40.275 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-08T06:07:40.275 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-08T06:07:40.275 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.h 2024-05-08T06:07:40.606 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 06:07:40 smithi150 systemd[1]: Stopping Ceph mds.h for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:40.933 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 06:07:40 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-h[63857]: 2024-05-08T06:07:40.606+0000 7fa355be7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:40.933 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 06:07:40 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-h[63857]: 2024-05-08T06:07:40.606+0000 7fa355be7700 -1 mds.h *** got signal Terminated *** 2024-05-08T06:07:44.173 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 06:07:43 smithi150 podman[70063]: 2024-05-08 06:07:43.836787357 +0000 UTC m=+3.335757155 container died cb34aefd585ee76e60737edd0acaa21f8ade4435982703008f9b2e2292c349c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-h, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-08T06:07:44.433 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 06:07:44 smithi150 podman[70063]: 2024-05-08 06:07:44.273446157 +0000 UTC m=+3.772415953 container cleanup cb34aefd585ee76e60737edd0acaa21f8ade4435982703008f9b2e2292c349c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-h, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:07:44.433 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 06:07:44 smithi150 bash[70063]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-h 2024-05-08T06:07:44.694 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.h.service' 2024-05-08T06:07:44.713 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 06:07:44 smithi150 podman[70078]: 2024-05-08 06:07:44.449622538 +0000 UTC m=+0.615809693 container remove cb34aefd585ee76e60737edd0acaa21f8ade4435982703008f9b2e2292c349c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-h, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-08T06:07:44.714 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 06:07:44 smithi150 systemd[1]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.h.service: Deactivated successfully. 2024-05-08T06:07:44.714 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 06:07:44 smithi150 systemd[1]: Stopped Ceph mds.h for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T06:07:44.714 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 08 06:07:44 smithi150 systemd[1]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.h.service: Consumed 3.111s CPU time. 2024-05-08T06:07:44.724 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:44.724 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-08T06:07:44.725 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-08T06:07:44.725 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.k 2024-05-08T06:07:45.026 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 08 06:07:44 smithi150 systemd[1]: Stopping Ceph mds.k for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:45.304 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 08 06:07:45 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-k[63224]: 2024-05-08T06:07:45.026+0000 7f89df927700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:45.304 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 08 06:07:45 smithi150 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-k[63224]: 2024-05-08T06:07:45.026+0000 7f89df927700 -1 mds.k *** got signal Terminated *** 2024-05-08T06:07:47.427 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 08 06:07:46 smithi150 podman[70183]: 2024-05-08 06:07:46.950161465 +0000 UTC m=+2.013982215 container died 16086b0e30d0de7f66a461aec29fdcb715537c7374f3200b9e2522db20f401c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-k, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-08T06:07:47.683 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 08 06:07:47 smithi150 podman[70183]: 2024-05-08 06:07:47.510777921 +0000 UTC m=+2.574598674 container cleanup 16086b0e30d0de7f66a461aec29fdcb715537c7374f3200b9e2522db20f401c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-k, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-08T06:07:47.683 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 08 06:07:47 smithi150 bash[70183]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-k 2024-05-08T06:07:47.889 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.k.service' 2024-05-08T06:07:47.919 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:47.920 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-08T06:07:47.920 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-08T06:07:47.920 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.c 2024-05-08T06:07:48.283 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 06:07:47 smithi162 systemd[1]: Stopping Ceph mds.c for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:48.566 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 06:07:48 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-c[65272]: 2024-05-08T06:07:48.290+0000 7f8bbc25a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:48.566 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 06:07:48 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-c[65272]: 2024-05-08T06:07:48.290+0000 7f8bbc25a700 -1 mds.c *** got signal Terminated *** 2024-05-08T06:07:52.115 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 06:07:51 smithi162 podman[71711]: 2024-05-08 06:07:51.833501891 +0000 UTC m=+3.709249008 container died e053083a8d5fc3a8259501c3a1da76e5543387563693f165f48565de0170e051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-08T06:07:52.390 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 06:07:52 smithi162 podman[71711]: 2024-05-08 06:07:52.20689306 +0000 UTC m=+4.082640180 container cleanup e053083a8d5fc3a8259501c3a1da76e5543387563693f165f48565de0170e051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-08T06:07:52.390 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 06:07:52 smithi162 bash[71711]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-c 2024-05-08T06:07:52.640 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.c.service' 2024-05-08T06:07:52.661 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 06:07:52 smithi162 podman[71735]: 2024-05-08 06:07:52.39125625 +0000 UTC m=+0.553456286 container remove e053083a8d5fc3a8259501c3a1da76e5543387563693f165f48565de0170e051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:07:52.662 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 06:07:52 smithi162 systemd[1]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.c.service: Deactivated successfully. 2024-05-08T06:07:52.662 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 06:07:52 smithi162 systemd[1]: Stopped Ceph mds.c for 7a07ff60-0cfd-11ef-bc97-c7b262605968. 2024-05-08T06:07:52.662 INFO:journalctl@ceph.mds.c.smithi162.stdout:May 08 06:07:52 smithi162 systemd[1]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.c.service: Consumed 7.461s CPU time. 2024-05-08T06:07:52.675 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:52.676 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-08T06:07:52.676 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-08T06:07:52.676 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.f 2024-05-08T06:07:52.985 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 06:07:52 smithi162 systemd[1]: Stopping Ceph mds.f for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:53.316 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 06:07:52 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-f[64961]: 2024-05-08T06:07:52.984+0000 7f07b47a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:53.316 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 06:07:52 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-f[64961]: 2024-05-08T06:07:52.984+0000 7f07b47a2700 -1 mds.f *** got signal Terminated *** 2024-05-08T06:07:56.210 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 06:07:55 smithi162 podman[71842]: 2024-05-08 06:07:55.824331524 +0000 UTC m=+2.931790492 container died f14a5f6fd940d1762bb23647832997d389bc12de26df02a96918d1c5a847da24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux ) 2024-05-08T06:07:56.566 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 06:07:56 smithi162 podman[71842]: 2024-05-08 06:07:56.287197258 +0000 UTC m=+3.394656223 container cleanup f14a5f6fd940d1762bb23647832997d389bc12de26df02a96918d1c5a847da24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-f, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux ) 2024-05-08T06:07:56.566 INFO:journalctl@ceph.mds.f.smithi162.stdout:May 08 06:07:56 smithi162 bash[71842]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-f 2024-05-08T06:07:56.727 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.f.service' 2024-05-08T06:07:56.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:56.760 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-08T06:07:56.760 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-08T06:07:56.760 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.i 2024-05-08T06:07:57.066 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 06:07:56 smithi162 systemd[1]: Stopping Ceph mds.i for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:57.566 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 06:07:57 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-i[65579]: 2024-05-08T06:07:57.098+0000 7f12e993a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:57.566 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 06:07:57 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-i[65579]: 2024-05-08T06:07:57.098+0000 7f12e993a700 -1 mds.i *** got signal Terminated *** 2024-05-08T06:08:00.124 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 06:07:59 smithi162 podman[71964]: 2024-05-08 06:07:59.834142239 +0000 UTC m=+2.858336029 container died 2f5505b532fa10eabf37d1d4bdf6f4276c3bf3a3432e557a34fbd27a998e458a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-08T06:08:00.441 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 06:08:00 smithi162 podman[71964]: 2024-05-08 06:08:00.215832758 +0000 UTC m=+3.240026556 container cleanup 2f5505b532fa10eabf37d1d4bdf6f4276c3bf3a3432e557a34fbd27a998e458a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-i, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-08T06:08:00.441 INFO:journalctl@ceph.mds.i.smithi162.stdout:May 08 06:08:00 smithi162 bash[71964]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-i 2024-05-08T06:08:00.569 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.i.service' 2024-05-08T06:08:00.600 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:08:00.600 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-08T06:08:00.600 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-08T06:08:00.600 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.l 2024-05-08T06:08:00.816 INFO:journalctl@ceph.mds.l.smithi162.stdout:May 08 06:08:00 smithi162 systemd[1]: Stopping Ceph mds.l for 7a07ff60-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:08:01.316 INFO:journalctl@ceph.mds.l.smithi162.stdout:May 08 06:08:00 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-l[64639]: 2024-05-08T06:08:00.919+0000 7f1c5838b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:08:01.316 INFO:journalctl@ceph.mds.l.smithi162.stdout:May 08 06:08:00 smithi162 ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-l[64639]: 2024-05-08T06:08:00.919+0000 7f1c5838b700 -1 mds.l *** got signal Terminated *** 2024-05-08T06:08:04.422 INFO:journalctl@ceph.mds.l.smithi162.stdout:May 08 06:08:04 smithi162 podman[72085]: 2024-05-08 06:08:04.157844991 +0000 UTC m=+3.345129316 container died 0ecda3dd22a53e2730167bceb89be4d1e063c4da35e21af50ee5d953c602d602 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:08:04.753 INFO:journalctl@ceph.mds.l.smithi162.stdout:May 08 06:08:04 smithi162 podman[72085]: 2024-05-08 06:08:04.519617483 +0000 UTC m=+3.706901816 container cleanup 0ecda3dd22a53e2730167bceb89be4d1e063c4da35e21af50ee5d953c602d602 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-l, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:08:04.753 INFO:journalctl@ceph.mds.l.smithi162.stdout:May 08 06:08:04 smithi162 bash[72085]: ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968-mds-l 2024-05-08T06:08:04.887 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7a07ff60-0cfd-11ef-bc97-c7b262605968@mds.l.service' 2024-05-08T06:08:04.916 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:08:04.917 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-08T06:08:04.917 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T06:08:05.235 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T06:08:10.278 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T06:08:10.466 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T06:08:12.880 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T06:08:13.068 INFO:teuthology.orchestra.run.smithi162.stdout:Deleting cluster with fsid: 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T06:08:15.367 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:08:15.453 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:08:15.485 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:08:15.510 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-08T06:08:15.512 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697245/remote/smithi112/crash 2024-05-08T06:08:15.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/crash -- . 2024-05-08T06:08:15.628 INFO:teuthology.orchestra.run.smithi112.stderr:tar: /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T06:08:15.628 INFO:teuthology.orchestra.run.smithi112.stderr:tar: Error is not recoverable: exiting now 2024-05-08T06:08:15.633 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697245/remote/smithi150/crash 2024-05-08T06:08:15.634 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/crash -- . 2024-05-08T06:08:15.725 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T06:08:15.725 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2024-05-08T06:08:15.727 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697245/remote/smithi162/crash 2024-05-08T06:08:15.728 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/crash -- . 2024-05-08T06:08:15.762 INFO:teuthology.orchestra.run.smithi162.stderr:tar: /var/lib/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T06:08:15.762 INFO:teuthology.orchestra.run.smithi162.stderr:tar: Error is not recoverable: exiting now 2024-05-08T06:08:15.763 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-08T06:08:15.764 DEBUG:teuthology.orchestra.run.smithi112:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-08T06:08:15.872 INFO:tasks.cephadm:Compressing logs... 2024-05-08T06:08:15.872 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T06:08:15.876 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T06:08:15.878 DEBUG:teuthology.orchestra.run.smithi162:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T06:08:15.903 INFO:teuthology.orchestra.run.smithi162.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-05-08T06:08:15.903 INFO:teuthology.orchestra.run.smithi162.stderr: /var/log/ceph/cephadm.log 2024-05-08T06:08:15.903 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.log 2024-05-08T06:08:15.904 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-08T06:08:15.904 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-08T06:08:15.904 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-08T06:08:15.904 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose --/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.f.log: /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-08T06:08:15.904 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-08T06:08:15.904 INFO:teuthology.orchestra.run.smithi162.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-08T06:08:15.904 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-08T06:08:15.909 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-08T06:08:15.909 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-08T06:08:15.909 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mgr.z.log: 87.0% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-08T06:08:15.909 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-08T06:08:15.909 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.i.log: 90.5% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-08T06:08:15.912 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-08T06:08:15.912 INFO:teuthology.orchestra.run.smithi150.stderr:: No such file or directory 2024-05-08T06:08:15.912 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.log 2024-05-08T06:08:15.912 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-08T06:08:15.912 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-08T06:08:15.912 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-08T06:08:15.912 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.7.log: --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-08T06:08:15.912 INFO:teuthology.orchestra.run.smithi150.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-08T06:08:15.913 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-08T06:08:15.913 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-08T06:08:15.913 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-08T06:08:15.913 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-08T06:08:15.913 INFO:teuthology.orchestra.run.smithi150.stderr: 87.0% 90.5% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-08T06:08:15.913 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-08T06:08:15.913 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-08T06:08:15.913 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-08T06:08:15.913 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-08T06:08:15.913 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-08T06:08:15.916 INFO:teuthology.orchestra.run.smithi162.stderr: 93.0% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-08T06:08:15.916 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-08T06:08:15.916 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-08T06:08:15.916 INFO:teuthology.orchestra.run.smithi162.stderr: 82.3% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-08T06:08:15.916 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-08T06:08:15.918 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.e.log: 92.9% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-08T06:08:15.918 INFO:teuthology.orchestra.run.smithi150.stderr: 94.6% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-08T06:08:15.918 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-08T06:08:15.919 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-08T06:08:15.922 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-08T06:08:15.929 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-08T06:08:15.930 INFO:teuthology.orchestra.run.smithi112.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T06:08:15.930 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-08T06:08:15.931 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.log 2024-05-08T06:08:15.932 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-08T06:08:15.933 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-08T06:08:15.934 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-08T06:08:15.935 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-08T06:08:15.936 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.g.log: 87.0% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-08T06:08:15.936 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-08T06:08:15.937 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-08T06:08:15.937 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.3.log: 90.3% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-08T06:08:15.938 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-08T06:08:15.939 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-08T06:08:15.941 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-08T06:08:15.942 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mon.a.log: 94.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-08T06:08:15.942 INFO:teuthology.orchestra.run.smithi112.stderr: 94.6% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-08T06:08:15.943 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.6.log: 94.6% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-08T06:08:15.944 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-volume.log: /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.l.log: 94.6% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-08T06:08:15.945 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-08T06:08:15.946 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-08T06:08:15.947 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-08T06:08:15.948 INFO:teuthology.orchestra.run.smithi112.stderr: 82.7% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-08T06:08:15.949 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-08T06:08:15.951 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-volume.log: /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-08T06:08:15.955 INFO:teuthology.orchestra.run.smithi162.stderr: 91.0% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-08T06:08:15.964 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-08T06:08:15.965 INFO:teuthology.orchestra.run.smithi112.stderr: 90.9% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-08T06:08:16.239 INFO:teuthology.orchestra.run.smithi150.stderr: 91.7% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-08T06:08:16.255 INFO:teuthology.orchestra.run.smithi112.stderr: 90.1% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-08T06:08:16.409 INFO:teuthology.orchestra.run.smithi150.stderr: 95.0% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-08T06:08:16.410 INFO:teuthology.orchestra.run.smithi162.stderr: 91.7% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-08T06:08:16.625 INFO:teuthology.orchestra.run.smithi162.stderr: 95.2% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-08T06:08:17.239 INFO:teuthology.orchestra.run.smithi112.stderr: 90.3% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-08T06:08:17.834 INFO:teuthology.orchestra.run.smithi150.stderr: 94.1% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-08T06:08:17.917 INFO:teuthology.orchestra.run.smithi112.stderr: 93.6% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-08T06:08:18.128 INFO:teuthology.orchestra.run.smithi162.stderr: 96.1% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-08T06:08:19.365 INFO:teuthology.orchestra.run.smithi150.stderr: 94.0% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-08T06:08:19.422 INFO:teuthology.orchestra.run.smithi162.stderr: 94.1% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-08T06:08:19.472 INFO:teuthology.orchestra.run.smithi162.stderr: 94.3% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-08T06:08:19.980 INFO:teuthology.orchestra.run.smithi112.stderr: 93.6% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-08T06:08:20.595 INFO:teuthology.orchestra.run.smithi162.stderr: 94.1% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-08T06:08:20.649 INFO:teuthology.orchestra.run.smithi112.stderr: 94.2% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-08T06:08:20.716 INFO:teuthology.orchestra.run.smithi162.stderr: 94.0% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-08T06:08:21.194 INFO:teuthology.orchestra.run.smithi150.stderr: 94.0% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-08T06:08:21.370 INFO:teuthology.orchestra.run.smithi112.stderr: 93.9% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-08T06:08:22.221 INFO:teuthology.orchestra.run.smithi150.stderr: 93.7% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-08T06:08:22.339 INFO:teuthology.orchestra.run.smithi162.stderr: 93.9% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-08T06:08:22.341 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-05-08T06:08:22.341 INFO:teuthology.orchestra.run.smithi162.stderr:real 0m6.448s 2024-05-08T06:08:22.341 INFO:teuthology.orchestra.run.smithi162.stderr:user 0m25.619s 2024-05-08T06:08:22.341 INFO:teuthology.orchestra.run.smithi162.stderr:sys 0m0.830s 2024-05-08T06:08:23.701 INFO:teuthology.orchestra.run.smithi150.stderr: 94.0% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-08T06:08:23.703 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-05-08T06:08:23.703 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m7.810s 2024-05-08T06:08:23.703 INFO:teuthology.orchestra.run.smithi150.stderr:user 0m24.740s 2024-05-08T06:08:23.703 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m0.894s 2024-05-08T06:08:38.125 INFO:teuthology.orchestra.run.smithi112.stderr: 93.5% -- replaced with /var/log/ceph/7a07ff60-0cfd-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-08T06:08:38.129 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-05-08T06:08:38.129 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m22.223s 2024-05-08T06:08:38.129 INFO:teuthology.orchestra.run.smithi112.stderr:user 0m36.779s 2024-05-08T06:08:38.129 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m3.063s 2024-05-08T06:08:38.130 INFO:tasks.cephadm:Archiving logs... 2024-05-08T06:08:38.131 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697245/remote/smithi112/log 2024-05-08T06:08:38.132 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T06:08:43.327 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697245/remote/smithi150/log 2024-05-08T06:08:43.328 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T06:08:46.613 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697245/remote/smithi162/log 2024-05-08T06:08:46.614 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T06:08:49.671 INFO:tasks.cephadm:Removing cluster... 2024-05-08T06:08:49.672 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 --force 2024-05-08T06:08:49.995 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T06:08:50.726 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 --force 2024-05-08T06:08:50.919 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T06:08:51.214 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7a07ff60-0cfd-11ef-bc97-c7b262605968 --force 2024-05-08T06:08:51.405 INFO:teuthology.orchestra.run.smithi162.stdout:Deleting cluster with fsid: 7a07ff60-0cfd-11ef-bc97-c7b262605968 2024-05-08T06:08:51.679 INFO:tasks.cephadm:Removing cephadm ... 2024-05-08T06:08:51.679 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T06:08:51.738 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T06:08:51.755 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T06:08:51.770 INFO:tasks.cephadm:Teardown complete 2024-05-08T06:08:51.771 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T06:08:51.783 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T06:08:51.783 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T06:08:51.791 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T06:08:51.797 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T06:08:51.931 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T06:08:51.942 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T06:08:51.942 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T06:08:51.949 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T06:08:51.951 DEBUG:teuthology.orchestra.run.smithi162:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T06:08:51.963 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-05-08T06:08:51.965 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-05-08T06:08:51.967 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T06:08:51.967 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-05-08T06:08:51.967 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 +265us[ +274us] +/- 58ms 2024-05-08T06:08:51.968 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 -577us[ -568us] +/- 83ms 2024-05-08T06:08:51.968 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 +205us[ +214us] +/- 81ms 2024-05-08T06:08:51.968 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:08:51.971 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T06:08:51.971 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-05-08T06:08:51.971 INFO:teuthology.orchestra.run.smithi162.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 +262us[ +269us] +/- 58ms 2024-05-08T06:08:51.971 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 -559us[ -559us] +/- 83ms 2024-05-08T06:08:51.971 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +219us[ +219us] +/- 81ms 2024-05-08T06:08:51.971 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:08:51.996 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-05-08T06:08:52.006 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T06:08:52.006 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-05-08T06:08:52.006 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv01.front.sepia.ceph.com 3 8 377 17 +258us[ +239us] +/- 58ms 2024-05-08T06:08:52.006 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 16 -525us[ -525us] +/- 83ms 2024-05-08T06:08:52.006 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +171us[ +171us] +/- 81ms 2024-05-08T06:08:52.006 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:08:52.007 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T06:08:52.017 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T06:08:52.017 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T06:08:52.049 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T06:08:52.259 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T06:08:52.324 DEBUG:teuthology.orchestra.run.smithi162:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T06:08:52.388 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T06:08:52.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:08:52.485 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T06:08:52.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:08:52.511 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T06:08:52.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:08:52.538 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T06:08:52.548 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T06:08:52.558 INFO:teuthology.task.internal:Duration was 2228.073232 seconds 2024-05-08T06:08:52.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T06:08:52.567 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T06:08:52.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T06:08:52.575 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T06:08:52.577 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T06:08:52.612 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T06:08:52.614 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T06:08:52.716 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T06:08:53.108 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T06:08:53.109 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-05-08T06:08:53.109 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T06:08:53.182 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-05-08T06:08:53.183 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T06:08:53.205 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi162.front.sepia.ceph.com 2024-05-08T06:08:53.205 DEBUG:teuthology.orchestra.run.smithi162:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T06:08:53.232 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T06:08:53.232 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T06:08:53.240 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T06:08:53.247 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T06:08:53.321 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T06:08:53.321 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T06:08:53.326 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T06:08:53.328 DEBUG:teuthology.orchestra.run.smithi162:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T06:08:54.281 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T06:08:54.293 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T06:08:54.294 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T06:08:54.370 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T06:08:54.399 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T06:08:54.423 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T06:08:54.434 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:08:54.441 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:08:54.444 DEBUG:teuthology.orchestra.run.smithi162:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:08:54.466 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-05-08T06:08:54.484 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = core 2024-05-08T06:08:54.532 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-05-08T06:08:54.576 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:08:54.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:08:54.622 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:08:54.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:08:54.637 DEBUG:teuthology.orchestra.run.smithi162:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:08:54.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:08:54.652 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T06:08:54.662 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T06:08:54.663 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697245/remote/smithi112 2024-05-08T06:08:54.663 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T06:08:54.771 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697245/remote/smithi150 2024-05-08T06:08:54.771 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T06:08:54.807 DEBUG:teuthology.misc:Transferring archived files from smithi162:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697245/remote/smithi162 2024-05-08T06:08:54.807 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T06:08:54.841 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T06:08:54.842 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T06:08:54.849 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T06:08:54.852 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T06:08:54.898 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T06:08:54.910 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T06:08:54.910 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T06:08:54.970 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T06:08:54.970 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T06:08:54.977 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T06:08:54.979 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T06:08:54.992 INFO:teuthology.orchestra.run.smithi150.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 06:08 /home/ubuntu/cephtest 2024-05-08T06:08:54.993 INFO:teuthology.orchestra.run.smithi162.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 06:08 /home/ubuntu/cephtest 2024-05-08T06:08:55.029 INFO:teuthology.orchestra.run.smithi112.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 06:08 /home/ubuntu/cephtest 2024-05-08T06:08:55.032 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T06:08:55.044 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T06:08:55.158 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/fsx}} duration: 2228.0732316970825 flavor: default owner: scheduled_vshankar@teuthology success: true 2024-05-08T06:08:55.158 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T06:08:55.234 INFO:teuthology.run:pass